A mechanical arm rapid calibration method and system for intelligent cockpit flat and curved center control screen automatic test

Abstract

This invention discloses a rapid calibration method and system for a robotic arm used in automated testing of planar and curved central control screens in smart cockpits. The method comprises the following steps: S1 Establishing communication and controlling the central control screen to display a calibration grid pattern with known pixel coordinates; S2 Controlling the end effector of the robotic arm to sequentially contact calibration points, synchronously recording the pixel coordinates and corresponding 3D coordinates of each calibration point to form a calibration dataset D; S3 Based on the 3D coordinates in the calibration dataset D, using least squares plane fitting and residual variance analysis, automatically determining the screen type and constructing a pixel-3D mapping model F; S4 Verifying the mapping model and then using it for automated testing, thereby achieving the calibration of the vehicle's central control screen. This method obtains coordinates through physical contact, requiring no visual assistance, and can quickly adapt to planar and curved screens, solving the problems of existing technologies being susceptible to environmental interference and unable to uniformly adapt to curved screens.

Description

Technical Field

[0001] This invention relates to the fields of industrial automation and automotive testing technology, specifically to a rapid calibration method and system for robotic arms used for automated testing of planar and curved central control screens in smart cockpits. Background Technology

[0002] As smart cockpit interfaces become increasingly complex, the demand for automated testing of central control screens is growing. Among these methods, physical touch testing based on a six-axis robotic arm paired with a bionic hand (stylus) can realistically simulate human hand operation. This method boasts advantages such as small size and sensitive movement, and can be tested in a real vehicle environment, accurately replicating the user's operating experience. During the vehicle development phase, simulating real user touch operations with a robotic arm is a crucial step in ensuring a positive user experience. The core of this type of testing lies in establishing a high-precision mapping relationship between the two-dimensional pixel coordinate system of the central control screen and the three-dimensional spatial coordinate system of the robotic arm—a process known as "calibration."

[0003] In vehicle development, test bench resources are relatively abundant, while real vehicle resources are scarce. The six-axis robotic arm needs frequent disassembly and reassembly to avoid occupying real vehicle resources for extended periods. After each disassembly and reassembly, it is essential to ensure that previously entered and debugged test cases remain usable, and the installation process should not take too long, typically requiring completion within 3-5 minutes. In existing testing solutions, the host computer obtains the names, coordinates, and attributes of vehicle infotainment elements via ADB or similar methods, converting the screen's two-dimensional coordinates into the robotic arm's three-dimensional coordinates, thereby controlling the robotic arm to perform click, double-click, and swipe operations on the central control screen elements. Therefore, quickly and accurately calibrating and aligning the two-dimensional pixel coordinate system with the three-dimensional spatial coordinate system is a prerequisite for achieving efficient automated testing.

[0004] Current mainstream calibration methods typically employ vision assistance, using cameras to identify markers or features on the screen. However, the accuracy of visual calibration methods is easily affected by ambient lighting; light spots, reflections, or shadows on the screen can all lead to recognition errors, failing to provide the information guidance needed for millimeter-level precision operations by robotic arms. Furthermore, the installation and debugging process of vision systems is complex, making it difficult to meet the rapid deployment requirements of scenarios with frequent disassembly. For non-visual calibration methods, existing technologies often use the "four-point calibration method" to calculate the two-dimensional transformation matrix, suitable for calibrating planar central control screens. However, the four-point method relies on the strong assumption that the central control screen is an ideal plane, making it unsuitable for current mainstream 2.5D or 3D curved central control screens. On curved screens, the depth coordinates of points in the same pixel row or column vary in three-dimensional space; using a planar model will lead to touch deviations at edge positions, causing test failure.

[0005] Chinese patent document CN110405731A discloses a rapid dual-arm base coordinate system calibration method. This method involves offline planning of multiple sets of motion trajectories for the left and right robotic arms, controlling the left and right robotic arms to move according to the trajectories, and recording the transformation relationship from the end-effector coordinate system to the base coordinate system. Simultaneously, the left and right cameras respectively acquire images containing the calibration plate. Subsequently, the cameras are calibrated to obtain the transformation relationship from the calibration plate coordinate system to the left and right camera coordinate systems. The TSAI hand-eye calibration method is then used to calculate the hand-eye transformation relationship of the left and right robotic arms respectively. Finally, by combining multiple sets of calibration images and the transformation relationships between various coordinate systems, the least squares method is used to fit the transformation relationship between the left and right robotic arm base coordinate systems, thereby completing the calibration of the dual-arm base coordinate system. While the existing technology can calibrate the transformation relationship between the base coordinate systems of two robotic arms, it still has the following problems: the method relies on vision sensors and external calibration boards, the calibration accuracy is easily affected by ambient light, and it can only establish the coordinate system transformation relationship between the two robotic arms. It cannot solve the mapping problem between two-dimensional pixel coordinates and three-dimensional spatial coordinates between the robotic arm and the central control screen. Furthermore, it cannot be directly applied to the calibration problem of non-planar objects such as curved central control screens.

[0006] Therefore, it is necessary to develop a rapid calibration method and system for robotic arms that can automatically identify screen types and build corresponding mapping models without visual assistance, so as to simultaneously meet the automated testing needs of planar and curved central control screens. Summary of the Invention

[0007] The technical problem solved by this invention is to provide a rapid calibration method and system for robotic arms for automated testing of planar and curved central control screens in smart cockpits, so as to solve the technical problems that existing visual calibration methods are easily affected by environmental interference and cannot be uniformly adapted to planar and curved central control screens.

[0008] To address the aforementioned technical issues, the first aspect of this invention provides a rapid calibration method for a robotic arm used in the automated testing of planar and curved central control screens in intelligent cockpits, comprising the following steps:

[0009] S1: Establish communication between the host computer, the robotic arm control system, and the vehicle-mounted central control screen; control the vehicle-mounted central control screen to display a calibration grid pattern with known pixel coordinates;

[0010] S2: Control the end effector of the robotic arm to contact each calibration point in the calibration grid pattern in sequence, and simultaneously record the pixel coordinates of the vehicle central control screen corresponding to each calibration point and the three-dimensional coordinates of the end effector of the robotic arm, forming a calibration dataset D;

[0011] S3: Based on the three-dimensional coordinates of the end effector of the robotic arm in the calibration dataset D, the least squares plane fitting and residual variance analysis method are used to determine whether the vehicle central control screen is a flat screen or a curved screen, and the corresponding mapping model F of the vehicle central control screen is constructed.

[0012] S4: Verify the mapping model F of the vehicle central control screen. After confirming that it is correct, start automated testing to calibrate the vehicle central control screen.

[0013] Using the above technical solution, pixel coordinates of calibration points and 3D coordinates of the robotic arm's end effector are collected through physical contact to form a dataset. Based on this data, least squares plane fitting and residual variance analysis are used to automatically determine the screen type and construct a corresponding coordinate mapping model for subsequent automated testing. This method requires no visual assistance and can quickly adapt to both planar and curved screens. The use of least squares plane fitting and residual variance analysis can obtain the globally optimal solution, ensuring accuracy and consistency while avoiding local optima and selection bias.

[0014] Preferably, in step S1, the vehicle's central control screen is controlled via a communication protocol to display a calibration grid pattern with known pixel coordinates. The calibration grid pattern consists of M rows × N columns of equally spaced calibration points. Each calibration point has known pixel coordinates (u) precisely pre-set in the pixel coordinate system of the vehicle's central control screen. i , v i ), where i=1,2,...,K,K=M×N.

[0015] Preferably, the specific process of step S2 is as follows:

[0016] S21: Move the end effector of the control robot arm to the calibration point P. i At the preset position in front of the central control screen;

[0017] S22: Control the end effector of the robotic arm to slowly move in a preset direction and approach the vehicle's central control screen until the pressure sensor detects that the contact force reaches the set threshold F. th If so, it is determined that there has been precise physical contact with the surface of the vehicle's central control screen;

[0018] S23: When the physical contact occurs, it is recorded synchronously, and a set of coordinate mapping pairs is formed: ;where (u i ,v i ) are the known coordinates of the calibration point; The calibration dataset consists of the three-dimensional coordinates of the end effector at the calibration point in the robot arm's base coordinate system, read from the robot arm control system. .

[0019] Preferably, the method for determining whether the central control screen is a flat screen or a curved screen in step S3 is as follows:

[0020] S31: Extract the three-dimensional coordinates of the end effector of the vehicle-mounted robotic arm corresponding to all calibration points from the calibration dataset D, forming a set of robotic arm coordinate points. ;

[0021] S32: Using the least squares method, perform plane fitting on the set of coordinate points S of the robotic arm to obtain the plane equation parameters A, B, and C that minimize the sum of the squared distances from all calibration points to the plane. Then, based on the plane equation parameters A, B, and C, obtain the best-fit plane equation. The formula for the best-fit plane equation is: ;

[0022] Wherein, the coordinates (X, Y, Z) represent the absolute position of the end effector of the robotic arm in three-dimensional space when it contacts the calibration point on the vehicle's central control screen in step S2; the specific solution steps for the plane equation parameters A, B, and C are as follows:

[0023] For any one of the robotic arm coordinate points in the set S of robotic arm coordinate points , and Substitute into the best-fit plane equation The coordinates of the robotic arm are obtained. The predicted height on the fitted plane is given by the formula:

[0024] ;

[0025] The coordinate points of the robotic arm residual for:

[0026] ;

[0027] To eliminate the cancellation of positive and negative biases and to increase the penalty for larger biases, a sum of squared residuals is constructed for all K points:

[0028] ;

[0029] Where Q represents the objective function, which is the sum of squared residuals of all robot arm coordinate points; K represents the total number of robot arm coordinate points; and i represents the i-th robot arm coordinate point.

[0030] To minimize Q(A,B,C), we need to take the partial derivatives with respect to parameters A, B, and C, and then set the partial derivatives equal to 0. The formula is as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] After expansion, differentiation, simplification, and rearrangement, we obtain the following system of linear equations, with the following formula:

[0035] ;

[0036] S33: For each calibration point P in the set of coordinate points S of the robotic arm i (X i ,Y i Z i ), calculate the calibration point P i The vertical distance d to the best-fit plane i The residual at the calibration point is calculated using the following formula:

[0037] ;

[0038] Based on the calibration point P i vertical distance d i Calculate the calibration point P i The statistical variance Var(Distance) is used to quantify the dispersion or fluctuation of all contact points relative to the fitted plane.

[0039] The formula for calculating the statistical variance Var(Distance) is:

[0040] ;

[0041] Where K is the total number of calibration points; d i Let P be the i-th calibration point. i The vertical distance to the best-fit plane (i.e., the residual); For all d i The average value is calculated using the following formula:

[0042] ;

[0043] S34: Set the plane determination threshold The statistical variance Var(Distance) is compared with the plane decision threshold. The comparison is performed, and the decision logic is executed. The decision logic is as follows:

[0044] like If the spatial positions of all calibration points are closely attached to the best-fit plane, the surface of the vehicle central control screen is geometrically represented as a plane, and the vehicle central control screen is determined to be a flat screen.

[0045] like If the curvature of the point cloud data is significantly different, it indicates that all calibration points cannot be effectively contained by a single planar model, and the surface of the vehicle central control screen is a complex curved surface that is not planar. In this case, the vehicle central control screen is determined to be a curved screen.

[0046] Preferably, if step S34 determines that the in-vehicle central control screen is a flat screen, a flat screen mapping model F is constructed. flat ;

[0047] For a flat screen, pixel coordinates are linearly related to the physical coordinates (X, Y) in the plane, while depth coordinates... The planar screen mapping model F is determined by the physical coordinates (X, Y) through the plane equation; flat The horizontal position transformation matrix M and the depth coordinates The best-fitting plane equation Z is composed of the following formula:

[0048] ;

[0049] ;

[0050] ;

[0051] Where, m 11 ,…,m 23 For affine transformation parameters, u and v represent two-dimensional pixel coordinates in the pixel coordinate system of the vehicle central control screen. u represents the index of the pixel in the horizontal direction (column), v represents the index of the pixel in the vertical direction (row), and A, B, and C are the optimal plane equation parameters obtained by fitting in step S32.

[0052] It is important to note the depth plane equation. The plane equation is obtained by fitting the robot arm calibration points using the least squares method. This plane is the "best-fit plane" (i.e., the plane with the smallest deviation from all calibration points). Therefore, the "depth plane equation" and the "best-fit plane equation" are consistent here, with the letter Z representing the depth (vertical height) of the plane, and the terminology is consistent.

[0053] Using K sets of data in the calibration dataset D The horizontal position transformation can be expressed as a system of linear equations, specifically:

[0054] ;

[0055] in, This represents the horizontal coordinate of the k-th calibration point in the pixel coordinate system of the vehicle's central control screen; This represents the vertical coordinate of the k-th calibration point in the pixel coordinate system of the vehicle's central control screen; This represents the three-dimensional physical coordinates (X and Y components) of the end effector of the robotic arm corresponding to the kth calibration point in the base coordinate system of the robotic arm.

[0056] Solving the above system of linear equations using the least squares method yields the parameter estimates for the transformation matrix M. Then, by combining the optimal plane equation parameters A, B, C obtained from step S32, the plane screen mapping model F is finally obtained. flat The formula is:

[0057] ;

[0058] Where T represents the matrix transpose operation.

[0059] Preferably, if step S34 determines that the in-vehicle central control screen is a curved screen, a curved screen mapping model F is constructed. curved The specific steps are as follows:

[0060] First, a bivariate quadratic polynomial surface is used to approximate the spatial shape of the screen surface, and the fitted X, Y, and Z coordinate axes are obtained, namely the X coordinate, Y coordinate, and Z coordinate; u represents the index of the pixel in the horizontal direction (column), and v represents the index of the pixel in the vertical direction (row).

[0061] The fitting formula for the X-coordinate is:

[0062] ;

[0063] Where, p 00 The constant term of the X-coordinate polynomial (the X value when u=0, v=0, reflecting the reference position of the screen in the X direction).

[0064] p 10 The coefficient of the first-order term of the X coordinate with respect to u (controls the rate at which X changes linearly with u, reflecting the horizontal tilt or translation in the X direction);

[0065] p 01 The coefficient of the first-order term of the X coordinate with respect to v (controls the rate at which X changes linearly with v, reflecting the vertical tilt or translation in the X direction).

[0066] p 20 Indicates the X coordinate with respect to u 2 The quadratic coefficient (controls the curvature of X as a function of u, reflecting the degree of horizontal curvature in the X direction).

[0067] p 11 The cross term coefficient of the X coordinate with respect to u and v (controls the degree of coupling of X with the interaction of u and v, reflecting the twisting or asymmetric bending in the X direction).

[0068] p 02 This indicates that the X coordinate is related to v. 2 The quadratic coefficient (controls the curvature of X as a function of v, reflecting the degree of vertical bending in the X direction).

[0069] The fitting formula for the Y-coordinate is:

[0070] ;

[0071] q 00 The constant term of the Y-coordinate polynomial (the Y value when u=0, v=0, reflecting the reference position of the screen in the Y direction).

[0072] q 10 The coefficient of the first term of the Y-coordinate with respect to u (controls the rate at which Y changes linearly with u, reflecting the horizontal tilt or translation in the Y direction);

[0073] q 01 It represents the coefficient of the first term of the Y coordinate with respect to v (controlling the rate of linear change of Y with v, reflecting the vertical tilt or translation in the Y direction);

[0074] q 20 Indicates the Y coordinate with respect to u 2 The quadratic coefficient (controls the curvature of Y as u changes quadratically, reflecting the degree of horizontal curvature in the Y direction);

[0075] q 11 The cross term coefficient of the Y coordinate with respect to u and v (controls the degree of coupling of Y with the interaction of u and v, reflecting the twisting or asymmetric bending in the Y direction);

[0076] q 02 Indicates the Y-coordinate with respect to v 2 The quadratic coefficient (controls the curvature of Y as it changes quadratically with v, reflecting the degree of vertical bending in the Y direction).

[0077] The fitting formula for the Z-coordinate is:

[0078] ;

[0079] r 00 The constant term of the Z-coordinate polynomial (the Z value when u=0, v=0, reflecting the reference position of the screen in the Z direction);

[0080] r 10 The coefficient of the linear term of the Z coordinate with respect to u (controls the rate at which Z changes linearly with u, reflecting the horizontal tilt or translation in the Z direction);

[0081] r 01It represents the coefficient of the first-order term of the Z-coordinate with respect to v (controlling the rate of linear change of Z with v, reflecting the vertical tilt or translation in the Z direction);

[0082] r 20 The Z-coordinate is represented by u. 2 The quadratic coefficient (controls the curvature of Z as a function of u, reflecting the degree of horizontal curvature in the Z direction);

[0083] r 11 The cross term coefficient of the Z coordinate with respect to u and v (controls the degree of coupling of Z with the interaction of u and v, reflecting the twisting or asymmetric bending in the Z direction);

[0084] r 02 The Z-coordinate with respect to v 2 The quadratic coefficient (controls the curvature of Z as a function of v, reflecting the degree of vertical bending in the Y direction).

[0085] For each calibration point P in the calibration dataset D i Construct feature vectors ;(u i ,v i () indicates that each calibration point is a known pixel coordinate that has been precisely set in the pixel coordinate system of the vehicle's central control screen, where i=1,2,...,K, K=M×N;

[0086] Then, for the three coordinate axes X, Y, and Z, the least squares equations for the X, Y, and Z coordinates are constructed respectively.

[0087] For example, the least squares equation formed by the X-axis is:

[0088] ;

[0089] Similarly, construct equations for the Y coordinate (parameter q) and Z coordinate (parameter r) respectively;

[0090] That is, the least squares solution is performed on the three equations corresponding to the X, Y, and Z coordinate axes respectively to obtain the corresponding parameter vectors. ; From all K calibration points P in the calibration dataset D i eigenvectors The unified feature matrix Φ, formed by vertically concatenating elements:

[0091] ;

[0092] The parameter vector of the X coordinate The expression is:

[0093] ;

[0094] Among them, X1, X2...X k These are the fitted X-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively.

[0095] The parameter vector of the Y coordinate The expression is:

[0096] ;

[0097] Among them, Y1, Y2...Y k These are the fitted Y-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively.

[0098] The parameter vector of the Z-coordinate The expression is:

[0099] ;

[0100] Among them, Z1, Z2...Z k These are the fitted Z-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively.

[0101] Finally, the curved screen mapping model F is obtained. curved The expression is:

[0102] ;

[0103] Where T represents the matrix transpose operation; p is the degree of u in the bivariate quadratic polynomial, and q is the degree of v in the bivariate quadratic polynomial, with values ​​ranging from 0 to p and q to 2, resulting in 6 combinations and 18 parameters; u p The term representing the power of pixel coordinate u, such as u 0 =1,u 1 =u,u 2 =u 2 ;v q This represents the power of the pixel coordinate v, such as v 0 =1,v 1 =v,v 2 =v 2 Used to construct polynomial basis functions.

[0104] Preferably, the specific steps of verifying the mapping model F in step S4 are as follows:

[0105] The vehicle's infotainment system displays a new test point P that was not involved in model training on the in-vehicle central control screen. test Get its pixel coordinates (u t ,v t ), input it into the vehicle central control screen mapping model F, and obtain the predicted coordinates (X). p ,Y p Z pThen, control the robotic arm to move and touch the test point, and verify the accuracy through the actual touch position feedback from the vehicle system.

[0106] The second aspect of this invention provides a rapid calibration system for robotic arms for automated testing of planar and curved central control screens in smart cockpits, which solves the problems of poor environmental adaptability, inability to uniformly adapt to planar and curved screens, and low deployment efficiency of existing visual calibration systems.

[0107] To address the aforementioned technical issues, the present invention provides the following solution: a rapid calibration system for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits, comprising a host computer, a robotic arm, an in-vehicle central control screen, a camera, and a server. The host computer is connected to the robotic arm, the in-vehicle central control screen, the camera, and the server. The camera records the display status of the in-vehicle central control screen and dashboard, as well as the robotic arm's operation process, in real time. The server stores calibration datasets, mapping model parameters, and test results, and supports the retrospective analysis of historical calibration data. The host computer, as the control core, executes the rapid calibration method for the robotic arm described in the first aspect of the present invention, completing the entire process of calibration point generation, data acquisition, determination of planar or curved screens, construction of mapping models, and accuracy verification.

[0108] Using the above technical solution, the host computer acts as the control core, achieving unified scheduling of the calibration process through communication connections established with the robotic arm, the vehicle-mounted central control screen, the camera, and the server. Specifically, the connection with the robotic arm is used to send motion commands and receive real-time coordinate feedback from the end effector; the connection with the vehicle-mounted central control screen (e.g., via the ADB protocol) is used to control the screen to display calibration patterns with known pixel coordinates and acquire UI element information; the camera is configured to record the operation process in real time for monitoring and traceability; and the server provides computing power support for the storage and processing of calibration data. Through the collaborative work of these components, the system can automatically construct a mapping model from the pixel coordinates of the calibration point to the three-dimensional coordinates of the robotic arm based on coordinate data acquired through physical contact, thus providing a precise coordinate transformation basis for subsequent automated testing.

[0109] Preferably, the robotic arm includes a bionic hand and a pressure sensor. The bionic hand is the end effector of the robotic arm, which performs operations on the vehicle's central control screen. The pressure sensor is installed on the bionic hand and integrates a touch-pressure feedback function to identify whether the robotic arm contacts the vehicle's central control screen when it moves toward it, and to identify the force used by the robotic arm when it touches the central control screen.

[0110] Preferably, the host computer includes a calibration process control module, a data acquisition and processing module, a mapping model construction module, and a coordinate transformation and testing engine;

[0111] The calibration process control module is used to coordinate the communication between the host computer, the robotic arm control system and the vehicle-mounted central control screen. Based on the preset calibration points and the approximate spatial position of the vehicle-mounted central control screen, it plans the motion path of the end effector of the robotic arm.

[0112] The data acquisition and processing module is used to simultaneously capture the theoretical pixel coordinates of the current calibration point and the real-time Cartesian coordinates of the end effector in the base coordinate system fed back by the robotic arm control system. It performs real-time verification of the data, removes outliers caused by robotic arm jitter, communication delay or abnormal touch, integrates the valid data into a calibration dataset D, and manages it.

[0113] The mapping model construction module is used to automatically determine the type of the vehicle central control screen, and select either the planar screen mapping model (linear model) or the curved screen mapping model (polynomial model) according to the determination result. The least squares method is used to fit and train the calibration dataset D to obtain the parameters of the mapping model (function) F.

[0114] The coordinate transformation and testing engine is used in the automated testing phase to call the mapping model (function) F to convert UI coordinates into robotic arm coordinates and execute the test sequence.

[0115] Compared with the prior art, the beneficial effects of the present invention are:

[0116] (1) Strong versatility: The same method and system, the system can be automatically calibrated, and the model can be automatically selected to adapt to curved screens of any curvature and various flat screens, which solves the core calibration problem of automated testing of curved central control screens;

[0117] (2) High precision and robustness: Calibration is achieved through physical contact triggering and known coordinate points, avoiding visual recognition errors. The coordinate source is accurate, and the three-dimensional spatial surface is accurately fitted through a high-order polynomial model, resulting in high mapping accuracy.

[0118] (3) Fully automated and highly efficient: The calibration process is highly automated and can be completed in one go, which significantly improves the deployment and changeover efficiency of the test system;

[0119] (4) The principle is clear and easy to implement: The model is based on the classic least squares regression method, which is stable and reliable in calculation. It does not require complex iterative optimization and is easy to implement on industrial controllers. Attached Figure Description

[0120] Figure 1 This is a flowchart illustrating the rapid calibration method for robotic arms used in the automated testing of planar and curved central control screens in intelligent cockpits according to the present invention.

[0121] Figure 2This is a schematic diagram of the overall structure of the robotic arm rapid calibration system for automated testing of planar and curved central control screens in intelligent cockpits according to the present invention.

[0122] Figure 3 This is a schematic diagram of the host computer module of the robotic arm rapid calibration system for automated testing of planar and curved central control screens in intelligent cockpits, as described in this invention. Detailed Implementation

[0123] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings.

[0124] Example: Figure 1 As shown, the rapid calibration method for a robotic arm used for automated testing of planar and curved central control screens in smart cockpits includes the following steps:

[0125] S1: Establish communication between the host computer, the robotic arm control system, and the vehicle-mounted central control screen; control the vehicle-mounted central control screen to display a calibration grid pattern with known pixel coordinates;

[0126] The method for controlling the display of a calibration grid pattern with known pixel coordinates on the vehicle's central control screen in step S1 is as follows: Through a communication protocol (such as ADB), the vehicle's central control screen is controlled to display a known calibration grid pattern covering the entire screen. The calibration grid pattern consists of M rows × N columns of equally spaced calibration points. Each calibration point has a known pixel coordinate (u) precisely set in the pixel coordinate system of the vehicle's central control screen. i ,v i ), where i=1,2,...,K,K=M×N.

[0127] S2: Control the end effector of the robotic arm to sequentially contact each calibration point in the calibration grid pattern, and simultaneously record the pixel coordinates of the vehicle's central control screen corresponding to each calibration point and the three-dimensional coordinates of the corresponding end effector of the robotic arm, forming a calibration dataset D; the specific process is as follows:

[0128] S21: Move the end effector of the control robot arm to the calibration point P. i The preset position in front of the vehicle's central control screen;

[0129] S22: Control the end effector of the robotic arm to slowly move along a preset direction (such as perpendicular to the approximate plane of the central control screen) and approach the vehicle's central control screen until the pressure sensor detects that the contact force reaches the set threshold F. th If so, it is determined that precise physical contact has occurred with the surface of the vehicle's central control screen;

[0130] S23: When the physical contact occurs, record synchronously and form a set of coordinate mapping pairs: ;where (u i,v i ) are the known coordinates of the calibration point; The calibration dataset consists of the three-dimensional coordinates of the end effector of the robotic arm at the calibration point in the base coordinate system of the robotic arm, read from the control system of the robotic arm; thus forming the calibration dataset. .

[0131] S3: Based on the three-dimensional coordinates of the end effector of the robotic arm in the calibration dataset D, the least squares plane fitting and residual variance analysis method are used to determine whether the vehicle central control screen is a flat screen or a curved screen, and the corresponding mapping model F of the vehicle central control screen is constructed.

[0132] The method for determining whether the central control screen is a flat screen or a curved screen in step S3 is as follows:

[0133] S31: Extract the three-dimensional coordinates of the end effector of the robotic arm corresponding to all calibration points from the calibration dataset D, forming a set of robotic arm coordinate points. ;

[0134] S32: Using the least squares method, perform plane fitting on the set of coordinate points S of the robotic arm to obtain the plane equation parameters A, B, and C that minimize the sum of the squared distances from all the calibration points to the plane. Then, based on the plane equation parameters A, B, and C, obtain the best-fit plane equation. The formula for the best-fit plane equation is:

[0135] ;

[0136] Wherein, the coordinates (X, Y, Z) represent the absolute position of the end effector of the robotic arm in three-dimensional space when it contacts the calibration point on the central control screen in step S2; the specific solution steps for the plane equation parameters A, B, and C are as follows:

[0137] For any one of the robotic arm coordinate points in the set S of robotic arm coordinate points , and Substitute into the best-fit plane equation The coordinates of the robotic arm are obtained. The predicted height on the fitted plane is given by the formula:

[0138] ;

[0139] The coordinate points of the robotic arm residual for:

[0140] ;

[0141] To eliminate the cancellation of positive and negative deviations and to increase the penalty for larger deviations, the sum of squared residuals (objective function Q) of all robot arm coordinate points is constructed, with the following formula:

[0142] ;

[0143] Where Q represents the objective function, which is the sum of squared residuals of all robot arm coordinate points; K represents the total number of robot arm coordinate points; and i represents the i-th robot arm coordinate point.

[0144] To minimize Q(A,B,C), we take the partial derivatives of the plane equation with respect to the parameters A, B, and C, and set the results of the partial derivatives equal to 0. The formula is as follows:

[0145] ;

[0146] ;

[0147] ;

[0148] Expanding and differentiating separately, we get:

[0149] ;

[0150] ;

[0151] ;

[0152] Where i represents the coordinate point of the i-th robotic arm; K represents the sum of the coordinate points of all robotic arms;

[0153] After simplification and rearranging terms, we obtain the following system of linear equations, with the following formula:

[0154] ;

[0155] S33: For each calibration point P in the set of coordinate points S of the robotic arm i (X i ,Y i Z i ), calculate the calibration point P i The vertical distance d to the best-fit plane i The residual at the calibration point is calculated using the following formula:

[0156] ;

[0157] Based on the calibration point P i vertical distance d i Calculate the calibration point P iThe statistical variance Var(Distance) is used to quantify the dispersion or fluctuation of all contact points relative to the fitted plane.

[0158] The formula for calculating the statistical variance Var(Distance) is:

[0159] ;

[0160] Where K is the total number of calibration points; d i Let P be the i-th calibration point. i The vertical distance to the best-fit plane (i.e., the residual); For all d i The average value is calculated using the following formula:

[0161] ;

[0162] S34: Set the plane determination threshold The statistical variance Var(Distance) is compared with the plane decision threshold. The comparison is performed, and the decision logic is executed. The decision logic is as follows:

[0163] like If the spatial positions of all calibration points are closely attached to the best-fit plane, the surface of the vehicle central control screen is geometrically represented as a plane, regardless of whether the plane is horizontal or inclined relative to the horizontal plane. At this time, the vehicle central control screen is determined to be a flat screen.

[0164] like If the curvature of the point cloud data is significantly different, it indicates that all calibration points cannot be effectively contained by a single planar model, and the surface of the vehicle central control screen is a complex curved surface that is not planar. In this case, the vehicle central control screen is determined to be a curved screen.

[0165] The method for constructing the corresponding in-vehicle central control screen mapping model F in step S3 is as follows:

[0166] If step S34 determines that the vehicle central control screen is a flat screen, a flat screen mapping model F is constructed. flat The specific steps are as follows:

[0167] For a flat screen, pixel coordinates are linearly related to the physical coordinates (X, Y) in the plane, while depth coordinates... The plane is determined by physical coordinates (X, Y) through plane equations; the screen mapping model F of the plane is... flat The horizontal position transformation matrix M and the depth coordinates The best-fit plane equation Z consists of:

[0168] ;

[0169] ;

[0170] ;

[0171] Where, m 11 ,…,m 23 For affine transformation parameters, u and v represent two-dimensional pixel coordinates in the pixel coordinate system of the vehicle central control screen. u represents the index of the pixel in the horizontal direction (column), v represents the index of the pixel in the vertical direction (row), and A, B, C are the optimal plane equation parameters obtained by fitting in step S32.

[0172] Using K sets of data in the calibration dataset D The horizontal position transformation can be expressed as a system of linear equations, specifically:

[0173] ;

[0174] in, This represents the horizontal coordinate of the k-th calibration point in the pixel coordinate system of the vehicle's central control screen; This represents the vertical coordinate of the k-th calibration point in the pixel coordinate system of the vehicle's central control screen; This represents the three-dimensional physical coordinates (X and Y components) of the end effector of the robotic arm corresponding to the kth calibration point in the base coordinate system of the robotic arm.

[0175] Solving the above system of linear equations using the least squares method yields the parameter estimates for the transformation matrix M. Then, by combining the optimal plane equation parameters A, B, C obtained from step S32, the plane screen mapping model F is finally obtained. flat The formula is:

[0176] ;

[0177] Where T represents the matrix transpose operation;

[0178] The method for constructing the corresponding in-vehicle central control screen mapping model F in step S3 is as follows:

[0179] If step S34 determines that the vehicle central control screen is a curved screen, a curved screen mapping model F is constructed. curved The specific steps are as follows:

[0180] A binary quadratic polynomial surface is used to approximate the spatial shape of the screen surface, and the fitted X, Y, and Z coordinate axes are obtained, namely the X coordinate, Y coordinate, and Z coordinate; u represents the index of the pixel in the horizontal direction (column), and v represents the index of the pixel in the vertical direction (row).

[0181] The fitting formula for the X-coordinate is:

[0182] ;

[0183] Where, p 00 The constant term of the X-coordinate polynomial (the X value when u=0, v=0, reflecting the reference position of the screen in the X direction).

[0184] p 10 The coefficient of the first-order term of the X coordinate with respect to u (controls the rate at which X changes linearly with u, reflecting the horizontal tilt or translation in the X direction);

[0185] p 01 The coefficient of the first-order term of the X coordinate with respect to v (controls the rate at which X changes linearly with v, reflecting the vertical tilt or translation in the X direction).

[0186] p 20 Indicates the X coordinate with respect to u 2 The quadratic coefficient (controls the curvature of X as a function of u, reflecting the degree of horizontal curvature in the X direction).

[0187] p 11 The cross term coefficient of the X coordinate with respect to u and v (controls the degree of coupling of X with the interaction of u and v, reflecting the twisting or asymmetric bending in the X direction).

[0188] p 02 This indicates that the X coordinate is related to v. 2 The quadratic coefficient (controls the curvature of X as a function of v, reflecting the degree of vertical bending in the X direction).

[0189] The fitting formula for the Y-coordinate is:

[0190] ;

[0191] q 00 The constant term of the Y-coordinate polynomial (the Y value when u=0, v=0, reflecting the reference position of the screen in the Y direction).

[0192] q 10 The coefficient of the first term of the Y-coordinate with respect to u (controls the rate at which Y changes linearly with u, reflecting the horizontal tilt or translation in the Y direction);

[0193] q 01 It represents the coefficient of the first term of the Y coordinate with respect to v (controlling the rate of linear change of Y with v, reflecting the vertical tilt or translation in the Y direction);

[0194] q 20 Indicates the Y coordinate with respect to u 2 The quadratic coefficient (controls the curvature of Y as u changes quadratically, reflecting the degree of horizontal curvature in the Y direction);

[0195] q 11 The cross term coefficient of the Y coordinate with respect to u and v (controls the degree of coupling of Y with the interaction of u and v, reflecting the twisting or asymmetric bending in the Y direction);

[0196] q 02 Indicates the Y-coordinate with respect to v 2 The quadratic coefficient (controls the curvature of Y as it changes quadratically with v, reflecting the degree of vertical bending in the Y direction).

[0197] The fitting formula for the Z-coordinate is:

[0198] ;

[0199] r 00 The constant term of the Z-coordinate polynomial (the Z value when u=0, v=0, reflecting the reference position of the screen in the Z direction);

[0200] r 10 The coefficient of the linear term of the Z coordinate with respect to u (controls the rate at which Z changes linearly with u, reflecting the horizontal tilt or translation in the Z direction);

[0201] r 01 It represents the coefficient of the first-order term of the Z-coordinate with respect to v (controlling the rate of linear change of Z with v, reflecting the vertical tilt or translation in the Z direction);

[0202] r 20 The Z-coordinate is represented by u. 2 The quadratic coefficient (controls the curvature of Z as a function of u, reflecting the degree of horizontal curvature in the Z direction);

[0203] r 11 The cross term coefficient of the Z coordinate with respect to u and v (controls the degree of coupling of Z with the interaction of u and v, reflecting the twisting or asymmetric bending in the Z direction);

[0204] r 02 The Z-coordinate with respect to v 2 The quadratic coefficient (controls the curvature of Z as a function of v, reflecting the degree of vertical bending in the Y direction).

[0205] Then, for each calibration point in the calibration dataset D, construct a feature vector. ;

[0206] Then, for the three coordinate axes X, Y, and Z, we construct the least squares equations for the X, Y, and Z coordinates respectively; for example, the least squares equation for the X coordinate axis is:

[0207]

[0208] Similarly, construct equations for the Y coordinate (parameter q) and Z coordinate (parameter r) respectively;

[0209] That is, the least squares solution is performed on the three equations corresponding to the X, Y, and Z coordinate axes respectively to obtain the corresponding parameter vectors. ; From all K calibration points P in the calibration dataset D i eigenvectors The unified feature matrix (design matrix) Φ, formed by vertically concatenating elements:

[0210] ;

[0211] The parameter vector of the X coordinate The expression is:

[0212] ;

[0213] Among them, X1, X2...X k These are the fitted X-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively.

[0214] The parameter vector of the Y coordinate The expression is:

[0215] ;

[0216] Among them, Y1, Y2...Y k These are the fitted Y-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively.

[0217] The parameter vector of the Z-coordinate The expression is:

[0218] ;

[0219] Among them, Z1, Z2...Z k These are the fitted Z-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively.

[0220] The final curved screen mapping model Fcurved is obtained, and the formula is:

[0221] ;

[0222] Where T represents the matrix transpose operation; p is the degree of u in the bivariate quadratic polynomial, and q is the degree of v in the bivariate quadratic polynomial, with values ​​ranging from 0 to p and q to 2, resulting in 6 combinations and 18 parameters; u p The term representing the power of pixel coordinate u, such as u 0 =1, u 1 =u,u 2 =u 2 ;vq This represents the power of the pixel coordinate v, such as v 0 =1, v 1 =v,v 2 =v 2 Used to construct polynomial basis functions.

[0223] S4: Verify the mapping model F of the vehicle central control screen. After confirming that it is correct, start automated testing to calibrate the vehicle central control screen. The method for verifying the mapping model F in step S4 is as follows:

[0224] The vehicle's infotainment system displays a new test point P that was not involved in model training on the in-vehicle central control screen. test Get its pixel coordinates (u t ,v t ), where t represents the test point P. test , pixel coordinates (u t ,v t Input the mapping model F of the vehicle central control screen to obtain the predicted coordinates (X). p ,Y p Z p Then, the robotic arm is controlled to move and contact the test point. The accuracy is verified by the actual touch position fed back by the vehicle's infotainment system. Here, the accuracy is verified by comparing whether the predicted coordinates and the actual touch position are consistent. During verification, the predicted coordinates (X, Y, Z) and the robotic arm's three-dimensional coordinates (X, Y, Z) output by the model are compared. p ,Y p Z p The actual touch position (the real three-dimensional coordinates (X, Y) fed back by the vehicle system after the robotic arm touches the test point) is compared with the actual touch position. a ,Y a Z a The commonly used error calculation method is to quantize the positional deviation in three-dimensional space using Euclidean distance, and the formula is:

[0225] ;

[0226] The result of "meeting the accuracy requirement" is: the error is less than the preset accuracy threshold (Error≤). Threshold It needs to be set according to the test requirements (such as industrial-grade touch testing usually requires an error of ≤0.5mm, or it can be adjusted according to the screen size and test scenario).

[0227] like Figure 2As shown, the rapid calibration system for robotic arms used for automated testing of planar and curved central control screens in intelligent cockpits includes a host computer, a robotic arm, an in-vehicle central control screen, a camera, and a server. The host computer is connected to the robotic arm, the in-vehicle central control screen, the camera, and the server. The camera is used to record the display status of the in-vehicle central control screen and the dashboard, as well as the operation process of the robotic arm, in real time. The server is used to store calibration datasets, mapping model parameters, and test results, and supports the retrospective analysis of historical calibration data. The host computer is used to execute the rapid calibration method for the robotic arm, completing the entire process of calibration point generation, data acquisition, determination of planar or curved screens, construction of mapping models, and accuracy verification.

[0228] The robotic arm includes a bionic hand and a pressure sensor. The bionic hand is the end effector of the robotic arm, which performs operations on the vehicle's central control screen. The pressure sensor is installed on the bionic hand and integrates a tactile feedback function to identify whether the robotic arm touches the screen of the vehicle's central control screen when it moves towards it, and to identify the force used by the robotic arm when it touches the screen.

[0229] like Figure 3 As shown, the host computer includes a calibration process control module, a data acquisition and processing module, a mapping model construction module, and a coordinate transformation and testing engine. The calibration process control module coordinates the communication between the host computer, the robotic arm control system, and the vehicle-mounted central control screen. Based on preset calibration points and the approximate spatial position of the central control screen, it plans the motion path of the robotic arm's end effector. The data acquisition and processing module simultaneously captures the theoretical pixel coordinates of the current calibration point and the real-time Cartesian coordinates of the end effector in the base coordinate system fed back by the robotic arm control system. It performs real-time verification of the data, eliminates outliers caused by robotic arm jitter, communication delays, or abnormal touches, integrates the valid data into a calibration dataset D, and manages it. The mapping model construction module automatically determines the type of the central control screen and selects either a planar linear model or a surface polynomial model based on the determination result. It uses the least squares method to fit and train the calibration dataset D to obtain the parameters of the mapping model (function) F. The coordinate transformation and testing engine calls the mapping model (function) F during the automated testing phase to convert UI coordinates into robotic arm coordinates and execute test sequences.

[0230] For those skilled in the art, the specific embodiments are merely illustrative descriptions of the present invention. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A rapid calibration method for a robotic arm used for automated testing of planar and curved surface central control screens in intelligent cockpits, characterized in that, Includes the following steps: S1: Establish communication between the host computer, the robotic arm control system, and the vehicle-mounted central control screen; control the vehicle-mounted central control screen to display a calibration grid pattern with known pixel coordinates; S2: Control the end effector of the robotic arm to contact each calibration point in the calibration grid pattern in sequence, and simultaneously record the pixel coordinates of the vehicle central control screen corresponding to each calibration point and the three-dimensional coordinates of the end effector of the robotic arm, thereby forming a calibration dataset D; S3: Based on the three-dimensional coordinates of the end effector of the robotic arm in the calibration dataset D, the least squares plane fitting and residual variance analysis method are used to determine whether the vehicle central control screen is a planar screen or a curved screen, and the corresponding mapping model F of the vehicle central control screen is constructed. S4: Verify the mapping model F of the vehicle central control screen. After confirming that it is correct, start automated testing to calibrate the vehicle central control screen.

2. The rapid calibration method for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits according to claim 1, characterized in that, In step S1, the vehicle's central control screen is controlled via a communication protocol to display a calibration grid pattern with known pixel coordinates. The calibration grid pattern consists of M rows × N columns of equally spaced calibration points, each of which has pre-defined known pixel coordinates (u) in the pixel coordinate system of the vehicle's central control screen. i ,v i ), where i=1,2,...,K,K=M×N.

3. The rapid calibration method for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits according to claim 2, characterized in that, The specific process of step S2 is as follows: S21: Control the end effector of the robotic arm to move to the calibration point P. i At a preset position in front of the vehicle's central control screen; S22: Control the end effector of the robotic arm to move along a preset direction and approach the vehicle's central control screen until the pressure sensor detects that the contact force reaches the set threshold F. th If so, it is determined that there has been physical contact with the surface of the vehicle's central control screen; S23: When physical contact occurs, record synchronously and form a set of coordinate mapping pairs: ;where (u i ,v i ) are the known pixel coordinates of the calibration point; The calibration dataset consists of the three-dimensional coordinates of the end effector at the calibration point in the robot arm's base coordinate system, read from the robot arm control system. .

4. The rapid calibration method for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits according to claim 3, characterized in that, The specific steps for determining whether the vehicle's central control screen is a flat screen or a curved screen in step S3 are as follows: S31: Extract the three-dimensional coordinates of the end effector of the robotic arm corresponding to all calibration points from the calibration dataset D, forming a set of robotic arm coordinate points. ; S32: Using the least squares method, perform plane fitting on the set of coordinate points S of the robotic arm to obtain the plane equation parameters A, B, and C that minimize the sum of the squared distances from all calibration points to the plane. Then, based on the plane equation parameters A, B, and C, obtain the best-fit plane equation. The formula for the best-fit plane equation is: ； Wherein, the coordinates (X,Y,Z) represent the absolute position of the end effector of the robotic arm in three-dimensional space when the end effector of the robotic arm in step S2 contacts the calibration point on the vehicle central control screen. The specific steps for solving the plane equation parameters A, B, and C are as follows: For any one of the robotic arm coordinate points in the set S of robotic arm coordinate points , and Substitute into the best-fit plane equation The coordinates of the robotic arm are obtained. The predicted height on the fitted plane is given by the formula: ； The coordinate points of the robotic arm residual for: ； Simultaneously, the sum of squared residuals of all robotic arm coordinate points is constructed, using the following formula: ； Where Q represents the objective function, which is the sum of squared residuals of all robot arm coordinate points; K represents the total number of robot arm coordinate points; and i represents the i-th robot arm coordinate point. In order to To obtain the minimum value, take the partial derivatives of the plane equation with respect to the parameters A, B, and C, and set the partial derivatives equal to 0. The formula is: ； ； ； After expanding, differentiating, simplifying, and rearranging terms, we obtain the system of linear equations, as follows: ； S33: For each calibration point P in the set of coordinate points S of the robotic arm i (X i ,Y i Z i ), calculate the calibration point P i vertical distance d to the best-fit plane i The residual at the calibration point is calculated using the following formula: ； Based on the calibration point P i vertical distance d i Calculate the calibration point P i The statistical variance Var(Distance) is used to quantify the dispersion or fluctuation of all contact points relative to the fitted plane. S34: Set the plane determination threshold The statistical variance Var(Distance) is compared with the plane decision threshold. The comparison is performed, and the decision logic is executed. The decision logic is as follows: like If the spatial positions of all calibration points are closely attached to the near-best-fit plane, the surface of the vehicle central control screen is geometrically represented as a plane, and the vehicle central control screen is determined to be a planar screen. like If the curvature of the point cloud data is significantly different, it indicates that all calibration points cannot be effectively contained by a single planar model, and the surface of the vehicle central control screen is a complex curved surface that is not planar. In this case, the vehicle central control screen is determined to be a curved screen.

5. The rapid calibration method for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits according to claim 4, characterized in that, If in step S34 it is determined that the in-vehicle central control screen is a flat screen, then a flat screen mapping model F is constructed. flat The specific steps are as follows: For a flat screen, pixel coordinates are linearly related to the physical coordinates (X, Y) in the plane, while depth coordinates... The planar screen is determined by physical coordinates (X, Y) through a planar equation; the mapping model F of the planar screen flat The horizontal position transformation matrix M and the depth coordinates The best-fitting plane equation Z is composed of the following formula: ； ； ； Where, m 11 ,…,m 23 For affine transformation parameters, u and v represent two-dimensional pixel coordinates in the pixel coordinate system of the vehicle central control screen, u represents the index of the pixel in the horizontal direction, v represents the index of the pixel in the vertical direction, and A, B, and C are the optimal plane equation parameters obtained by fitting in step S32. Using K sets of data in the calibration dataset D The horizontal position transformation can be expressed as a system of linear equations, specifically: ； in, This represents the horizontal coordinate of the k-th calibration point in the pixel coordinate system of the vehicle's central control screen; This represents the vertical coordinate of the k-th calibration point in the pixel coordinate system of the vehicle's central control screen; This represents the three-dimensional physical coordinates of the end effector of the robotic arm corresponding to the k-th calibration point in the base coordinate system of the robotic arm; Solving the above system of linear equations using the least squares method yields the parameter estimates for the transformation matrix M. Then, by combining the optimal plane equation parameters A, B, C obtained from step S32, the plane screen mapping model F is finally obtained. flat The formula is: ； Where T represents the matrix transpose operation.

6. The rapid calibration method for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits according to claim 4, characterized in that, If step S34 determines that the in-vehicle central control screen is a curved screen, then a curved screen mapping model F is constructed. curved The specific steps are as follows: First, a bivariate quadratic polynomial surface is used to approximate the spatial shape of the screen surface, and the fitted X, Y, and Z coordinate axes are obtained, namely the X coordinate, Y coordinate, and Z coordinate. For each calibration point P in the calibration dataset D i Construct feature vectors ;(u i ,v i () indicates that each calibration point is a known pixel coordinate pre-set in the pixel coordinate system of the vehicle central control screen, where i=1,2,...,K, K=M×N; Then, for the three coordinate axes X, Y, and Z, least-squares equations are constructed for the X, Y, and Z coordinates respectively; that is, the least-squares equations corresponding to the three coordinate axes X, Y, and Z are solved respectively to obtain the corresponding parameter vectors. ; From all K calibration points P in the calibration dataset D i eigenvectors The unified feature matrix Φ formed by vertically concatenating elements: ； The parameter vector of the X coordinate The expression is: ； Among them, X1, X2...X k These are the fitted X-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively. The parameter vector of the Y coordinate The expression is: ； Among them, Y1, Y2...Y k These are the fitted Y-coordinates of the 1st, 2nd, ..., Kth calibration points, respectively. The parameter vector of the Z coordinate The expression is: ； Among them, Z1, Z2...Z k The fitted Z-coordinates are the first, second, ..., Zth calibration points, respectively. Finally, the curved screen mapping model F is obtained. curved The expression is: ； Where T denotes the matrix transpose operation; p is the degree of u in the bivariate quadratic polynomial, and q is the degree of v in the bivariate quadratic polynomial; u p The term representing the power of pixel coordinate u; v q This represents the power term of the pixel coordinate v, used to construct the polynomial basis function.

7. The rapid calibration method for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits according to claim 4, characterized in that, The specific steps for verifying the mapping model F in step S4 are as follows: The vehicle's infotainment system displays a new test point P that was not involved in model training on the in-vehicle central control screen. test Get its pixel coordinates (u t ,v t ), where t represents the test point P. test , pixel coordinates (u t ,v t Input the mapping model F of the vehicle central control screen to obtain the predicted coordinates (X). p ,Y p Z p Then control the robotic arm to move and touch the test point, and verify the accuracy through the actual touch position feedback from the vehicle system.

8. A rapid calibration system for a robotic arm used for automated testing of planar and curved central control screens in intelligent cockpits, characterized in that, The system includes a host computer, a robotic arm, a vehicle-mounted central control screen, a camera, and a server. The host computer is connected to the robotic arm, the vehicle-mounted central control screen, the camera, and the server. The camera is used to record the display status of the vehicle-mounted central control screen and the dashboard, as well as the operation process of the robotic arm, in real time. The server is used to store calibration datasets, mapping model parameters, and test results, and supports the retrospective analysis of historical calibration data. The host computer is used to execute the rapid calibration method for the robotic arm as described in any one of claims 1 to 7, completing the entire process of calibration point generation, data acquisition, determination of planar or curved screens, construction of mapping models, and accuracy verification.

9. The rapid calibration system for robotic arms used in automated testing of planar and curved central control screens in intelligent cockpits according to claim 8, characterized in that, The robotic arm includes a bionic hand and a pressure sensor. The bionic hand is the end effector of the robotic arm, which performs operations on the vehicle's central control screen. The pressure sensor is installed on the bionic hand and integrates a tactile feedback function to identify whether the robotic arm contacts the vehicle's central control screen when it moves toward it, and to identify the force used by the robotic arm to touch the central control screen when it does.

10. The rapid calibration system for robotic arms used in automated testing of planar and curved central control screens in intelligent cockpits according to claim 8, characterized in that, The host computer includes a calibration process control module, a data acquisition and processing module, a mapping model construction module, and a coordinate transformation and testing engine. The calibration process control module is used to coordinate the communication between the host computer, the robotic arm control system and the vehicle-mounted central control screen. Based on the preset calibration points and combined with the spatial position of the vehicle-mounted central control screen, it plans the motion path of the end effector of the robotic arm. The data acquisition and processing module is used to simultaneously capture the theoretical pixel coordinates of the current calibration point and the real-time Cartesian coordinates of the end effector in the base coordinate system fed back by the robotic arm control system. It performs real-time verification of the data, removes outliers caused by robotic arm jitter, communication delay or abnormal touch, integrates the valid data into a calibration dataset D, and manages it. The mapping model construction module is used to automatically determine the type of the vehicle central control screen, and select a planar screen mapping model or a curved screen mapping model according to the determination result. The least squares method is used to fit and train the calibration dataset D to obtain the mapping model F. The coordinate transformation and testing engine is used in the automated testing phase to call the mapping model F, realize the conversion of UI coordinates into robotic arm coordinates, and execute the test sequence.

Images