Device and method for detecting smoothness of curved screen of automobile

Abstract

The invention relates to the technical field of automobile curved screen detection, in particular to a device and method for detecting the smoothness of an automobile curved screen, and the method comprises the steps: obtaining a two-dimensional plane image of the automobile curved screen through a 2D camera; mapping the positions of the pixels on the two-dimensional plane image into a three-dimensional space of the 3D camera to obtain corresponding three-dimensional position information of the curved screen; three-dimensional position information of a target icon on the automobile curved screen is determined through a 3D camera, a mechanical arm is controlled to accurately click the target icon on the automobile curved screen according to navigation coordinates, and high-frame-rate cameras arranged in different directions are synchronously started; acquiring continuous images of changes of different areas of the curved screen when the mechanical arm operates on the automobile curved screen; and inputting the continuous image information into a fluency detection algorithm for analysis, and evaluating the fluency performance of the automobile curved screen according to an output result of the fluency detection algorithm. According to the invention, high-precision and automatic detection of the fluency of the curved screen can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile curved screen detection, and in particular to a device and method for detecting the smoothness of an automobile curved screen. Background Art

[0002] With the continuous development of automobile technology, car screens have gradually evolved from traditional flat screens to curved screens, providing drivers with a more immersive visual experience. However, the introduction of curved screens has also brought new challenges, especially the problem of screen smoothness. Screen smoothness refers to the smoothness and response speed of screen switching when the screen displays images or videos. On car curved screens, the smoothness problem becomes more complicated due to the particularity of the screen shape and display technology.

[0003] At present, there are some methods for testing the smoothness of flat screens on the market, which usually use a single camera for testing. This method may have the technical problem of not being able to capture all the subtle movements and changes on the flat screen, let alone the detection of curved screens. In addition, there is a shortcoming: curved screens have different curvatures and viewing angles. These characteristics lead to differences in the display effect and smoothness of the screen at different positions. It is difficult for existing detection methods to fully cover these differences. Therefore, it is particularly important to develop a method specifically for testing the smoothness of automotive curved screens. Summary of the invention

[0004] One purpose of the present invention is to provide a device for detecting the smoothness of a car's curved screen, which can achieve high-precision and automated detection of the smoothness of the curved screen.

[0005] Another object of the present invention is to provide a method for detecting the smoothness of a car's curved screen, using a device for detecting the smoothness of a car's curved screen as described above.

[0006] To achieve this object, the present invention adopts the following technical solutions:

[0007] A device for detecting the smoothness of a car curved screen, comprising:

[0008] A two-dimensional plane image acquisition module is used to capture the image of the car's curved screen to obtain a two-dimensional plane image;

[0009] The 3D robotic arm module is connected to the control module and the 2D plane image acquisition module, and is used to obtain the 3D position information of the target icon in the 2D plane image and the operation information of the control module, and move to the front of the car curved screen to perform corresponding operations;

[0010] The image acquisition module is connected to the control module and is used to capture continuous images of changes in different areas of the curved screen when the robotic arm module operates on the curved screen of the car from different directions according to the operation information of the control module.

[0011] An image processing and analysis module is used to receive the two-dimensional plane image acquired by the two-dimensional plane image acquisition module, and execute an affine transformation algorithm to obtain the 3D position information of the car curved screen; and receive and analyze multiple groups of continuous images acquired by the image acquisition module, and execute a fluency detection algorithm to obtain a fluency detection result;

[0012] The control module is used to receive the user's operation instructions and send corresponding operation information to the 3D robot arm module and the analysis and image acquisition module;

[0013] The output module is used to output the identified smoothness detection results for evaluating the smoothness performance of the curved screen.

[0014] Preferably, the two-dimensional plane image acquisition module includes a 2D camera and a 2D camera control unit;

[0015] The 2D camera is used to capture the image of the curved screen to obtain a two-dimensional plane image;

[0016] The 2D camera control unit is used to adjust the shooting parameters of the 2D camera.

[0017] Preferably, the 3D robotic arm module comprises a 3D camera, a robotic arm and a robotic arm control unit;

[0018] The 3D camera is used to obtain the three-dimensional position information of the target icon on the current car curved screen;

[0019] The mechanical arm is used to move in front of the car curved screen according to the three-dimensional position information;

[0020] The robotic arm control unit is used to accurately control the movement of the robotic arm.

[0021] Preferably, the analysis image acquisition module includes a plurality of high frame rate cameras;

[0022] The multiple high frame rate cameras are respectively located at different positions in front of the car curved screen, and the multiple high frame rate cameras respectively shoot different areas of the car curved screen. The multiple high frame rate cameras are used to shoot continuous images of changes in different areas of the curved screen from different directions when the robotic arm module operates on the car curved screen.

[0023] Preferably, a method for detecting the smoothness of a car curved screen, using the device for detecting the smoothness of a car curved screen as described above, comprises the following steps:

[0024] S1, using a 2D camera to obtain a two-dimensional plane image of the car's curved screen;

[0025] S2, by optimizing and affine transforming the pixels on the two-dimensional plane image, mapping the positions of the pixels on the two-dimensional plane image to the three-dimensional space of the 3D camera, and obtaining the corresponding three-dimensional position information of the curved screen;

[0026] S3, determining the three-dimensional position information of the target icon on the car surface screen through the 3D camera, and converting it into the navigation coordinates of the robot arm;

[0027] S4, the robotic arm accurately clicks the target icon on the car's curved screen according to the navigation coordinates, and simultaneously starts high-frame rate cameras set in different directions to obtain continuous images of changes in different areas of the curved screen when the robotic arm operates on the car's curved screen;

[0028] S5, stitching the images in the same frame in the multiple groups of continuous images to form a continuous stitching image of the screen changes when the robot arm clicks on the entire car curved screen, and preprocessing the continuous stitching image to obtain processed continuous image information;

[0029] S6. Input the continuous image information into a smoothness detection algorithm for analysis, and evaluate the smoothness performance of the car curved screen according to the output result of the smoothness detection algorithm.

[0030] Preferably, in S2, the optimizing and affine transforming of pixels on the two-dimensional plane image comprises the following steps:

[0031] S21, accurately locating pixels on the two-dimensional plane image using an edge detection method or a corner detection method;

[0032] S22, calibrate the 2D camera in stereo, and use the least squares method to obtain the internal parameter covariance matrix of the 3D camera and the external parameter covariance matrix

[0033] S23. Using uncertainty propagation to calculate the internal parameter covariance matrix and the external parameter covariance matrix Evaluate the reprojection error between the final 2D camera and the 3D camera:

[0034]

[0035] Among them, P u Represents the coordinates of a two-dimensional plane image point, usually expressed as P u = (u, v), is the point projected onto the 2D image plane by the 3D camera, or written as a column vector Represents the pixel coordinates on a two-dimensional plane image;

[0036] K represents the camera's intrinsic parameter matrix, which describes the camera's focal length, principal point position, and possible pixel ratio. Usually, the intrinsic parameter matrix is ​​a 3*3 matrix, expressed as:

[0037]

[0038] R represents the rotation matrix of the camera coordinate system relative to the world coordinate system;

[0039] T represents the translation matrix of the camera coordinate system relative to the world coordinate system;

[0040] X w Represents a point in a three-dimensional world coordinate system, usually represented as a column vector:

[0041]

[0042] i means that the reprojection error of the i-th point is being calculated;

[0043] S24, when the reprojection error meets the preset threshold, the pixel coordinate system of the pixel position on the 2D camera is converted into the image coordinate system, and the coordinate of the pixel position in the pixel coordinate system is set to (P u ,P v ), the coordinates of the corresponding pixel in the image coordinate system are (x, y), then the relationship is:

[0044]

[0045] Among them, dx, dy represent the actual physical size of each pixel in the pixel coordinate system, in mm; u0, v0 represent the translation distance of each pixel in the pixel coordinate system;

[0046] By translating the pixel coordinate system, the pixel coordinate system can be converted to the image coordinate system. The calculation formula is:

[0047]

[0048] S25, convert the image coordinate system into the 3D camera coordinate system, according to the triangle similarity principle, let the three-dimensional coordinates be (X c ,Y c ,Z c ), the focal length of the 3D camera is f, then the relationship is:

[0049]

[0050] Convert the above relationship into:

[0051]

[0052] Then convert the pixel coordinate system to the coordinate system of the 3D image. The conversion formula is:

[0053]

[0054] in is the intrinsic parameter matrix of the 3D camera, set to K;

[0055] S26, transform the 3D camera coordinate system into the world coordinate system (X w ,Y w ,Z w ), the conversion formula is:

[0056]

[0057] Among them, R is the rotation matrix and T is the translation matrix.

[0058] Preferably, in S4, the robotic arm accurately clicks the target icon on the car curved screen according to the navigation coordinates, so that the control system performs control algorithm and motion planning on the robotic arm, specifically including the following steps:

[0059] S41, the control algorithm is controlled by a PID controller, specifically:

[0060] The output u(t) of the PID controller is a weighted sum based on the error e(t) = r(t) - y(t);

[0061] Where r(t) is the set point and y(t) is the output;

[0062] First, calculate the error e(t) = r(t) - y(t);

[0063] Substituting into the mathematical form of the PID controller:

[0064]

[0065] in,

[0066] u(t): control signal (controller output);

[0067] e(t): Error, the difference between the set point and the actual output;

[0068] K p : Proportional gain, which determines the immediate response to the error;

[0069] K i : Integral gain, which determines the effect of error accumulation over time;

[0070] K d : Differential gain, which determines the response of the error change rate;

[0071] The integral term takes into account the accumulation of all past errors and is mainly used to eliminate steady-state errors;

[0072] The derivative term predicts future error changes and helps reduce overshoot;

[0073] Finally, u(t) is obtained and applied to the control object;

[0074] S42. Motion planning includes path planning and trajectory planning: path planning is to calculate the optimal path of the robot arm from the starting point to the target point in 3D space; trajectory planning is to define the speed and acceleration of the robot arm during the movement.

[0075] Preferably, in S5, the collected continuous images are preprocessed to obtain processed continuous image information; specifically, the following steps are included:

[0076] S51, generating corresponding color images and grayscale images from continuous images, and performing denoising processing; wherein the image grayscale is converted to a weighted average method, and the denoising process uses a Gaussian filter to smooth the image;

[0077] S52, using the frame difference method to detect the difference of a certain pixel in two consecutive image frames: define two consecutive image frames as I1(x, y) and I2(x, y), then the frame difference D(x, y) is expressed as:

[0078] D(x,y)=|I1(x,y)-I2(x,y)|

[0079] S53, perform action detection, and extract significant change areas by applying thresholding to the frame difference result: Combine the frame difference D(x, y), and convert it into a binary image T(x, y) through thresholding operation:

[0080]

[0081] Wherein, t is the set transformation threshold. When the frame difference D(x, y) is greater than the set transformation threshold t, it indicates a changed area, and the changed area on the binary image T(x, y) is marked as white with RGB values ​​of 255; when the frame difference D(x, y) is less than the set transformation threshold t, it indicates an unchanged area, and the unchanged area on the binary image T(x, y) is marked as black with RGB values ​​of 0;

[0082] S54, using morphological operations to remove small noise areas and fill small holes in the binary image T(x,y), including dilation operations and erosion operations;

[0083] The expansion operation is expressed by the following formula:

[0084]

[0085] Among them, A is a binary image, B is a structural element, and + represents a dilation operation.

[0086] The corrosion operation is expressed by the following formula:

[0087]

[0088] Among them, - represents the corrosion operation;

[0089] S55, identifying the specific location of the action by extracting the contour from the binary image T(x,y) after the morphological operation:

[0090] The extracted contour is calculated by the following formula:

[0091] C={(x,y)|T(x,y)=255}

[0092] Among them, the contour C contains the pixel positions of all the changed areas;

[0093] By analyzing the contour C, the features of the contour C are calculated, including the area A and the perimeter P:

[0094]

[0095] in, is the image gradient, which is used to calculate the edge changes;

[0096] S56. Define the specific action or event to which the contour C belongs based on the features of the contour C.

[0097] Preferably, in S6, the continuous image information is input into a fluency detection algorithm for analysis, which specifically includes the following steps:

[0098] S61, if the continuous image information contains noise interference information, applying Gaussian blur to the continuous image to remove high-frequency noise; or removing salt and pepper noise by median filtering;

[0099] S62, if there is artifact information in the continuous image information, performing local comparison on the artifact area;

[0100] S63, setting a pixel difference threshold, a constant area threshold, and a change area threshold to distinguish and define the change area and change situation of two consecutive image frames: if the frame difference D(x, y) is greater than the pixel difference threshold, it is considered that the pixel has changed in the two consecutive image frames, and the adjacent pixels that are greater than the set pixel difference threshold with the pixel are merged and defined as the change area;

[0101] Otherwise, the pixel is considered unchanged in two consecutive frames of images, and the adjacent pixels whose difference with the pixel is less than the set pixel difference threshold are merged and defined as the unchanged area;

[0102] If the area of ​​the changing region is between the constant area threshold and the changing area threshold, and such a stable change relationship is maintained between multiple frames of continuous images, it means that the changing region is regarded as a normal dynamic change, indicating that the sliding smoothness is high;

[0103] If the area of ​​the changed region is larger than the change area threshold, and such a stable change relationship is maintained between multiple frames of continuous images, it means that the changed region indicates frame rate fluctuation or some image quality problems;

[0104] If the area of ​​the unchanged region is greater than the unchanged area threshold, there is no change between multiple consecutive image frames, which means that the unchanged region does not produce a gradual change effect between every two consecutive image frames in a certain time period. The response time during this period is not high enough, and there may be a freeze phenomenon.

[0105] One of the above technical solutions has the following beneficial effects:

[0106] 1. Improve detection accuracy: By integrating 2D cameras, 3D cameras and high frame rate cameras, it is possible to capture the information of the curved screen more comprehensively, improving the accuracy and reliability of detection. In addition, the high frame rate camera can capture subtle movements and changes on the screen, ensuring the accuracy of smoothness detection.

[0107] 2. Realize automated testing: The introduction of the robotic arm realizes automated operation, reduces manual intervention, and improves testing efficiency. The control system can automatically execute the preset test script to ensure the consistency of each test.

[0108] 3. Comprehensive coverage of the screen area: Three high-frame rate cameras can cover different areas of the screen to ensure comprehensive fluency detection. This layout helps capture subtle changes in the edges and corners of the screen to avoid missing potential fluency issues. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 The present invention is a physical schematic diagram of a device for detecting the smoothness of a curved screen of an automobile. DETAILED DESCRIPTION

[0110] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.

[0111] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0113] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0114] A device for detecting the smoothness of a car curved screen, comprising:

[0115] A two-dimensional plane image acquisition module is used to capture the image of the car's curved screen to obtain a two-dimensional plane image;

[0116] The 3D robotic arm module is connected to the control module and the 2D plane image acquisition module, and is used to obtain the 3D position information of the target icon in the 2D plane image and the operation information of the control module, and move to the front of the car curved screen to perform corresponding operations;

[0117] The image acquisition module is connected to the control module and is used to capture continuous images of changes in different areas of the curved screen when the robotic arm module operates on the curved screen of the car from different directions according to the operation information of the control module.

[0118] An image processing and analysis module is used to receive the two-dimensional plane image acquired by the two-dimensional plane image acquisition module, and execute an affine transformation algorithm to obtain the 3D position information of the car curved screen; and receive and analyze multiple groups of continuous images acquired by the image acquisition module, and execute a fluency detection algorithm to obtain a fluency detection result;

[0119] The control module is used to receive the user's operation instructions and send corresponding operation information to the 3D robot arm module and the analysis and image acquisition module;

[0120] The output module is used to output the identified smoothness detection results for evaluating the smoothness performance of the curved screen.

[0121] To further illustrate, the two-dimensional plane image acquisition module includes a 2D camera and a 2D camera control unit;

[0122] The 2D camera is used to capture the image of the curved screen to obtain a two-dimensional plane image;

[0123] The 2D camera control unit is used to adjust the shooting parameters of the 2D camera.

[0124] To further illustrate, the 3D robotic arm module includes a 3D camera, a robotic arm, and a robotic arm control unit;

[0125] The 3D camera is used to obtain the three-dimensional position information of the target icon on the current car curved screen;

[0126] The mechanical arm is used to move in front of the car curved screen according to the three-dimensional position information;

[0127] The robotic arm control unit is used to accurately control the movement of the robotic arm.

[0128] As a further explanation, the analysis image acquisition module includes a plurality of high frame rate cameras;

[0129] The multiple high frame rate cameras are respectively located at different positions in front of the car curved screen, and the multiple high frame rate cameras respectively shoot different areas of the car curved screen. The multiple high frame rate cameras are used to shoot continuous images of changes in different areas of the curved screen from different directions when the robotic arm module operates on the car curved screen.

[0130] The working principle of this device is to achieve accurate detection of the smoothness of the car's curved screen by integrating multiple sensors and working with a robotic arm. The specific steps are as follows:

[0131] Initialization and calibration: Before starting the inspection, the 2D camera, 3D camera and three high frame rate cameras are calibrated to ensure that they can accurately capture the image information of the curved screen and are consistent with the coordinate system of the robotic arm. The robotic arm also needs to be initialized to ensure that it can move accurately along the preset path.

[0132] Image capture and 3D mapping: 2D cameras capture 2D images of curved screens, providing basic data for subsequent image processing and affine transformation. 3D cameras use stereo vision technology or structured light technology to obtain 3D spatial information of curved screens. This information will be used to build a 3D model of the curved screen and provide navigation for subsequent precise operation of the robotic arm.

[0133] Robotic arm operation and image acquisition: The control system commands the robotic arm to move to a specific position on the curved screen (such as in front of the target icon) according to the preset test script. The robotic arm performs a click operation to simulate the action of a user touching the screen.

[0134] At the same time as the robotic arm clicks, three high-frame rate cameras start synchronously to capture continuous images of screen changes at a high frame rate. They can cover different areas of the screen and capture more subtle movements and changes.

[0135] Image processing and fluency analysis: The captured continuous images are sent to the image processing module of the control system for preprocessing and feature extraction. Subsequently, these images are input into the fluency detection algorithm, which analyzes parameters such as the response speed of the screen during the click of the robot arm and the smoothness of the screen switching. Finally, a fluency evaluation result is output to evaluate the fluency performance of the curved screen.

[0136] In summary, the device of the present invention has the following beneficial effects:

[0137] 1. Improve detection accuracy: By integrating 2D cameras, 3D cameras and high frame rate cameras, it is possible to capture the information of the curved screen more comprehensively, improving the accuracy and reliability of detection. In addition, the high frame rate camera can capture subtle movements and changes on the screen, ensuring the accuracy of smoothness detection.

[0138] 2. Realize automated testing: The introduction of the robotic arm realizes automated operation, reduces manual intervention, and improves testing efficiency. The control system can automatically execute the preset test script to ensure the consistency of each test.

[0139] 3. Comprehensive coverage of the screen area: Three high-frame rate cameras can cover different areas of the screen to ensure comprehensive fluency detection. This layout helps capture subtle changes in the edges and corners of the screen to avoid missing potential fluency issues.

[0140] Wherein, the robotic arm has a kinematic basis and posture control;

[0141] The kinematics includes forward kinematics and inverse kinematics: forward kinematics refers to calculating the position and posture of the end effector in 3D space given the angles of each joint, by using a homogeneous transformation matrix to describe the relationship between joints; inverse kinematics refers to calculating the required angle of each joint based on the target position and posture;

[0142] The posture control is to accurately control the position and posture of the end effector according to the 3D position information, by: expressing the target position and posture as a 4x4 posture matrix for calculation; using linear interpolation or spline interpolation to plan the motion trajectory.

[0143] Further explanation, a method for detecting the smoothness of a car curved screen, using the device for detecting the smoothness of a car curved screen as described above, comprises the following steps:

[0144] S1, using a 2D camera to obtain a two-dimensional plane image of the car's curved screen;

[0145] S2, by optimizing and affine transforming the pixels on the two-dimensional plane image, mapping the positions of the pixels on the two-dimensional plane image to the three-dimensional space of the 3D camera, and obtaining the corresponding three-dimensional position information of the curved screen;

[0146] S3, determining the three-dimensional position information of the target icon on the car surface screen through the 3D camera, and converting it into the navigation coordinates of the robot arm;

[0147] S4, the robotic arm accurately clicks the target icon on the car's curved screen according to the navigation coordinates, and simultaneously starts high-frame rate cameras set in different directions to obtain continuous images of changes in different areas of the curved screen when the robotic arm operates on the car's curved screen;

[0148] S5, stitching the images in the same frame in the multiple groups of continuous images to form a continuous stitching image of the screen changes when the robot arm clicks on the entire car curved screen, and preprocessing the continuous stitching image to obtain processed continuous image information;

[0149] S6. Input the continuous image information into a smoothness detection algorithm for analysis, and evaluate the smoothness performance of the car curved screen according to the output result of the smoothness detection algorithm.

[0150] The method of the present invention is mainly used to detect the smoothness of the curved screen of an automobile, and its working principle can be divided into the following steps:

[0151] 2D image acquisition (S1): Use a 2D camera to capture a 2D plane image of the car's curved screen. The purpose of this step is to obtain preliminary visual information of the screen and provide a basis for subsequent 3D space mapping.

[0152] Pixel mapping to three-dimensional space (S2): Optimize the pixels on the two-dimensional plane image, including removing noise, enhancing contrast, etc. Apply affine transformation to map the optimized pixel position from the two-dimensional plane image to the three-dimensional space of the 3D camera. The purpose of this step is to obtain the precise position information of the screen pixels in the three-dimensional space to provide accurate navigation for the operation of the robot arm.

[0153] Target icon three-dimensional position determination (S3): Use a 3D camera to identify and determine the three-dimensional position information of the target icon (such as an APP icon) on the car's curved screen. The three-dimensional position information of the target icon is converted into the navigation coordinates of the robot arm to provide guidance for the precise click operation of the robot arm.

[0154] Robotic arm click and image acquisition (S4): The robot arm accurately clicks the target icon on the car's curved screen according to the navigation coordinates. High frame rate cameras set in different directions are started synchronously to capture continuous images of the screen changes when the robot arm clicks. The high frame rate camera can ensure that subtle movements and changes on the screen are captured.

[0155] Image processing (S5): Preprocessing the continuous images, such as denoising, contrast enhancement, etc., to improve image quality. The processed continuous image information is used for subsequent analysis.

[0156] Fluency detection and analysis (S6): The processed continuous image information is input into the fluency detection algorithm for analysis, so as to evaluate the fluency and response speed of the screen during the robot arm clicking process, and output the evaluation results.

[0157] In summary, the method of the present invention has the following beneficial effects:

[0158] 1. Improve detection accuracy: By combining 2D and 3D technologies, the three-dimensional position information of pixels on the screen can be measured more accurately, thereby improving detection accuracy.

[0159] 2. Using a high frame rate camera to inspect the curved screen from different directions can capture various subtle movements and changes on the screen, further improving the accuracy and reliability of the inspection.

[0160] 3. Comprehensive testing: A variety of camera configurations (three high-frame rate cameras, one 3D camera, and one 2D workstation camera) are used to fully cover different areas and angles of the curved screen, achieving a more comprehensive test of the screen smoothness.

[0161] 4. Simulate real-life usage scenarios: By using a robotic arm to perform click and other operations, the system simulates the operations of real users when using the car screen, allowing for a more realistic evaluation of the screen's smoothness and responsiveness.

[0162] 5. Optimize user experience: Accurately evaluating the smoothness of curved screens can help automakers optimize screen design and performance and improve user experience.

[0163] 6. Timely discovery and resolution of fluency issues can reduce user inconvenience and complaints during use.

[0164] In summary, the method of the present invention provides an accurate, comprehensive and practical solution for smoothness detection of automobile curved screens, which helps to improve the quality of automobile screens and user experience.

[0165] To further illustrate, in S2, the optimization processing and affine transformation of the pixels on the two-dimensional plane image include the following steps:

[0166] S21, accurately locating pixels on the two-dimensional plane image using an edge detection method or a corner detection method;

[0167] Since there are errors in the pixel positions, the above optimization steps can reduce the uncertainty of the pixel positions.

[0168] S22, calibrate the 2D camera in stereo, and use the least squares method to obtain the internal parameter covariance matrix of the 3D camera and the external parameter covariance matrix

[0169] In the 3D reconstruction process, uncertainty comes from many aspects, such as inaccurate camera calibration, measurement errors, ambient lighting changes, etc. When using the least squares method to obtain the intrinsic covariance matrix of the 3D camera and the external parameter covariance matrix The uncertainty propagation method can be reused to estimate the reprojection error between the final 2D camera and the 3D camera.

[0170] S23. Using uncertainty propagation to calculate the internal parameter covariance matrix and the external parameter covariance matrix Evaluate the reprojection error between the final 2D camera and the 3D camera:

[0171]

[0172] Among them, P u Represents the coordinates of a two-dimensional plane image point, usually expressed as P u = (u, v), is the point projected onto the 2D image plane by the 3D camera, or written as a column vector Represents the pixel coordinates on a two-dimensional plane image;

[0173] K represents the camera's intrinsic parameter matrix, which describes the camera's focal length, principal point position, and possible pixel ratio. Usually, the intrinsic parameter matrix is ​​a 3*3 matrix, expressed as:

[0174]

[0175] R represents the rotation matrix of the camera coordinate system relative to the world coordinate system;

[0176] T represents the translation matrix of the camera coordinate system relative to the world coordinate system;

[0177] X w Represents a point in a three-dimensional world coordinate system, usually represented as a column vector:

[0178]

[0179] i means that the reprojection error of the i-th point is being calculated;

[0180] In the camera model, the reprojection error is an important metric for evaluating the difference between a 2D point and its projected 3D point in the image. When the reprojection error does not meet the preset threshold, using a robust estimation method such as RANSAC can effectively exclude outliers from the data, thereby improving the accuracy of the affine transformation.

[0181] S24, when the reprojection error meets the preset threshold, the pixel coordinate system of the pixel position on the 2D camera is converted into the image coordinate system, and the coordinate of the pixel position in the pixel coordinate system is set to (P u ,P v ), the coordinates of the corresponding pixel in the image coordinate system are (x, y), then the relationship is:

[0182]

[0183] Among them, dx, dy represent the actual physical size of each pixel in the pixel coordinate system, in mm; u0, v0 represent the translation distance of each pixel in the pixel coordinate system;

[0184] By translating the pixel coordinate system, the pixel coordinate system can be converted to the image coordinate system. The calculation formula is:

[0185]

[0186] S25, convert the image coordinate system into the 3D camera coordinate system, according to the triangle similarity principle, let the three-dimensional coordinates be (X c ,Y c ,Z c ), the focal length of the 3D camera is f, then the relationship is:

[0187]

[0188] Convert the above relationship into:

[0189]

[0190] Then convert the pixel coordinate system to the coordinate system of the 3D image. The conversion formula is:

[0191]

[0192] in is the intrinsic parameter matrix of the 3D camera, set to K;

[0193] S26, transform the 3D camera coordinate system into the world coordinate system (X w ,Y w ,Z w ), the conversion formula is:

[0194]

[0195] Among them, R is the rotation matrix and T is the translation matrix.

[0196] It should be noted that the optimization process may also include using filters to process dynamic scenes and adopting data fusion methods:

[0197] In dynamic scenes, using techniques such as Kalman filtering and particle filtering can help us track objects in time series and deal with noise and errors in time series data. These filters can continuously update state estimates based on new observations, thereby improving overall accuracy.

[0198] In multiple viewpoints or multi-sensor systems, data fusion can effectively reduce uncertainty. For example, combining lidar and camera data can obtain a more accurate 3D point cloud, and sensor fusion technology (such as extended Kalman filter, EKF) can be used to combine the measurement information of different sensors.

[0199] Further explanation, in S4, the robotic arm accurately clicks the target icon on the car surface screen according to the navigation coordinates, and the control system performs control algorithm and motion planning on the robotic arm, which specifically includes the following steps:

[0200] S41, the control algorithm is mainly controlled by the PID controller, which can effectively reduce the steady-state error and overshoot; specifically:

[0201] The output u(t) of the PID controller is a weighted sum based on the error e(t) = r(t) - y(t);

[0202] Where r(t) is the set point and y(t) is the output;

[0203] First, calculate the error e(t) = r(t) - y(t);

[0204] Substituting into the mathematical form of the PID controller:

[0205]

[0206] in,

[0207] u(t): control signal (controller output);

[0208] e(t): Error, the difference between the set point and the actual output;

[0209] K p : Proportional gain, which determines the immediate response to the error;

[0210] K i : Integral gain, which determines the effect of error accumulation over time;

[0211] K d : Differential gain, which determines the response of the error change rate;

[0212] The integral term takes into account the accumulation of all past errors and is mainly used to eliminate steady-state errors;

[0213] The derivative term predicts future error changes and helps reduce overshoot;

[0214] Finally, u(t) is obtained and applied to the control object;

[0215] Robust control can also be used: external disturbances and changes in system parameters are taken into account to ensure accurate control under uncertainty.

[0216] S42. Motion planning includes path planning and trajectory planning: path planning is to calculate the optimal path of the robot arm from the starting point to the target point in 3D space to avoid obstacles; trajectory planning is to define the speed and acceleration of the robot arm during the movement so that the movement is smooth and meets physical limitations.

[0217] In addition, the robot arm can also add a sensor system to provide real-time feedback and environmental perception, mainly including position sensors and force and tactile sensors. Position sensors include encoders: installed on each joint, real-time feedback of the angular position of the joint; laser rangefinders or depth cameras: used to detect the precise position of the end effector in 3D space, especially in dynamic environments. Force and tactile sensors include force sensors: monitor the force applied by the end effector to ensure proper contact and prevent damage during operation; tactile sensors: provide tactile feedback to help the robot arm be more precise when handling flexible objects or complex surfaces.

[0218] Furthermore, the control system also has repeatability, which refers to the ability of the robot to reach the same position when repeating the same action multiple times. It can be evaluated by calculating the standard deviation of the end effector position through repeated experiments. The use of high-quality servo motors and feedback systems can ensure high repeatability. The current accuracy of the robot and subsequent optimization strategies are measured based on the estimated deviation results.

[0219] To further illustrate, in S5, the continuous spliced ​​images are preprocessed to obtain processed continuous image information; specifically, the following steps are included:

[0220] S51, generating corresponding color images and grayscale images from continuous images, and performing denoising processing; wherein the image grayscale is converted to a weighted average method, and the denoising process uses a Gaussian filter to smooth the image;

[0221] S52, using the frame difference method to detect the difference of a certain pixel in two consecutive image frames: define two consecutive image frames as I1(x, y) and I2(x, y), then the frame difference D(x, y) is expressed as:

[0222] D(x,y)=|I1(x,y)-I2(x,y)|

[0223] By calculating the pixel differences between adjacent frames, changes in the scene can be identified;

[0224] S53, perform action detection, and extract significant change areas by applying thresholding to the frame difference result: Combine the frame difference D(x, y), and convert it into a binary image T(x, y) through thresholding operation:

[0225]

[0226] Wherein, t is the set transformation threshold. When the frame difference D(x, y) is greater than the set transformation threshold t, it indicates a changed area, and the changed area on the binary image T(x, y) is marked as white with RGB values ​​of 255; when the frame difference D(x, y) is less than the set transformation threshold t, it indicates an unchanged area, and the unchanged area on the binary image T(x, y) is marked as black with RGB values ​​of 0;

[0227] S54, using morphological operations to remove small noise areas and fill small holes in the binary image T(x,y), including dilation operations and erosion operations;

[0228] The expansion operation is expressed by the following formula:

[0229]

[0230] Among them, A is a binary image, B is a structural element, and + represents a dilation operation.

[0231] The corrosion operation is expressed by the following formula:

[0232]

[0233] Among them, - represents the corrosion operation;

[0234] These morphological operations help to remove some irrelevant noise and keep important action areas;

[0235] S55, identifying the specific location of the action by extracting the contour from the binary image T(x,y) after the morphological operation:

[0236] The extracted contour is calculated by the following formula:

[0237] C={(x,y)|T(x,y)=255}

[0238] Among them, the contour C contains the pixel positions of all the changed areas;

[0239] By analyzing the contour C, the features of the contour C are calculated, including the area A and the perimeter P:

[0240]

[0241] in, is the image gradient, which is used to calculate the edge changes;

[0242] S56. Define the specific action or event to which the contour C belongs based on the features of the contour C.

[0243] For example, assuming we focus on a specific action (such as clicking, sliding), we can classify it according to the shape and change pattern of the contour.

[0244] Assuming that the area A and perimeter P of the extracted contour meet certain conditions (such as A>100 and P<200), we can mark the area as a specific action.

[0245] Furthermore, due to the huge amount of data generated by high frame rate cameras, real-time processing is an important challenge. Therefore, optimization methods can also be used to improve the detection speed. Common optimization methods are: multi-threaded processing: image reading, preprocessing and detection tasks can be assigned to different threads to improve efficiency; GPU acceleration: using GPU acceleration technologies such as CUDA or OpenCL can significantly improve the speed of image processing, especially under high frame rate and high resolution conditions.

[0246] To further illustrate, in S6, the continuous image information is input into the fluency detection algorithm for analysis, which specifically includes the following steps:

[0247] S61, setting a pixel difference threshold, a constant area threshold, and a change area threshold to distinguish and define the change area and change situation of two consecutive frames of images: if the frame difference D(x, y) is greater than the pixel difference threshold, it is considered that the pixel has changed in the two consecutive frames of images, and the adjacent pixels that are greater than the set pixel difference threshold with the pixel are merged and defined as the change area;

[0248] Otherwise, the pixel is considered unchanged in two consecutive frames of images, and the adjacent pixels whose difference with the pixel is less than the set pixel difference threshold are merged and defined as the unchanged area;

[0249] If the area of ​​the changing region is between the constant area threshold and the changing area threshold, and such a stable change relationship is maintained between multiple frames of continuous images, it means that the changing region is regarded as a normal dynamic change, indicating that the sliding smoothness is high;

[0250] If the area of ​​the changed region is larger than the change area threshold, and such a stable change relationship is maintained between multiple frames of continuous images, it means that the changed region indicates frame rate fluctuation or some image quality problems;

[0251] If the area of ​​the unchanged region is greater than the unchanged area threshold, there is no change between multiple consecutive frames of images, which means that the unchanged region does not produce a gradual change effect between every two frames in a certain period of time. The response time during this period of time is not high enough, and there may be a freeze phenomenon.

[0252] It should be noted that the pixel difference threshold, the unchanged area threshold and the changed area threshold are determined by comparing multiple scenes or test sets, and gradually adjusting the thresholds until the best balance point for distinguishing changes and maintaining invariance is found.

[0253] The effectiveness of the above analysis method is reflected in the following aspects: pixel-by-pixel comparison is used. During the gradient process, the color or brightness of the image changes gradually, usually continuously. Using the pixel difference method, the two frames of images are compared pixel by pixel, which can detect the slight changes of each pixel in the gradient process, especially when the color smoothness of the display is high, and can capture the subtle differences in the gradient effect; inter-frame change detection can be performed. During the gradient process, there are usually continuous and linear changes between frames. For example, the screen brightness gradually increases or decreases, and the color gradient changes smoothly and gradually. The difference between the two frames of images (especially in the middle area of ​​the gradient) can be detected by a simple pixel difference or SSIM method to identify the area where the image changes frame by frame; noise and artifact processing. For most UI transition animations, the background changes are usually simple, and the movement or fading of UI elements is easier to detect. Compared with complex scene changes (such as fast motion or complex object changes in video playback), the changes in UI transitions are usually clean, and the detection and processing of noise and artifacts are relatively easy, so the interference of noise and artifacts is relatively small.

[0254] In UI transitions, usually not the entire screen is changing. Many UI transition animations only affect a portion of the elements or areas (such as the smooth transition of a pop-up window or the displacement of a card). If global pixel differences or SSIM calculations are used, these local changes may be mistaken for changes in a larger area, resulting in unnecessary calculations and errors. Therefore, local contrast may sometimes be more effective, especially when dealing with certain UI transitions that only affect local areas. The contrast processing strategy needs to be adjusted according to the specific situation; Edge processing issues: If the UI interface transition process involves some complex edge animations (such as fading buttons, icons, etc.), these edge changes may appear as small fluctuations at the pixel level between different frames. Traditional image difference methods may not be able to effectively judge these details, resulting in small changes in the transition animation being ignored or misjudged.

[0255] In order to more comprehensively evaluate the smoothness of the curved surface, the screen's refresh rate, response speed, animation smoothness, and any issues that may cause freezes or delays are also tested.

[0256] Screen refresh rate: Refresh rate refers to the number of times the screen redraws the image per second, usually in Hertz (Hz). By shooting the screen and playing it through slow video (high frame rate camera), you can directly observe the screen refresh rate. For example, when shooting a 60Hz screen, you will see the screen change 60 times per second, and a 120Hz screen will change 120 times per second. A high refresh rate generally means smoother animation and response.

[0257] Response speed: Response speed refers to the time it takes for a pixel to change from one color to another, usually measured in milliseconds (ms). Long response times may cause problems such as smearing and blurring, especially in fast-moving images. For image elements within the car screen that may change, the time it takes for the display to respond is recorded by a high frame rate camera. The response speed can be estimated by analyzing the image movement blur in the video. The shorter the response time, the better the dynamic performance of the screen, reducing motion blur and smearing. The response time of the screen is usually expected to be controlled below 5ms. For situations where the refresh rate requirements are relatively high, the lower the screen response time, the better.

[0258] Animation smoothness: Animation smoothness refers to the smoothness of the screen when displaying dynamic content. It can also be called the smoothness index, which directly determines the smoothness of the screen. It is usually affected by the refresh rate and response time. High refresh rate and low response time can effectively improve the smoothness of animation. The relevant calculation formula can be written as:

[0259] in,

[0260] F smooth It is the smoothness index, which indicates the smoothness of screen sliding. The higher the value, the smoother the experience.

[0261] R is the refresh rate of the screen, measured in Hz.

[0262] T response It is the response time of the screen in milliseconds (ms).

[0263] The technical principle of the present invention is described above in conjunction with specific embodiments. These descriptions are only for explaining the principle of the present invention and cannot be interpreted as limiting the scope of protection of the present invention in any way. Based on the explanations herein, those skilled in the art can associate other specific embodiments of the present invention without creative work, and these equivalent variations or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A device for detecting the smoothness of a car's curved screen, characterized in that: include: A two-dimensional plane image acquisition module is used to capture the image of the car's curved screen to obtain a two-dimensional plane image; The 3D robotic arm module is connected to the control module and the 2D plane image acquisition module, and is used to obtain the 3D position information of the target icon in the 2D plane image and the operation information of the control module, and move to the front of the car curved screen to perform corresponding operations; The image acquisition module is connected to the control module and is used to capture continuous images of changes in different areas of the curved screen when the robotic arm module operates on the curved screen of the car from different directions according to the operation information of the control module. An image processing and analysis module is used to receive the two-dimensional plane image acquired by the two-dimensional plane image acquisition module, and execute an affine transformation algorithm to obtain the 3D position information of the car curved screen; And, receiving and analyzing multiple groups of continuous images collected by the image acquisition module, and executing a fluency detection algorithm to obtain a fluency detection result; The control module is used to receive the user's operation instructions and send corresponding operation information to the 3D robot arm module and the analysis and image acquisition module; The output module is used to output the identified smoothness detection results for evaluating the smoothness performance of the curved screen.

2. The device for detecting the smoothness of a car curved screen according to claim 1, characterized in that: The two-dimensional plane image acquisition module includes a 2D camera and a 2D camera control unit; The 2D camera is used to capture the image of the curved screen to obtain a two-dimensional plane image; The 2D camera control unit is used to adjust the shooting parameters of the 2D camera.

3. The device for detecting the smoothness of a car curved screen according to claim 1, characterized in that: The 3D robotic arm module includes a 3D camera, a robotic arm and a robotic arm control unit; The 3D camera is used to obtain the three-dimensional position information of the target icon on the current car curved screen; The mechanical arm is used to move in front of the car curved screen according to the three-dimensional position information; The robotic arm control unit is used to accurately control the movement of the robotic arm.

4. The device for detecting the smoothness of a car curved screen according to claim 1, characterized in that: The analysis image acquisition module includes a plurality of high frame rate cameras; The multiple high frame rate cameras are respectively located at different positions in front of the car curved screen, and the multiple high frame rate cameras respectively shoot different areas of the car curved screen. The multiple high frame rate cameras are used to shoot continuous images of changes in different areas of the curved screen from different directions when the robotic arm module operates on the car curved screen.

5. A method for detecting the smoothness of a car curved screen, characterized in that: The device for detecting the smoothness of a curved screen of an automobile as claimed in any one of claims 1 to 4 comprises the following steps: S1, using a 2D camera to obtain a two-dimensional plane image of the car's curved screen; S2, by optimizing and affine transforming the pixels on the two-dimensional plane image, mapping the positions of the pixels on the two-dimensional plane image to the three-dimensional space of the 3D camera, and obtaining the corresponding three-dimensional position information of the curved screen; S3, determining the three-dimensional position information of the target icon on the car surface screen through the 3D camera, and converting it into the navigation coordinates of the robot arm; S4, the robotic arm accurately clicks the target icon on the car's curved screen according to the navigation coordinates, and simultaneously starts high-frame rate cameras set in different directions to obtain continuous images of changes in different areas of the curved screen when the robotic arm operates on the car's curved screen; S5, stitching the images in the same frame in the multiple groups of continuous images to form a continuous stitching image of the screen changes when the robot arm clicks on the entire car curved screen, and preprocessing the continuous stitching image to obtain processed continuous image information; S6. Input the continuous image information into a smoothness detection algorithm for analysis, and evaluate the smoothness performance of the car curved screen according to the output result of the smoothness detection algorithm.

6. The method for detecting the smoothness of a car curved screen according to claim 5, characterized in that: In S2, the optimization processing and affine transformation of the pixels on the two-dimensional plane image include the following steps: S21, accurately locating pixels on the two-dimensional plane image using an edge detection method or a corner detection method; S22, calibrate the 2D camera in stereo, and use the least squares method to obtain the internal parameter covariance matrix of the 3D camera and the external parameter covariance matrix S23. Using uncertainty propagation to calculate the internal parameter covariance matrix and the external parameter covariance matrix Evaluate the reprojection error between the final 2D camera and the 3D camera: Among them, P u Represents the coordinates of a two-dimensional plane image point, usually expressed as P u = (u, v), is the point projected onto the 2D image plane by the 3D camera, or written as a column vector Represents the pixel coordinates on a two-dimensional plane image; K represents the camera's intrinsic parameter matrix, which describes the camera's focal length, principal point position, and possible pixel ratio. Usually, the intrinsic parameter matrix is ​​a 3*3 matrix, expressed as: R represents the rotation matrix of the camera coordinate system relative to the world coordinate system; T represents the translation matrix of the camera coordinate system relative to the world coordinate system; X w Represents a point in a three-dimensional world coordinate system, usually represented as a column vector: i means that the reprojection error of the i-th point is being calculated; S24, when the reprojection error meets the preset threshold, the pixel coordinate system of the pixel position on the 2D camera is converted into the image coordinate system, and the coordinate of the pixel position in the pixel coordinate system is set to (P u ,P v ), the coordinates of the corresponding pixel in the image coordinate system are (x, y), then the relationship is: Among them, dx, dy represent the actual physical size of each pixel in the pixel coordinate system, in mm; u0, v0 represent the translation distance of each pixel in the pixel coordinate system; By translating the pixel coordinate system, the pixel coordinate system can be converted to the image coordinate system. The calculation formula is: S25, convert the image coordinate system into the 3D camera coordinate system, according to the triangle similarity principle, let the three-dimensional coordinates be (X c ,Y c ,Z c ), the focal length of the 3D camera is f, then the relationship is: Convert the above relationship into: Then convert the pixel coordinate system to the coordinate system of the 3D image. The conversion formula is: in is the intrinsic parameter matrix of the 3D camera, set to K; S26, convert the 3D camera coordinate system into the world coordinate system (X w ,Y w ,Z w ), the conversion formula is: Among them, R is the rotation matrix and T is the translation matrix.

7. The method for detecting the smoothness of a car curved screen according to claim 5, characterized in that: In S4, the robotic arm accurately clicks the target icon on the car surface screen according to the navigation coordinates, and the control system performs control algorithms and motion planning on the robotic arm, which specifically includes the following steps: S41, the control algorithm is controlled by a PID controller, specifically: The output u(t) of the PID controller is a weighted sum based on the error e(t) = r(t) - y(t); Where r(t) is the set point and y(t) is the output; First, calculate the error e(t) = r(t) - y(t); Substituting into the mathematical form of the PID controller: in, u(t): control signal (controller output); e(t): Error, the difference between the set point and the actual output; K p : Proportional gain, which determines the immediate response to the error; K i : Integral gain, which determines the effect of error accumulation over time; K d : Differential gain, which determines the response of the error change rate; The integral term takes into account the accumulation of all past errors and is mainly used to eliminate steady-state errors; The derivative term predicts future error changes and helps reduce overshoot; Finally, u(t) is obtained and applied to the control object; S42. Motion planning includes path planning and trajectory planning: path planning is to calculate the optimal path of the robot arm from the starting point to the target point in 3D space; trajectory planning is to define the speed and acceleration of the robot arm during the movement.

8. The method for detecting the smoothness of a car curved screen according to claim 5, characterized in that: In S5, the continuous stitching images are preprocessed to obtain processed continuous image information; specifically, the following steps are included: S51, generating corresponding color images and grayscale images from continuous images, and performing denoising processing; wherein the image grayscale is converted to a weighted average method, and the denoising process uses a Gaussian filter to smooth the image; S52, using the frame difference method to detect the difference of a certain pixel in two consecutive image frames: define two consecutive image frames as I1(x, y) and I2(x, y), then the frame difference D(x, y) is expressed as: D(x,y)=|I1(x,y)-I2(x,y)| S53, perform action detection, and extract significant change areas by applying thresholding to the frame difference result: Combine the frame difference D(x, y), and convert it into a binary image T(x, y) through thresholding operation: Wherein, t is the set transformation threshold. When the frame difference D(x, y) is greater than the set transformation threshold t, it indicates a changed area, and the changed area on the binary image T(x, y) is marked as white with RGB values ​​of 255; when the frame difference D(x, y) is less than the set transformation threshold t, it indicates an unchanged area, and the unchanged area on the binary image T(x, y) is marked as black with RGB values ​​of 0; S54, using morphological operations to remove small noise areas and fill small holes in the binary image T(x,y), including dilation operations and erosion operations; The expansion operation is expressed by the following formula: Among them, A is a binary image, B is a structural element, and + represents a dilation operation. The corrosion operation is expressed by the following formula: Among them, - represents the corrosion operation; S55, identifying the specific location of the action by extracting the contour from the binary image T(x,y) after the morphological operation: The extracted contour is calculated by the following formula: C={(x,y)|T(x,y)=255} Among them, the contour C contains the pixel positions of all the changed areas; By analyzing the contour C, the features of the contour C are calculated, including the area A and the perimeter P: in, is the image gradient, which is used to calculate the edge changes; S56. Define the specific action or event to which the contour C belongs based on the features of the contour C.

9. The method for detecting the smoothness of a car curved screen according to claim 8, characterized in that: In S6, the continuous image information is input into a fluency detection algorithm for analysis, which specifically includes the following steps: S61, if the continuous image information contains noise interference information, applying Gaussian blur to the continuous image to remove high-frequency noise; or removing salt and pepper noise by median filtering; S62, if there is artifact information in the continuous image information, performing local comparison on the artifact area; S63, setting a pixel difference threshold, a constant area threshold, and a change area threshold to distinguish and define the change area and change situation of two consecutive image frames: if the frame difference D(x, y) is greater than the pixel difference threshold, it is considered that the pixel has changed in the two consecutive image frames, and the adjacent pixels that are greater than the set pixel difference threshold with the pixel are merged and defined as the change area; Otherwise, the pixel is considered unchanged in two consecutive frames of images, and the adjacent pixels whose difference with the pixel is less than the set pixel difference threshold are merged and defined as the unchanged area; If the area of ​​the changing region is between the constant area threshold and the changing area threshold, and such a stable change relationship is maintained between multiple frames of continuous images, it means that the changing region is regarded as a normal dynamic change, indicating that the sliding smoothness is high; If the area of ​​the changed region is larger than the change area threshold, and such a stable change relationship is maintained between multiple frames of continuous images, it means that the changed region indicates frame rate fluctuation or some image quality problems; If the area of ​​the unchanged region is greater than the unchanged area threshold, there is no change between multiple consecutive image frames, which means that the unchanged region does not produce a gradual change effect between every two consecutive image frames in a certain time period. The response time during this period is not high enough, and there may be a freeze phenomenon.

Images