A high-precision assembly and calibration system for display screen robots

Abstract

The present invention belongs to the technical field of assembly and calibration, and specifically discloses a high-precision assembly and calibration system for a display screen robot, the system comprising: a display screen robot, an assembly guidance module, an image matching module and a screen calibration module. The present invention introduces pixel displacement correction and pixel color correction, and by judging the matching similarity between the user's assembly effect and the standard assembly effect step by step, accurately locates and promptly corrects the wrong steps, and ensures the accurate assembly of multi-display screen devices; by evaluating the distribution of the user's visual attention through eye tracking, the user's viewing needs and usage habits are fully considered, the interactivity between the user and the multi-screen device is improved, and a more personalized user experience is created.

Description

Technical Field

[0001] The invention relates to the technical field of assembly and calibration, and in particular to a high-precision assembly and calibration system for a display screen robot. Background Art

[0002] A display screen robot high-precision assembly and calibration system refers to a system that uses computer vision technology to provide assembly guidance and calibration optimization for multi-display screen equipment. Display screens are increasingly used in electronic products, and multi-display screen equipment requires high-precision assembly and calibration to ensure the display effect of the display screen.

[0003] Existing traditional assembly solutions lack a customized assembly guidance method for multi-display devices;

[0004] Among the existing similar solutions, for example, CN111240567B discloses a method for adjusting the angle of a display screen and an electronic device. This solution aims to solve the technical problem that users usually adjust the folding angle of the display screen manually in the prior art. By calculating the angle of a stylus pen for angle compensation, a method for adjusting the angle of a display screen is provided, which achieves the technical effect of improving the user experience. However, the display screen calibration method is relatively single, and does not combine the user's personalized viewing needs and usage habits, and the dynamic performance is poor.

[0005] Therefore, the traditional assembly and calibration system has the technical problem of lacking a customized assembly guidance method for multi-display devices; there is a technical problem that the display screen calibration method is relatively single, does not take into account the user's personalized screen viewing needs and usage habits, and has poor dynamics. Summary of the invention

[0006] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a display screen robot high-precision assembly and calibration system;

[0007] In view of the technical problem of lack of user-oriented assembly guidance method for multi-display screen devices, the present invention introduces pixel displacement correction and pixel color correction, and accurately locates and promptly corrects wrong steps by judging the matching similarity between the user assembly effect and the standard assembly effect step by step, thereby ensuring accurate assembly of multi-display screen devices; in view of the technical problem that the display screen calibration method is relatively single, does not combine the user's personalized screen viewing needs and usage habits, and has poor dynamics, the present invention evaluates the distribution of the user's visual attention through eye tracking, fully considers the user's screen viewing needs and usage habits, improves the interactivity between the user and the multi-screen device, and creates a more personalized user experience;

[0008] The technical solution adopted by the present invention is as follows: The present invention provides a high-precision assembly and calibration system for a display screen robot, the high-precision assembly and calibration system for a display screen robot comprising a display screen robot, an assembly guidance module, an image matching module and a screen calibration module; the screen calibration module comprises a head-mounted display and a camera;

[0009] The display screen robot specifically refers to a screen device composed of multiple display screens;

[0010] The assembly guidance module uses an assembly effect verification method to collect the expected assembly effect image of the display screen robot and the user assembly image to verify whether the user's assembly operation of the display screen robot is correct;

[0011] The image matching module uses a multi-scale SSIM evaluation method combined with displacement color correction on the expected assembly effect image and the user assembly image to evaluate the similarity between the assembly steps of the display screen robot and the expected assembly effect, obtain the assembly effect index, and send the assembly effect index to the assembly guidance module;

[0012] The screen calibration module uses a spatiotemporal gaze visualization method to adjust the relative positions and pitch angles of all display screens of the display screen robot.

[0013] The assembly guidance module uses an assembly effect verification method, and the assembly effect verification method specifically includes the following steps:

[0014] Step A1: extracting standards, specifically, importing the display screen robot assembly manual into the assembly guidance module, and the assembly guidance module extracts the expected assembly effect image from the display screen robot assembly manual;

[0015] Step A2: User assembly, specifically, the user takes a picture of the user assembly image of the current step q, and uploads the user assembly image to the image matching module;

[0016] The user assembly image specifically refers to the effect image of the user following the display robot assembly instructions to the qth step assembly stage;

[0017] Step A3: Data transmission, specifically, the assembly guidance module transmits the expected assembly effect image to the image matching module, the image matching module obtains the assembly effect index according to the user assembly image and the expected assembly effect image, and the image matching module transmits the assembly effect index to the assembly guidance module:

[0018] Step A4: effect verification. If the assembly effect index is lower than the preset threshold, the assembly step is not performed correctly and the verification fails. The user is prompted to re-execute the qth assembly step and repeat step A2.

[0019] If the assembly effect index is greater than or equal to the preset threshold, the assembly step is executed correctly and the verification is successful. The user continues to execute the q+1th assembly step and repeats step A2 until all assembly steps in the display robot assembly manual are completed.

[0020] The image matching module uses a multi-scale SSIM evaluation method combined with displacement color correction, and the multi-scale SSIM evaluation method combined with displacement color correction specifically includes the following steps:

[0021] Step B1: Selecting the region of interest. Specifically, the image matching module uses contour detection technology to divide the region of interest in the expected assembly effect image and the user assembly image, and uses pixel frames of different scales to select the region of interest in the user assembly image.

[0022] Step B2: Offset correction. The specific steps are as follows:

[0023] Step B21: Calculate local statistics, specifically, use the SSIM index calculation method to calculate the local mean A and local standard deviation A of the expected assembly effect image, and record the expected assembly effect image as image A;

[0024] Step B22: pixel matching, specifically, using a block matching algorithm to find the best matching pixel points between the user assembly image and the expected assembly effect image, and calculating the pixel offsets in the horizontal and vertical directions according to the relative positions of the best matching pixel points;

[0025] Step B23: offset calculation, specifically, record the user assembly image as image B, correct the offset of the user assembly image relative to the expected assembly effect image, and calculate the horizontal and vertical coordinates of the pixels after the offset correction. The formula used is as follows:

[0026] ;

[0027] In the formula, represents the horizontal and vertical coordinates of the pixels in image B after offset correction, and Respectively represent the original horizontal and vertical coordinates of the pixels in image B, represents the pixel offset in the horizontal direction, represents the pixel offset of the pixel in the vertical direction;

[0028] Step B24: channel division, specifically, converting the image B into a YUV color space, where each pixel consists of three component channels, namely, a brightness value, a U component, and a V component;

[0029] Step B25: mean value calculation, specifically, according to the horizontal and vertical coordinates of the pixels after offset correction, a neighborhood weighted method is used to calculate the local mean value B of the image B. The formula used is as follows:

[0030] μ C B (x, y) = ∑ i = x - k x + k ∑ j = y - k y + k [ω(i - x, j - y) · B c (i', j')] ;

[0031] In the formula, represents the local mean B of image B, The coordinates are The neighborhood area of ​​the pixel, represents the neighborhood radius, represents the Gaussian weighting function, Represents the value of each component channel after offset correction;

[0032] Step B26: Standard deviation calculation, calculate the local standard deviation B of image B, the formula used is as follows:

[0033] σ c B (x, y) = ∑ i = x - k x + k ∑ j = y - k y + k {[ω(i - x, j - y) · B c (i', j') 2 ] - ( μ C B (x, y)) 2 } ;

[0034] In the formula, represents the local standard deviation B;

[0035] Step B3: Similarity estimation: estimate the similarity between image A and image B from the three aspects of structure, brightness value and contrast. The specific steps are as follows:

[0036] Step B31: Covariance calculation is used to measure the linear correlation between image A and image B. Specifically, the covariance between image A and image B is calculated based on the offset-corrected pixel horizontal and vertical coordinates. The formula used is as follows:

[0037] ;

[0038] In the formula, represents the covariance, Represents the value of each component channel of image A, Represents the value of each component channel of image B after offset correction. and Represent the local mean A of image A and the local mean B of image B respectively;

[0039] Step B32: Structural similarity metric calculation, the formula used is as follows:

[0040] ;

[0041] In the formula, represents the structural similarity measure, represents the covariance between image A and image B, represents the local standard deviation A of image A, represents the local standard deviation B of image B, , Indicates the bit depth;

[0042] Step B33: color correction, specifically, calculating the global color offset between image A and image B according to the values ​​of the horizontal coordinate, vertical coordinate and component channel of the pixel after the offset correction;

[0043] Step B34: Calculate the brightness value similarity. Calculate the brightness value similarity based on the global color offset between image A and image B. The formula used is as follows:

[0044] ;

[0045] In the formula, represents the similarity of brightness values, and Represent the local mean A of image A and the local mean B of image B, respectively. Represents the global color offset, , Indicates the bit depth;

[0046] Step B35: Calculate the contrast similarity using the following formula:

[0047] ;

[0048] In the formula, represents contrast similarity, and They represent the local standard deviation A of image A and the local standard deviation B of image B, respectively. , Indicates the bit depth;

[0049] Step B36: Calculate the comprehensive quality score using the following formula:

[0050] ;

[0051] In the formula, Represents the comprehensive quality score. , and Respectively represent weight parameters;

[0052] Step B37: Channel integration is used to integrate the local calculation results of all pixel frames. Specifically, the structural similarity indexes of the three component channels are calculated respectively, and the structural similarity indexes are weighted averaged. The image matching module transmits the weighted average result as the assembly effect index to the assembly guidance module. The calculation formula of the structural similarity index is as follows:

[0053] ;

[0054] In the formula, represents the structural similarity index, and Represents the image width and height respectively, It means to traverse all pixels and find the cumulative sum.

[0055] The screen calibration module adopts a spatiotemporal gaze visualization method, and the spatiotemporal gaze visualization method specifically includes the following steps:

[0056] Step C1: Gaze point capture, specifically, the screen calibration module uses a head mounted display, adopts simultaneous positioning and mapping technology and eye tracking technology to capture the participant's gaze point;

[0057] Step C2: positioning the display screen, specifically, the screen calibration module uses a camera to collect an environment image, and the environment image specifically refers to an image of the display screen robot that is completely captured and not blocked;

[0058] Step C3: unifying the coordinate system, specifically, integrating and mapping the gaze point and the environment image into a unified spatial coordinate system to obtain a gaze plane map;

[0059] Step C4: constructing a cube, specifically, constructing a three-dimensional cube, wherein the xz plane of the cube represents the gaze plane graph, and the y axis of the cube represents time;

[0060] Step C5: drawing a gaze trajectory, specifically, in the cube, connecting the gaze points in the cube time by time to draw the gaze trajectory;

[0061] Step C6: area division, specifically, dividing the display screen area of ​​the display screen robot in the plane view and numbering all the display screen areas;

[0062] Step C7: quantify the influence. Specifically, use the kernel density estimation method to smooth the gaze trajectory, record the number of gaze points belonging to each display area, and calculate the influence of each display. The formula used is as follows:

[0063] ;

[0064] In the formula, It represents the influence of the fixation point in any numbered display area on the center position of the display. represents the cumulative sum of the influence on the gaze points belonging to the display screen, The coordinates of the center position of the display screen, Indicates The coordinates of the gaze point, represents the scaling factor;

[0065] Step C8: Color coding, using Method, shades and renders the display screen area in the cube according to the influence;

[0066] Step C9: The screen calibration module adjusts the relative positions and pitch angles of all display screens of the display robot according to the colored cube.

[0067] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0068] (1) In order to solve the technical problem of lack of customized assembly guidance method for multi-display devices, the present invention introduces pixel displacement correction and pixel color correction. By judging the matching similarity between the user assembly effect and the standard assembly effect step by step, the wrong steps are accurately located and corrected in time, thereby ensuring the accurate assembly of the multi-display device.

[0069] (2) In order to address the technical problems that the display screen calibration method is relatively simple, does not take into account the user's personalized screen viewing needs and usage habits, and has poor dynamics, the present invention uses eye tracking to evaluate the distribution of the user's visual attention, fully considers the user's screen viewing needs and usage habits, improves the interactivity between the user and multi-screen devices, and creates a more personalized user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A module connection diagram of a display screen robot high-precision assembly and calibration system provided by the present invention.

[0071] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0073] Example 1: See Figure 1 , this embodiment provides a display screen robot high-precision assembly and calibration system, the display screen robot high-precision assembly and calibration system includes a display screen robot, an assembly guidance module, an image matching module and a screen calibration module; the screen calibration module includes a head-mounted display and a camera;

[0074] The display screen robot specifically refers to a screen device composed of multiple display screens;

[0075] The assembly guidance module uses an assembly effect verification method to collect the expected assembly effect image of the display screen robot and the user assembly image to verify whether the user's assembly operation of the display screen robot is correct;

[0076] The image matching module uses a multi-scale SSIM evaluation method combined with displacement color correction on the expected assembly effect image and the user assembly image to evaluate the similarity between the assembly steps of the display screen robot and the expected assembly effect, obtain the assembly effect index, and send the assembly effect index to the assembly guidance module;

[0077] The screen calibration module uses a spatiotemporal gaze visualization method to adjust the relative positions and pitch angles of all display screens of the display screen robot.

[0078] Example 2: See Figure 1 This embodiment is based on the above embodiment, and the assembly guidance module uses an assembly effect verification method, and the assembly effect verification method specifically includes the following steps:

[0079] Step A1: extracting standards, specifically, importing the display screen robot assembly manual into the assembly guidance module, and the assembly guidance module extracts the expected assembly effect image from the display screen robot assembly manual;

[0080] Step A2: User assembly, specifically, the user takes a picture of the user assembly image of the current step q, and uploads the user assembly image to the image matching module;

[0081] The user assembly image specifically refers to the effect image of the user following the display robot assembly instructions to the qth step assembly stage;

[0082] Step A3: Data transmission, specifically, the assembly guidance module transmits the expected assembly effect image to the image matching module, the image matching module obtains the assembly effect index according to the user assembly image and the expected assembly effect image, and the image matching module transmits the assembly effect index to the assembly guidance module:

[0083] Step A4: effect verification. If the assembly effect index is lower than the preset threshold, the assembly step is not performed correctly and the verification fails. The user is prompted to re-execute the qth assembly step and repeat step A2.

[0084] If the assembly effect index is greater than or equal to the preset threshold, the assembly step is executed correctly and the verification is successful. The user continues to execute the q+1th assembly step and repeats step A2 until all assembly steps in the display robot assembly manual are completed.

[0085] Example 3: See Figure 1 This embodiment is based on the above embodiment. The image matching module uses a multi-scale SSIM evaluation method combined with displacement color correction. The multi-scale SSIM evaluation method combined with displacement color correction specifically includes the following steps:

[0086] Step B1: Selecting the region of interest. Specifically, the image matching module uses contour detection technology to divide the region of interest in the expected assembly effect image and the user assembly image, and uses pixel frames of different scales to select the region of interest in the user assembly image.

[0087] Step B2: Offset correction. The specific steps are as follows:

[0088] Step B21: Calculate local statistics, specifically, use the SSIM index calculation method to calculate the local mean A and local standard deviation A of the expected assembly effect image, and record the expected assembly effect image as image A;

[0089] Step B22: pixel matching, specifically, using a block matching algorithm to find the best matching pixel points between the user assembly image and the expected assembly effect image, and calculating the pixel offsets in the horizontal and vertical directions according to the relative positions of the best matching pixel points;

[0090] Step B23: offset calculation, specifically, record the user assembly image as image B, correct the offset of the user assembly image relative to the expected assembly effect image, and calculate the horizontal and vertical coordinates of the pixels after the offset correction. The formula used is as follows:

[0091] ;

[0092] In the formula, represents the horizontal and vertical coordinates of the pixels in image B after offset correction, and Respectively represent the original horizontal and vertical coordinates of the pixels in image B, represents the pixel offset in the horizontal direction, represents the pixel offset of the pixel in the vertical direction;

[0093] Step B24: channel division, specifically, converting the image B into a YUV color space, where each pixel consists of three component channels, namely, a brightness value, a U component, and a V component;

[0094] Step B25: mean value calculation, specifically, according to the horizontal and vertical coordinates of the pixels after offset correction, a neighborhood weighted method is used to calculate the local mean value B of the image B. The formula used is as follows:

[0095] μ C B (x, y) = ∑ i = x - k x + k ∑ j = y - k y + k [ω(i - x, j - y) · B c (i', j')] ;

[0096] In the formula, represents the local mean B of image B, The coordinates are The neighborhood area of ​​the pixel, represents the neighborhood radius, represents the Gaussian weighting function, Represents the value of each component channel after offset correction;

[0097] Step B26: Standard deviation calculation, calculate the local standard deviation B of image B, the formula used is as follows:

[0098] σ c B (x, y) = ∑ i = x - k x + k ∑ j = y - k y + k {[ω(i - x, j - y) · B c (i', j') 2 ] - ( μ C B (x, y)) 2 } ;

[0099] In the formula, represents the local standard deviation B;

[0100] Step B3: Similarity estimation: estimate the similarity between image A and image B from the three aspects of structure, brightness value and contrast. The specific steps are as follows:

[0101] Step B31: Covariance calculation is used to measure the linear correlation between image A and image B. Specifically, the covariance between image A and image B is calculated based on the offset-corrected pixel horizontal and vertical coordinates. The formula used is as follows:

[0102] ;

[0103] In the formula, represents the covariance, Represents the value of each component channel of image A, Represents the value of each component channel of image B after offset correction. and Represent the local mean A of image A and the local mean B of image B respectively;

[0104] Step B32: Structural similarity metric calculation, the formula used is as follows:

[0105] ;

[0106] In the formula, represents the structural similarity measure, represents the covariance between image A and image B, represents the local standard deviation A of image A, represents the local standard deviation B of image B, , Indicates the bit depth;

[0107] Step B33: color correction, specifically, calculating the global color offset between image A and image B according to the values ​​of the horizontal coordinate, vertical coordinate and component channel of the pixel after the offset correction;

[0108] Step B34: Calculate the brightness value similarity. Calculate the brightness value similarity based on the global color offset between image A and image B. The formula used is as follows:

[0109] ;

[0110] In the formula, Indicates the similarity of brightness values, and Represent the local mean A of image A and the local mean B of image B, respectively. Represents the global color offset, , Indicates the bit depth;

[0111] Step B35: Calculate the contrast similarity using the following formula:

[0112] ;

[0113] In the formula, represents contrast similarity, and They represent the local standard deviation A of image A and the local standard deviation B of image B, respectively. , Indicates the bit depth;

[0114] Step B36: Calculate the comprehensive quality score using the following formula:

[0115] ;

[0116] In the formula, Represents the comprehensive quality score. , and Respectively represent weight parameters;

[0117] Step B37: Channel integration is used to integrate the local calculation results of all pixel frames. Specifically, the structural similarity indexes of the three component channels are calculated respectively, and the structural similarity indexes are weighted averaged. The image matching module transmits the weighted average result as the assembly effect index to the assembly guidance module. The calculation formula of the structural similarity index is as follows:

[0118] ;

[0119] In the formula, represents the structural similarity index, and Represents the image width and height respectively, It means to traverse all pixels and find the cumulative sum.

[0120] Example 4: See Figure 1 This embodiment is based on the above embodiment. The screen calibration module adopts a spatiotemporal gaze visualization method. The spatiotemporal gaze visualization method specifically includes the following steps:

[0121] Step C1: Gaze point capture, specifically, the screen calibration module uses a head mounted display, adopts simultaneous positioning and mapping technology and eye tracking technology to capture the participant's gaze point;

[0122] Step C2: positioning the display screen, specifically, the screen calibration module uses a camera to collect an environment image, and the environment image specifically refers to an image of the display screen robot that is completely captured and not blocked;

[0123] Step C3: unifying the coordinate system, specifically, integrating and mapping the gaze point and the environment image into a unified spatial coordinate system to obtain a gaze plane map;

[0124] Step C4: constructing a cube, specifically, constructing a three-dimensional cube, wherein the xz plane of the cube represents the gaze plane graph, and the y axis of the cube represents time;

[0125] Step C5: drawing a gaze trajectory, specifically, in the cube, connecting the gaze points in the cube time by time to draw the gaze trajectory;

[0126] Step C6: area division, specifically, dividing the display screen area of ​​the display screen robot in the plane view and numbering all the display screen areas;

[0127] Step C7: quantify the influence. Specifically, use the kernel density estimation method to smooth the gaze trajectory, record the number of gaze points belonging to each display area, and calculate the influence of each display. The formula used is as follows:

[0128] ;

[0129] In the formula, It represents the influence of the fixation point in any numbered display area on the center position of the display. represents the cumulative sum of the influence on the gaze points belonging to the display screen, The coordinates of the center position of the display screen, Indicates The coordinates of the gaze point, represents the scaling factor;

[0130] Step C8: Color coding, using Method, shades and renders the display screen area in the cube according to the influence;

[0131] Step C9: The screen calibration module adjusts the relative positions and pitch angles of all display screens of the display robot according to the colored cube.

[0132] Embodiment 5: This embodiment is based on the above embodiment. In step A4, the preset threshold is 0.5.

[0133] Embodiment 6: This embodiment is based on the above embodiment. In step B1, the pixel frames of different scales specifically refer to rectangular frames of 20×20 pixels, 50×50 pixels, 100×100 pixels, 200×200 pixels and 500×500 pixels.

[0134] Embodiment 7: This embodiment is based on the above embodiment. In step B25, the neighborhood radius is 5.

[0135] Embodiment 8: This embodiment is based on the above embodiment. The display screen robot is specifically a combination screen composed of spliced ​​luminous screens of CN103744262B. The display screen robot can splice multiple small display screens in an array to form a large display screen.

[0136] Embodiment 9: This embodiment is based on the above embodiment. The display screen robot is specifically a multi-screen combination display of CN111219576B. The display screen robot can change the number of screens according to customer needs.

[0137] Embodiment 10: This embodiment is based on the above embodiment. In step C8, the color coding is specifically that the display screen area with greater influence in the cube is closer to red, and the display screen area with less influence in the cube is closer to blue.

[0138] Embodiment 11: This embodiment is based on the above embodiment. In step C6, the area division is specifically to identify the display screens in the gaze plan view, uniquely number all the display screens, and discard the image areas outside the display screens in the gaze plan view.

[0139] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0140] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0141] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A display screen robot high-precision assembly and calibration system, characterized in that: It includes a display screen robot, an assembly guidance module, an image matching module and a screen calibration module; the screen calibration module includes a head-mounted display and a camera; The display screen robot specifically refers to a screen device composed of multiple display screens; The assembly guidance module uses an assembly effect verification method to collect the expected assembly effect image of the display screen robot and the user assembly image to verify whether the user's assembly operation of the display screen robot is correct; The image matching module uses a multi-scale SSIM evaluation method combined with displacement color correction on the expected assembly effect image and the user assembly image to evaluate the similarity between the assembly steps of the display screen robot and the expected assembly effect, obtain the assembly effect index, and send the assembly effect index to the assembly guidance module; The screen calibration module uses a spatiotemporal gaze visualization method to adjust the relative positions and pitch angles of all display screens of the display screen robot.

2. A display screen robot high-precision assembly and calibration system according to claim 1, characterized in that: The assembly guidance module uses an assembly effect verification method, and the assembly effect verification method specifically includes the following steps: Step A1: extracting standards, specifically, importing the display screen robot assembly manual into the assembly guidance module, and the assembly guidance module extracts the expected assembly effect image from the display screen robot assembly manual; Step A2: User assembly, specifically, the user takes a picture of the user assembly image of the current step q, and uploads the user assembly image to the image matching module; The user assembly image specifically refers to the effect image of the user following the display robot assembly instructions to the qth step assembly stage; Step A3: Data transmission, specifically, the assembly guidance module transmits the expected assembly effect image to the image matching module, the image matching module obtains the assembly effect index according to the user assembly image and the expected assembly effect image, and the image matching module transmits the assembly effect index to the assembly guidance module: Step A4: effect verification. If the assembly effect index is lower than the preset threshold, the assembly step is not performed correctly and the verification fails. The user is prompted to re-execute the qth assembly step and repeat step A2. If the assembly effect index is greater than or equal to the preset threshold, the assembly step is executed correctly and the verification is successful. The user continues to execute the q+1th assembly step and repeats step A2 until all assembly steps in the display robot assembly manual are completed.

3. The high-precision display robot assembly and calibration system according to claim 2 is characterized in that: The image matching module uses a multi-scale SSIM evaluation method combined with displacement color correction, and the multi-scale SSIM evaluation method combined with displacement color correction specifically includes the following steps: Step B1: Selecting the region of interest. Specifically, the image matching module uses contour detection technology to divide the region of interest in the expected assembly effect image and the user assembly image, and uses pixel frames of different scales to select the region of interest in the user assembly image. Step B2: Offset correction. The specific steps are as follows: Step B21: Calculate local statistics, specifically, use the SSIM index calculation method to calculate the local mean A and local standard deviation A of the expected assembly effect image, and record the expected assembly effect image as image A; Step B22: pixel matching, specifically, using a block matching algorithm to find the best matching pixel points between the user assembly image and the expected assembly effect image, and calculating the pixel offsets in the horizontal and vertical directions according to the relative positions of the best matching pixel points; Step B23: offset calculation, specifically, record the user assembly image as image B, correct the offset of the user assembly image relative to the expected assembly effect image, and calculate the horizontal and vertical coordinates of the pixels after the offset correction. The formula used is as follows: ； In the formula, represents the horizontal and vertical coordinates of the pixels in image B after offset correction, and Respectively represent the original horizontal and vertical coordinates of the pixels in image B, represents the pixel offset in the horizontal direction, represents the pixel offset of the pixel in the vertical direction; Step B24: channel division, specifically, converting the image B into a YUV color space, where each pixel consists of three component channels, namely, a brightness value, a U component, and a V component; Step B25: mean value calculation, specifically, calculating the local mean value B of the image B using a neighborhood weighted method according to the offset-corrected pixel horizontal and vertical coordinates; Step B26: standard deviation calculation, calculating the local standard deviation B of image B; Step B3: Similarity estimation: estimate the similarity between image A and image B from the three aspects of structure, brightness value and contrast. The specific steps are as follows: Step B31: Covariance calculation is used to measure the linear correlation between image A and image B. Specifically, the covariance between image A and image B is calculated based on the offset-corrected pixel horizontal and vertical coordinates. The formula used is as follows: ； In the formula, represents the covariance, The coordinates are The neighborhood area of ​​the pixel, represents the neighborhood radius, represents the Gaussian weighting function, Represents the value of each component channel of image A, Represents the value of each component channel of image B after offset correction. and Represent the local mean A of image A and the local mean B of image B respectively; Step B32: Structural similarity measurement calculation; Step B33: color correction, specifically, calculating the global color offset between image A and image B according to the values ​​of the horizontal coordinate, vertical coordinate and component channel of the pixel after the offset correction; Step B34: calculating brightness value similarity, and calculating brightness value similarity according to the global color offset between image A and image B; Step B35: Calculate contrast similarity; Step B36: Calculate the comprehensive quality score; Step B37: Channel integration is used to integrate the local calculation results of all pixel frames. Specifically, the structural similarity indexes of the three component channels are calculated respectively, and the structural similarity indexes are weighted averaged. The image matching module transmits the weighted average result as an assembly effect indicator to the assembly guidance module.

4. The high-precision display screen robot assembly and calibration system according to claim 3 is characterized in that: The screen calibration module adopts a spatiotemporal gaze visualization method, and the spatiotemporal gaze visualization method specifically includes the following steps: Step C1: Gaze point capture, specifically, the screen calibration module uses a head mounted display, adopts simultaneous positioning and mapping technology and eye tracking technology to capture the participant's gaze point; Step C2: positioning the display screen, specifically, the screen calibration module uses a camera to collect an environment image, and the environment image specifically refers to an image of the display screen robot that is completely captured and not blocked; Step C3: unifying the coordinate system, specifically, integrating and mapping the gaze point and the environment image into a unified spatial coordinate system to obtain a gaze plane map; Step C4: constructing a cube, specifically, constructing a three-dimensional cube, wherein the xz plane of the cube represents the gaze plane graph, and the y axis of the cube represents time; Step C5: drawing a gaze trajectory, specifically, in the cube, connecting the gaze points in the cube time by time to draw the gaze trajectory; Step C6: area division, specifically, dividing the display screen area of ​​the display screen robot in the plane view and numbering all the display screen areas; Step C7: quantify the influence. Specifically, use the kernel density estimation method to smooth the gaze trajectory, record the number of gaze points belonging to each display area, and calculate the influence of each display. The formula used is as follows: ； In the formula, It represents the influence of the fixation point in any numbered display area on the center position of the display. represents the cumulative sum of the influence on the gaze points belonging to the display screen, The coordinates of the center position of the display screen, Indicates The coordinates of the gaze point, represents the scaling factor; Step C8: Color coding, using Method, shades and renders the display screen area in the cube according to the influence; Step C9: The screen calibration module adjusts the relative positions and pitch angles of all display screens of the display robot according to the colored cube.

Images