Eye Box Center Positioning Method and Eye Box Center Positioning System

Abstract

This invention discloses an eye box center positioning method and an eye box center positioning system. The eye box center positioning method includes the following steps: (a) determining whether the posture feature value of the AR device meets a preset posture threshold based on a posture test image projected by the optical module of an AR device captured by an industrial camera, wherein when the posture feature value of the AR device does not meet the preset posture threshold, the posture of the AR device is adjusted in the w, u and / or v directions; and (b) determining whether the position grayscale feature value of the AR device meets a preset grayscale threshold based on a position test image projected by the optical module of the AR device captured by the industrial camera, wherein when the position grayscale feature value of the AR device does not meet the preset grayscale threshold, the position of the AR device is adjusted in the x and / or z directions, thereby positioning the center of the eye box and the center of the industrial camera, and improving alignment efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to near-eye display devices, and particularly to a method and system for centering an eye-box. Background Technology

[0002] In recent years, AR (Augmented Reality) technology has matured, and AR-based products (such as AR glasses) have become increasingly popular. The eye box is a key concept in AR devices; it refers to the cone-shaped area between the device's optical module and the user's eyeball, which is also the area where the displayed content is clearest. When a user wears an AR device, if the center of the eye box is aligned with the center of the user's receiving eye, a perfect image can be obtained, helping the user achieve a good visual experience. Therefore, AR devices need to be tested and calibrated before leaving the factory to ensure that the center of the user's receiving eye is aligned as closely as possible with the center of the eye box when the user wears the AR device. This prevents deviations in brightness, brightness uniformity, contrast, field of view, distortion, and resolution caused by eye box center offset. Currently, when testing AR devices, testers manually fix the AR devices to the fixture and manually adjust the relative positions of the AR devices and industrial cameras (devices used to simulate human eyes) to test and calibrate the eye box parameters of the AR devices using images captured by the industrial cameras. This method is easily affected by the subjective consciousness of the testers, resulting in poor calibration results for the AR devices and poor consistency when testing and calibrating batches of AR devices. Summary of the Invention

[0003] One object of the present invention is to provide an eye box center positioning method and an eye box center positioning system, wherein when an AR device is being tested and calibrated, the eye box center positioning method can improve the accuracy of the alignment between the eye box center and the center of the industrial camera, thereby improving the accuracy of testing and calibration.

[0004] One object of the present invention is to provide an eye box center positioning method and an eye box center positioning system, wherein when the AR device is being tested and calibrated, the eye box center positioning method can automatically align the center of the eye box with the center of the industrial camera, thereby reducing the interference of human factors of testers and improving the accuracy of testing and calibration.

[0005] One object of the present invention is to provide an eye box center positioning method and an eye box center positioning system, wherein when the AR device is being tested and calibrated, the eye box center positioning method can automatically align the center of the eye box with the center of the industrial camera, thereby ensuring the consistency of optical parameters of the AR devices in subsequent batches.

[0006] One object of the present invention is to provide an eye box center positioning method and an eye box center positioning system, wherein the eye box center positioning method can improve testing and calibration efficiency, making the eye box center positioning method particularly suitable for testing and calibrating batches of the AR devices.

[0007] One object of the present invention is to provide an eye box center positioning method and an eye box center positioning system, wherein the eye box center positioning method can determine the offset between the center of the eye box and the center of the industrial camera based on the posture test map and the position test map captured by the industrial camera, and adjust the posture and position of the AR device based on the offset to automatically align the center of the eye box and the center of the industrial camera, thereby improving alignment efficiency and accuracy.

[0008] According to one aspect of the present invention, the present invention provides a method for centering an eye box, wherein the method for centering an eye box includes the following steps:

[0009] (a) Based on a posture test image projected by the optical module of an AR device captured by an industrial camera, determine whether the posture feature value of the AR device meets a preset posture threshold, wherein when the posture feature value of the AR device does not meet the preset posture threshold, adjust the posture of the AR device in the w direction, u direction, and / or v direction; and

[0010] (b) Based on a position test map projected by the optical module of the AR device captured by the industrial camera, determine whether the position grayscale feature value of the AR device meets a preset grayscale threshold. When the position grayscale feature value of the AR device does not meet the preset grayscale threshold, adjust the position of the AR device in the x direction and / or z direction to locate the center of the eye box and the center of the industrial camera.

[0011] According to an embodiment of the present invention, step (a) further includes the following steps:

[0012] (a.1) A first preset attitude threshold R for preset angle rotation, a second preset attitude threshold Y for left and right swaying, and a third preset attitude threshold P for pitch oscillation;

[0013] (a.2) Based on the posture test image captured by the industrial camera, obtain the posture feature values ​​Roll, Yaw, and Pitch of the AR device; and

[0014] (a.3) Compare the magnitudes of |Roll| and R, |Yaw| and Y, and |Pitch| and P respectively. If the posture feature value of the AR device does not satisfy |Roll|≤R, |Yaw|≤Y, and |Pitch|≤P, then it is determined that the posture feature value of the AR device does not satisfy the preset posture threshold.

[0015] According to one embodiment of the present invention, in step (a.3), if |Roll|>R, the posture of the AR device is adjusted in the v direction; if |Yaw|>Y, the posture of the AR device is adjusted in the u direction; if |Pitch|>P, the posture of the AR device is adjusted in the w direction.

[0016] According to an embodiment of the present invention, in the above method, when Roll>R, the posture of the AR device is adjusted in the v+ direction; when Roll<-R, the posture of the AR device is adjusted in the v- direction; when Yaw>Y, the posture of the AR device is adjusted in the u+ direction; when Yaw<-Y, the posture of the AR device is adjusted in the u- direction; when Pitch>P, the posture of the AR device is adjusted in the w+ direction; when Pitch<-P, the posture of the AR device is adjusted in the w- direction.

[0017] According to one embodiment of the present invention, the attitude test diagram is a cross-shaped diagram.

[0018] According to one embodiment of the present invention, the position test map is a uniform pure white image.

[0019] According to an embodiment of the present invention, step (b) further includes the following steps:

[0020] (b.1) The preset grayscale threshold at the center of the preset eye box;

[0021] (b.2) Divide the position test map captured by the industrial camera into an N×M grid array;

[0022] (b.3) Obtain the grayscale feature values ​​GH(A), GH(B), GH(C), and GH(D) of the four corner regions A, B, C, and D of the N×M grid array; and

[0023] (b.4) Compare (Max(GH(AD))-Min(GH(AD))) and G. If GH(A), GH(B), GH(C) and GH(D) do not satisfy (Max(GH(AD))-Min(GH(AD))) / Max(GH(AD))≤G, then it is determined that the position grayscale feature value of the AR device does not satisfy the preset grayscale threshold.

[0024] According to an embodiment of the present invention, in the step (b.4), if GH(A)>GH(C) and GH(A)>GH(B), then adjust the position of the AR device 100 in the x+ direction and the z- direction; if GH(A)>GH(C) and GH(A)<GH(B), then adjust the position of the AR device 100 in the x- direction and the z- direction; if GH(A)<GH(C) and GH(A)>GH(B), then adjust the position of the AR device 100 in the x+ direction and the z+ direction; if GH(A)<GH(C) and GH(A)<GH(B), then adjust the position of the AR device 100 in the x- direction and the z+ direction.

[0025] According to an embodiment of the present invention, before the step (a), the eye box center positioning method further includes the step of: (c) calibrating the uniformity of the industrial camera.

[0026] According to an embodiment of the present invention, the step (c) further includes the steps of:

[0027] (c.1) Close to a uniform surface light source to the lens of the industrial camera, to allow the industrial camera to capture an image of the surface light source;

[0028] (c.2) Take the gray value of the radius area of 0.1%-0.2% of the image center as the calibration reference gray level; and (c.3) Based on the calibration reference gray level, calibrate the gray level at each pixel in the image to the calibration reference gray level to complete the uniformity calibration of the industrial camera.

[0029] According to another aspect of the present invention, the present invention further provides an eye box center positioning system, which includes:

[0030] An image acquisition unit, wherein the image acquisition unit includes an industrial camera;

[0031] A product moving unit, wherein the product moving unit includes a robotic arm, and the industrial camera and the robotic arm are disposed adjacent to each other;

[0032] An analysis and processing unit, wherein the analysis and processing unit is connected to the industrial camera; and

[0033] A control unit, wherein the industrial camera, the robotic arm and the analysis and processing unit are respectively connected to the control unit, and the control unit is configured to:

[0034] (A) Control an optical module of an AR device installed on the robotic arm to sequentially project a pose test pattern and a position test pattern;

[0035] (B) Control the industrial camera to sequentially capture the attitude test diagram and the position test diagram;

[0036] (C) Based on the posture test image captured by the industrial camera, determine whether the posture feature value of the AR device meets a preset posture threshold, wherein when the posture feature value of the AR device does not meet the preset posture threshold, control the robotic arm to adjust the posture of the AR device in the w direction, u direction, and / or v direction; and

[0037] (D) Based on the position test map captured by the industrial camera, determine whether the position grayscale feature value of the AR device meets a preset grayscale threshold, wherein when the position grayscale feature value of the AR device does not meet the preset grayscale threshold, control the robotic arm to adjust the position of the AR device in the x direction and / or z direction.

[0038] According to one embodiment of the present invention, the attitude test diagram is a cross-shaped diagram.

[0039] According to one embodiment of the present invention, the position test map is a uniform pure white image.

[0040] According to one embodiment of the present invention, the analysis and processing unit is configured as follows:

[0041] Based on the posture test image captured by the industrial camera, obtain the posture feature values ​​Roll, Yaw, and Pitch of the AR device; and

[0042] Compare the values ​​of |Roll| and R, |Yaw| and Y, and |Pitch| and P respectively. If the posture feature value of the AR device does not satisfy |Roll|≤R, |Yaw|≤Y, and |Pitch|≤P, then it is determined that the posture feature value of the AR device does not satisfy the preset posture threshold.

[0043] According to one embodiment of the present invention, the analysis and processing unit is further configured to: if |Roll|>R, then adjust the posture of the AR device in the v direction; if |Yaw|>Y, then adjust the posture of the AR device in the u direction; if |Pitch|>P, then adjust the posture of the AR device in the w direction.

[0044] According to one embodiment of the present invention, when Roll>R, the posture of the AR device is adjusted in the v+ direction; when Roll<-R, the posture of the AR device is adjusted in the v- direction; when Yaw>Y, the posture of the AR device is adjusted in the u+ direction; when Yaw<-Y, the posture of the AR device is adjusted in the u- direction; when Pitch>P, the posture of the AR device is adjusted in the w+ direction; when Pitch<-P, the posture of the AR device is adjusted in the w- direction.

[0045] According to an embodiment of the present invention, the analysis and processing unit is configured to:

[0046] Divide the position test diagram captured by the industrial camera into an N×M grid array;

[0047] Obtain the gray-scale feature values GH(A), GH(B), GH(C), and GH(D) of regions A, B, C, and D at the four corners of the N×M grid array; and

[0048] Compare (Max(GH(A - D)) - Min(GH(A - D))) and G. If GH(A), GH(B), GH(C), and GH(D) do not satisfy (Max(GH(A - D)) - Min(GH(A - D))) / Max(GH(A - D)) ≤ G, it is determined that the position gray-scale feature value of the AR device does not satisfy the preset gray-scale threshold.

[0049] According to an embodiment of the present invention, the analysis and processing unit is further configured to: If GH(A) > GH(C) and GH(A) > GH(B), adjust the position of the AR device 100 in the x+ direction and the z- direction; if GH(A) > GH(C) and GH(A) < GH(B), adjust the position of the AR device 100 in the x- direction and the z- direction; if GH(A) < GH(C) and GH(A) > GH(B), adjust the position of the AR device in the x+ direction and the z+ direction; if GH(A) < GH(C) and GH(A) < GH(B), adjust the position of the AR device 100 in the x- direction and the z+ direction. Description of the Drawings

[0050] Figure 1 Shows a block diagram of an eye box center positioning system according to a preferred embodiment of the present invention.

[0051] Figure 2 Shows the specific structure of the eye box center positioning system according to the above preferred embodiment of the present invention.

[0052] Figure 3 Shows the positioning process of the eye box center positioning system according to the above preferred embodiment of the present invention.

[0053] Figure 4 Shows a posture test diagram of the eye box center positioning system according to the above preferred embodiment of the present invention.

[0054] Figure 5 Shows the posture positioning process of the eye box center positioning system according to the above preferred embodiment of the present invention.

[0055] Figure 6 A position test diagram of the eye box center positioning system according to the above-described preferred embodiment of the present invention is shown.

[0056] Figure 7 The positioning process of the eye box center positioning system according to the above preferred embodiment of the present invention is shown. Detailed Implementation

[0057] Before detailing any embodiment of the invention, it should be understood that the invention is not limited in its application to the details of the construction and arrangement of the components set forth in the following description or illustrated in the following figures. The invention can have other embodiments and can be practiced or carried out in various ways. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “comprising,” “including,” or “having,” and variations thereof is intended to cover the items set forth below and their equivalents, as well as any additional items. Unless otherwise specified or limited, the terms “installation,” “connection,” “support,” and “linkage,” and variations thereof are used broadly and cover both direct and indirect installation, connection, support, and linking. Moreover, “connection” and “linkage” are not limited to physical or mechanical connections or links.

[0058] Furthermore, firstly, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention. Secondly, the term "a" should be understood as "at least one" or "one or more," that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple. The term "a" should not be construed as a limitation on the quantity.

[0059] See attached diagrams in the reference manual. Figures 1 to 3 According to a preferred embodiment of the present invention, the eye-box center positioning system includes an image acquisition unit 10, a product movement unit 20, an analysis and processing unit 30, and a control unit 40. The image acquisition unit 10 includes an industrial camera 11, the product movement unit 20 includes a robotic arm 21, the industrial camera 11 and the robotic arm 21 are arranged adjacent to each other, the analysis and processing unit 30 is connected to the industrial camera 11, the control unit 40 is connected to the analysis and processing unit 30, and the industrial camera 11 and the robotic arm 21 are controllably connected to the control unit 40.

[0060] An AR device 100 can be mounted on the robotic arm 21 of the product moving unit 20, and the industrial camera 11 of the image acquisition unit 10 faces the AR device 100 so that the AR device 100 can simulate a user's receiving eye to receive the image (which can be a picture or video) projected by the optical module of the AR device 100. At this time, an eye box is formed on the front side of the industrial camera 11. After the AR device 100 is mounted on the robotic arm 21, the AR device 100 can be controllably connected to the control unit 40 so that the control unit 40 can control the operating state of the AR device 100.

[0061] It is worth mentioning that the way the AR device 100 is mounted on the robotic arm 21 is not limited in the eye box center positioning system of the present invention. For example, the robotic arm 21 may be provided with a clamping end for clamping the AR device 100.

[0062] Normally, the position of the industrial camera 11 of the image acquisition unit 10 remains fixed, while the state of the robotic arm 21 of the product moving unit 20 can be adjusted to adjust the posture and position of the AR device 100 mounted on the robotic arm 21. In this way, the center of the eye box can be adjusted so that the center of the eye box and the center of the industrial camera 11 can be aligned, thereby facilitating the subsequent testing and calibration of the optical parameters of the AR device 100.

[0063] To align the state of the robotic arm 21 of the product moving unit 20 with the center of the eye box and the center of the industrial camera 11, firstly, the AR device 100 mounted on the robotic arm 21 is powered on, allowing the optical module of the AR device 100 to project an image toward the industrial camera 11. Secondly, the industrial camera 11 captures the image projected by the optical module of the AR device 100, and the captured image is transmitted to the analysis and processing unit 30. Thirdly, the analysis and processing unit 30 performs image recognition, data processing, and data analysis on the image captured by the industrial camera 11 to determine the offset between the center of the eye box and the center of the industrial camera 11, and generates a control command based on this offset. Finally, the control unit 40 adjusts the state of the robotic arm 21 of the product moving unit 20 according to the control command to adjust the posture and / or position of the AR device 100 mounted on the robotic arm 21, thus aligning it with the center of the eye box and the center of the industrial camera 11, thereby facilitating subsequent testing and calibration of the optical parameters of the AR device 100.

[0064] Preferably, the control unit 40 can control the robotic arm 21 of the product moving unit 20 to perform six degrees of freedom of attitude and / or position adjustment according to the control command, thereby adjusting the position and attitude of the AR device 100 installed on the robotic arm 21.

[0065] Reference Appendix Figure 6 and Figure 7 The working process of the eye box center positioning system is as follows.

[0066] Step S1: Perform uniformity calibration on the industrial camera 11 of the image acquisition unit 10 to correct the problem of insufficient uniformity caused by the industrial camera 11 itself.

[0067] Specifically, step S1 further includes the following steps: Step S11, placing a uniform surface light source close to the lens of the industrial camera 11 to capture an image, allowing the industrial camera 11 to obtain an image of the surface light source. Step S12, taking the grayscale value of a radius region of 0.1%-0.2% from the center of the image as a calibration reference grayscale. Step S13, based on the calibration reference grayscale, calibrating the grayscale of each pixel in the image to the reference grayscale level to ensure that the uniformity of the calibrated image is not less than 99%, thus completing the uniformity calibration of the industrial camera 11.

[0068] Step S2, allowing the industrial camera 11 to capture a posture test image projected by the optical module of the AR device 100.

[0069] Specifically, step S2 further includes the following steps: Step S21, the control unit 40 sends a command to the AR device 100 to control the optical module of the AR device 100 to project the posture test image toward the industrial camera 11. Step S22, the industrial camera 11 captures the posture test image projected by the optical module of the AR device 100.

[0070] It is worth mentioning that the posture test diagram is set up to test the angular rotation, left and right swaying and pitch swaying of the AR device 100 relative to the center of the industrial camera 11.

[0071] It is worth mentioning that the specific type of the posture test diagram projected by the optical module of the AR device 100 is not limited in the eye-box center positioning system of the present invention; it can be a cross-shaped diagram, as shown in the attached figure. Figure 4 As shown, it can also be any other test diagram that satisfies the requirement of the industrial camera 11 to determine the posture of the AR device 100.

[0072] Step S3: Adjust the projection brightness of the optical module of the AR device 100.

[0073] Specifically, step S3 further includes the following steps: Step S31, the analysis and processing unit 30 acquires the posture test image projected by the optical module of the AR device 100 from the industrial camera 11. Step S32, it determines whether the brightness of the posture test image captured by the industrial camera 11 meets a first preset brightness threshold α. If the brightness of the posture test image captured by the industrial camera 11 meets the first preset brightness threshold α, then step S3 is completed. If the brightness of the posture test image captured by the industrial camera 11 does not meet the first preset brightness threshold α, then step S33 is performed, the analysis and processing unit 30 generates a first brightness adjustment command, and the control unit 40 adjusts the projection brightness of the optical module of the AR device 100 when executing the first brightness adjustment command, so that the brightness of the posture test image captured by the industrial camera 11 meets the first preset brightness threshold α.

[0074] Step S4: After the projection brightness of the optical module of the AR device 100 meets the test brightness, the posture of the AR device 100 is analyzed and adjusted.

[0075] Specifically, step S4 further includes the following steps: Step S41, setting a first preset attitude threshold R for angle rotation, a second preset attitude threshold Y for left and right swaying, and a third preset attitude threshold P for pitch swaying. Step S42, after performing image recognition, data processing, and data analysis on the images captured by the industrial camera 11, the analysis and processing unit 30 obtains the attitude feature values ​​of the AR device 100, wherein the feature values ​​include Roll (angle rotation), Yaw (left and right swaying), and Pitch (pitch swaying). Step S43, determining whether the attitude feature values ​​of the AR device 100 satisfy a preset attitude threshold, wherein when the attitude feature values ​​of the AR device 100 satisfy |Roll|≤R and |Yaw|≤Y and |Pitch|≤P, the attitude adjustment of the AR device 100 is completed; correspondingly, when the attitude feature values ​​of the AR device 100 do not satisfy |Roll|≤R and |Yaw|≤Y and |Pitch|≤P, subsequent steps are performed. Step S44: When the attitude feature value |Roll| of the AR device 100 is greater than the first preset attitude threshold R (i.e., |Roll|>R), the control unit 40 controls the robotic arm 21 to adjust the attitude of the AR device 100 in the v direction. Specifically, when Roll>R, the attitude adjustment direction of the robotic arm 21 is the v+ direction, and when Roll<-R, the attitude adjustment direction of the robotic arm 21 is the v- direction. When the attitude feature value |Yaw| of the AR device 100 is greater than the second preset attitude threshold Y (i.e., |Yaw|>Y), the control unit 40 controls the robotic arm 21 to adjust the attitude in the u direction. The posture of the AR device 100 is adjusted as follows: when Yaw > Y, the posture adjustment direction of the robotic arm 21 is the u+ direction; when Yaw < - Y, the posture adjustment direction of the robotic arm 21 is the u- direction. When the posture feature value |Pitch| of the AR device 100 is greater than the third preset posture threshold P (i.e., |Pitch| > P), the control unit 40 controls the robotic arm 21 to adjust the posture of the AR device 100 in the w direction. Specifically, when Pitch > P, the posture adjustment direction of the robotic arm 21 is the w+ direction; when Pitch < - P, the posture adjustment direction of the robotic arm 21 is the w- direction.

[0076] It is understood that after executing step S44 to complete the adjustment of the posture of the AR device 100, steps S42 and S43 can be executed again to determine whether the posture feature value of the AR device 100 meets the preset posture threshold. If it meets the threshold, the posture adjustment of the AR device 100 is completed. If it does not meet the threshold, step S44 is executed again to further adjust the posture of the AR device 100.

[0077] It is worth mentioning that in step S44, the w+ direction, w- direction, u+ direction, u- direction, v+ direction, and v- direction only represent the directions of movement of the robotic arm 21 when adjusting its posture. Furthermore, the adjustment amounts of the robotic arm 21 in the v, u, and w directions are obtained through iterative calculation of the posture feature values.

[0078] Step S5, the industrial camera 11 is allowed to capture a position test map projected by the optical module of the AR device 100.

[0079] Specifically, step S5 further includes the following steps: S51, the control unit 40 sends a command to the AR device 100 to control the optical module of the AR device 100 to project the position test map toward the industrial camera 11; step S22, the industrial camera 11 captures the position test map projected by the optical module of the AR device 100.

[0080] It is worth mentioning that the position test map is set up to test the left-right and up-down position of the AR device 100 relative to the center of the industrial camera 11.

[0081] It is worth mentioning that the specific type of the position test map projected by the optical module of the AR device 100 is not limited in the eye-box center positioning system of the present invention; it can be a uniform pure white image, as shown in the attached image. Figure 5 As shown, it can also be any other test diagram that satisfies the requirement of the industrial camera 11 to determine the position of the AR device 100.

[0082] Step S6: Adjust the projection brightness of the optical module of the AR device 100.

[0083] Specifically, step S6 further includes the following steps: Step S61, the analysis and processing unit 30 acquires the position test map projected by the optical module of the AR device 100 from the industrial camera 11. Step S62, it is determined whether the brightness of the position test map captured by the industrial camera 11 meets a second preset brightness threshold β. If the brightness of the position test map captured by the industrial camera 11 meets the second preset brightness threshold β, then step S6 is completed. If the brightness of the position test map captured by the industrial camera 11 does not meet the second preset brightness threshold β, then step S33 is performed, the analysis and processing unit 30 generates a second brightness adjustment command, and the control unit 40 adjusts the projection brightness of the optical module of the AR device 100 when executing the second brightness adjustment command, so that the brightness of the position test map captured by the industrial camera 11 meets the second preset brightness threshold β.

[0084] Step S7: After the projection brightness of the optical module of the AR device 100 meets the test brightness, analyze and adjust the position of the AR device 100.

[0085] Specifically, step S7 further includes the following steps: Step S71: Preset a preset gray threshold G for the position of the center of the eyebox. Step S72: First, divide the position test pattern projected by the optical module of the AR device 100 captured by the industrial camera 11 into an N×M grid array, where N and M are positive integers. For example, in the Figure 5 specific embodiment shown in the appendix, divide the position test pattern projected by the optical module of the AR device 100 captured by the industrial camera 11 into a 4×4 grid array; secondly, take the regions A, B, C, and D at the four corners of the N×M grid array, and calculate the gray values of regions A, region B, region C, and region D, and their gray values are GH(A), GH(B), GH(C), and GH(D) respectively. Step S73: Determine whether the gray values of the regions A, B, C, and D at the four corners of the N×M grid array meet the preset gray threshold G. When the gray values of the regions A, B, C, and D at the four corners of the N×M grid array meet (Max(GH(A-D))-Min(GH(A-D))) / Max(GH(A-D))≤G, the position adjustment of the AR device 100 is completed. Correspondingly, when the gray values of the regions A, B, C, and D at the four corners of the N×M grid array do not meet (Max(GH(A-D))-Min(GH(A-D))) / Max(GH(A-D))≤G, the subsequent steps are performed. Step S74: If GH(A)>GH(C) and GH(A)>GH(B), the control unit 40 controls the robotic arm 21 to adjust the position of the AR device 100 in the x+ direction and the z- direction. If GH(A)>GH(C) and GH(A)<GH(B), the control unit 40 controls the robotic arm 21 to adjust the position of the AR device 100 in the x- direction and the z- direction. If GH(A)<GH(C) and GH(A)>GH(B), the control unit 40 controls the robotic arm 21 to adjust the position of the AR device 100 in the x+ direction and the z+ direction. If GH(A)<GH(C) and GH(A)<GH(B), the control unit 40 controls the robotic arm 21 to adjust the position of the AR device 100 in the x- direction and the z+ direction.

[0086] It is understandable that after executing step S74 to complete the adjustment of the position of the AR device 100, steps S72 and S73 can be executed again to determine whether the pose feature value of the AR device 100 meets the preset grayscale threshold G. If it meets the threshold, the position adjustment of the AR device 100 is completed. If it does not meet the threshold, step S74 is executed again to further adjust the position of the AR device 100.

[0087] It is worth mentioning that in step S74, the x+ direction, x- direction, z+ direction, and z- direction only represent the directions of movement of the robotic arm 21 when adjusting its position. Furthermore, the adjustment amount of the robotic arm 21 in the x+, x-, z+, and z- directions is obtained by iterative calculation using position feature values.

[0088] It is understood that after steps S1 to S7 are completed, the center of the eye box and the center of the industrial camera 11 can be aligned, and the AR device 100 can be tested and calibrated in the future. For example, the optical parameters of the AR device 100, such as brightness, brightness uniformity, contrast, field of view, distortion, and resolution, can be tested and calibrated.

[0089] In other words, the eye box center positioning system of the present invention includes the industrial camera 11, the robotic arm 21, the analysis and processing unit 30, and the control unit 40, wherein the analysis and processing unit 30 is connected to the industrial camera 11, and wherein the industrial camera 11, the robotic arm 21, and the analysis and processing unit 30 are respectively connected to the control unit 40, wherein the control unit 40 is configured as follows:

[0090] (A) Control the optical module of the AR device 100 installed on the robotic arm 21 to project the attitude test map and the position test map in sequence;

[0091] (B) Control the industrial camera 11 to sequentially capture the attitude test diagram and the position test diagram;

[0092] (C) Based on the posture test image captured by the industrial camera 11, determine whether the posture feature value of the AR device 100 meets the preset posture threshold, wherein when the posture feature value of the AR device 100 does not meet the preset posture threshold, control the robotic arm 21 to adjust the posture of the AR device 100 in the w direction, u direction, and / or v direction; and

[0093] (D) Based on the position test image captured by the industrial camera 11, determine whether the position grayscale feature value of the AR device 100 meets the preset grayscale threshold, wherein when the position grayscale feature value of the AR device 100 does not meet the preset grayscale threshold, control the robotic arm 21 to adjust the position of the AR device in the x direction and / or z direction.

[0094] The eye box center positioning system of the present invention can automatically locate the center of the eye box and the center of the industrial camera 11, thereby reducing the interference of human factors of testers. On the other hand, the eye box center positioning system can automatically locate the center of the eye box and the center of the industrial camera 11, thereby improving the efficiency and accuracy of aligning the center of the eye box and the center of the industrial camera 11. Furthermore, after subsequent testing and calibration of the optical parameters of the AR device 100, the consistency of the optical parameters of a batch of AR devices 100 can be guaranteed.

[0095] Further, the analysis and processing unit 30 is configured to: first, obtain the posture feature values ​​Roll, Yaw, and Pitch of the AR device based on the posture test image captured by the industrial camera 11; second, compare the magnitudes of |Roll| and R, |Yaw| and Y, and |Pitch| and P respectively. If the posture feature value of the AR device does not satisfy |Roll|≤R, |Yaw|≤Y, and |Pitch|≤P, then it is determined that the posture feature value of the AR device does not satisfy the preset posture threshold. Preferably, the analysis and processing unit 30 can also be configured as a first preset posture threshold R for rotation at a preset angle, a second preset posture threshold Y for left and right swaying, and a third preset posture threshold P for pitch swaying.

[0096] Further, the analysis and processing unit 30 is configured to: first, divide the position test map captured by the industrial camera 11 into an N×M grid array; second, obtain the grayscale feature values ​​GH(A), GH(B), GH(C), and GH(D) of the four corner regions A, B, C, and D of the N×M grid array; third, compare (Max(GH(AD))-Min(GH(AD))) and G. If GH(A), GH(B), GH(C), and GH(D) do not satisfy (Max(GH(AD))-Min(GH(AD))) / Max(GH(AD))≤G, then determine that the position grayscale feature value of the AR device 100 does not meet the preset grayscale threshold. Preferably, the analysis and processing unit 30 can also be configured to preset the preset grayscale threshold at the center of the eye box.

[0097] According to another aspect of the present invention, the present invention further provides a method for center positioning of an eye box, wherein the method for center positioning of an eye box includes the following steps:

[0098] (a) Based on the attitude test image projected by the optical module of the AR device 100 captured by the industrial camera 11, determine whether the attitude feature value of the AR device 100 meets the preset attitude threshold, wherein when the attitude feature value of the AR device 100 does not meet the preset attitude threshold, adjust the attitude of the AR device 100 in the w direction, u direction, and / or v direction; and

[0099] (b) Based on the position test map projected by the optical module of the AR device 100 captured by the industrial camera 11, determine whether the position grayscale feature value of the AR device 100 meets the preset grayscale threshold. When the position grayscale feature value of the AR device 100 does not meet the preset grayscale threshold, adjust the position of the AR device 100 in the x direction and / or z direction to locate the center of the eye box and the center of the industrial camera.

[0100] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

Claims

1. A method for centering a viewing box, characterized in that, The method for locating the center of the eye box includes the following steps: (a) Based on a posture test image projected by the optical module of an AR device captured by an industrial camera, determine whether the posture feature value of the AR device meets a preset posture threshold, wherein when the posture feature value of the AR device does not meet the preset posture threshold, adjust the posture of the AR device in the w direction, u direction and / or v direction. and (b) Based on a position test map projected by the optical module of the AR device captured by the industrial camera, determine whether the position grayscale feature value of the AR device meets a preset grayscale threshold G, wherein when the position grayscale feature value of the AR device does not meet the preset grayscale threshold G, adjust the position of the AR device in the x direction and / or z direction, thereby positioning the center of the eye box and the center of the industrial camera; Step (b) further includes the following steps: (b.1) The preset grayscale threshold G at the center of the preset eye box; (b.2) Divide the position test map captured by the industrial camera into an N×M grid array; (b.3) Obtain the grayscale feature values ​​GH(A), GH(B), GH(C), and GH(D) of the four corner regions A, B, C, and D of the N×M grid array; and (b.4) Compare (Max(GH(AD))-Min(GH(AD))) and G. If GH(A), GH(B), GH(C), and GH(D) do not satisfy (Max(GH(AD))-Min(GH(AD))) / Max(GH(AD))≤G, then it is determined that the position grayscale feature value of the AR device does not satisfy the preset grayscale threshold G. Prior to step (a), the method further includes the following step: Determine whether the brightness of the posture test image meets a first preset brightness threshold α. If it does not meet the first preset brightness threshold α, adjust the projection brightness of the optical module of the AR device so that the brightness of the posture test image meets the first preset brightness threshold α. Prior to step (b), the following step is also included: Determine whether the brightness of the location test map meets a second preset brightness threshold β. If it does not meet the second preset brightness threshold β, adjust the projection brightness of the optical module of the AR device so that the brightness of the location test map meets the second preset brightness threshold β.

2. The eye box center positioning method according to claim 1, wherein step (a) further includes the following steps: (a.1) A first preset attitude threshold R for preset angle rotation, a second preset attitude threshold Y for left and right swaying, and a third preset attitude threshold P for pitch oscillation; (a.2) Based on the posture test image captured by the industrial camera, obtain the posture feature values ​​Roll, Yaw, and Pitch of the AR device; and (a.3) Compare the magnitudes of |Roll| and R, |Yaw| and Y, and |Pitch| and P respectively. If the attitude eigenvalue of the AR device does not satisfy |Roll| ≤ R and |Yaw| ≤ Y and |Pitch| ≤ P, then it is determined that the attitude eigenvalue of the AR device does not satisfy the preset attitude threshold.

3. The eye box center positioning method according to claim 2, wherein in the step (a.3), if |Roll| > R, then adjust the attitude of the AR device in the v direction; if |Yaw| > Y, then adjust the attitude of the AR device in the u direction; if |Pitch| > P, then adjust the attitude of the AR device in the w direction.

4. The eye box center positioning method according to claim 3, wherein in the above method, when Roll > R, adjust the attitude of the AR device in the v+ direction, when Roll < -R, adjust the attitude of the AR device in the v- direction; when Yaw > Y, adjust the attitude of the AR device in the u+ direction, when Yaw < -Y, adjust the attitude of the AR device in the u- direction; when Pitch > P, adjust the attitude of the AR device in the w+ direction, when Pitch < -P, adjust the attitude of the AR device in the w- direction.

5. The eye box center positioning method according to any one of claims 1 to 4, wherein the attitude test pattern is a "plus" shaped pattern.

6. The eye box center positioning method according to any one of claims 1 to 4, wherein the position test pattern is a uniformly pure white image.

7. The eye box center positioning method according to claim 1, wherein in the step (b.4), if GH(A) > GH(C) and GH(A) > GH(B), then adjust the position of the AR device 100 in the x+ direction and the z- direction; if GH(A) > GH(C) and GH(A) < GH(B), then adjust the position of the AR device 100 in the x- direction and the z- direction; if GH(A) < GH(C) and GH(A) > GH(B), then adjust the position of the AR device in the x+ direction and the z+ direction; if GH(A) < GH(C) and GH(A) < GH(B), then adjust the position of the AR device 100 in the x- direction and the z+ direction.

8. The eye box center positioning method according to any one of claims 1 to 4, wherein before the step (a), the eye box center positioning method further comprises the step of: (c) calibrating the uniformity of the industrial camera.

9. The eye box center positioning method according to claim 8, wherein the step (c) further comprises the steps of: (c.1) Place a uniform surface light source close to the lens of the industrial camera to allow the industrial camera to capture an image of the surface light source; (c.2) Take the gray scale value of a radius region of 0.1% - 0.2% of the image center as the calibration reference gray scale; and (c.3) Based on the calibration reference gray scale, calibrate the gray scale at each pixel in the image to the calibration reference gray scale level to complete the uniformity calibration of the industrial camera.

10. A single-view box center positioning system, characterized in that, Includes: An image acquisition unit, wherein the image acquisition unit includes an industrial camera; A product moving unit, wherein the product moving unit includes a robotic arm, and the industrial camera and the robotic arm are disposed adjacent to each other; An analysis and processing unit, wherein the analysis and processing unit is connected to the industrial camera; and A control unit, wherein the industrial camera, the robotic arm, and the analysis and processing unit are respectively connected to the control unit, wherein the control unit is configured to: (A) Control the optical module of an AR device installed on the robotic arm to project a posture test map and a position test map in sequence; (B) Control the industrial camera to sequentially capture the attitude test diagram and the position test diagram; (C) Based on the posture test image captured by the industrial camera, determine whether the posture feature value of the AR device meets a preset posture threshold, wherein when the posture feature value of the AR device does not meet the preset posture threshold, control the robotic arm to adjust the posture of the AR device in the w direction, u direction and / or v direction. as well as (D) Based on the position test map captured by the industrial camera, determine whether the position grayscale feature value of the AR device meets a preset grayscale threshold G, wherein when the position grayscale feature value of the AR device does not meet the preset grayscale threshold G, control the robotic arm to adjust the position of the AR device in the x direction and / or z direction. The analysis and processing unit is configured as follows: The position test map captured by the industrial camera is divided into an N×M square array; Obtain the grayscale feature values ​​GH(A), GH(B), GH(C), and GH(D) of the four corner regions A, B, C, and D of an N×M grid array; and Compare (Max(GH(AD))-Min(GH(AD))) with G. If GH(A), GH(B), GH(C), and GH(D) do not satisfy (Max(GH(AD))-Min(GH(AD))) / Max(GH(AD))≤G, then it is determined that the position grayscale feature value of the AR device does not satisfy the preset grayscale threshold G. Prior to step (C), the following step is also included: Determine whether the brightness of the posture test image meets a first preset brightness threshold α. If it does not meet the first preset brightness threshold α, adjust the projection brightness of the optical module of the AR device so that the brightness of the posture test image meets the first preset brightness threshold α. Before step (D), the following step is also included: Determine whether the brightness of the location test map meets a second preset brightness threshold β. If it does not meet the second preset brightness threshold β, adjust the projection brightness of the optical module of the AR device so that the brightness of the location test map meets the second preset brightness threshold β.

11. The eye box center positioning system according to claim 10, wherein the posture test diagram is a cross-shaped diagram.

12. The eye box center positioning system according to claim 10, wherein the position test image is a uniform pure white image.

13. The eye box center positioning system according to any one of claims 10 to 12, wherein the analysis and processing unit is configured to: Obtain the attitude characteristic values Roll, Yaw, and Pitch of the AR device according to the attitude test image captured by the industrial camera; and Compare the magnitudes of |Roll| and R, |Yaw| and Y, and |Pitch| and P respectively. If the attitude characteristic values of the AR device do not satisfy |Roll| ≤ R and |Yaw| ≤ Y and |Pitch| ≤ P, it is determined that the attitude characteristic values of the AR device do not satisfy the preset attitude threshold.

14. The eye box center positioning system according to claim 13, wherein the analysis and processing unit is further configured to: if |Roll| > R, adjust the attitude of the AR device in the v direction; if |Yaw| > Y, adjust the attitude of the AR device in the u direction; if |Pitch| > P, adjust the attitude of the AR device in the w direction.

15. The eye box center positioning system according to claim 14, wherein when Roll > R, adjust the attitude of the AR device in the v+ direction, and when Roll < -R, adjust the attitude of the AR device in the v- direction; when Yaw > Y, adjust the attitude of the AR device in the u+ direction, and when Yaw < -Y, adjust the attitude of the AR device in the u- direction; when Pitch > P, adjust the attitude of the AR device in the w+ direction, and when Pitch < -P, adjust the attitude of the AR device in the w- direction.

16. The eye box center positioning system according to claim 10, wherein the analysis and processing unit is further configured to: if GH(A) > GH(C) and GH(A) > GH(B), adjust the position of the AR device 100 in the x+ direction and the z- direction; if GH(A) > GH(C) and GH(A) < GH(B), adjust the position of the AR device 100 in the x- direction and the z- direction; if GH(A) < GH(C) and GH(A) > GH(B), adjust the position of the AR device in the x+ direction and the z+ direction; if GH(A) < GH(C) and GH(A) < GH(B), adjust the position of the AR device 100 in the x- direction and the z+ direction.

Images