Method, system, apparatus and electronic device for aligning binocular camera with optical assembly

Abstract

This application provides a method, system, apparatus, and electronic device for aligning a binocular camera with an optical component. The method includes: acquiring a first current display screen corresponding to two target cameras, wherein the first current display screen is obtained by the optical component projecting light emitted from a first display onto the corresponding target camera; adjusting the pose of the binocular cameras based on the first current display screen, so that each target camera is aligned with the optical axis of the corresponding optical component; and determining the final distance between the binocular cameras and the optical component based on the grayscale brightness of each pixel in the edge region of two second current display screens. The two second current display screens are the current display screens corresponding to the two target cameras after alignment, which can uniformly align the left and right optical axes, save time, improve alignment efficiency, and no longer rely on the experience and skill level of the operator, thus greatly improving alignment accuracy.

Description

Technical Field

[0001] This application relates to the field of head-mounted display technology, and more particularly to a method, system, device, and electronic device for aligning a binocular camera and optical components. Background Technology

[0002] Head-mounted displays, as one of the core devices in virtual reality (VR) and augmented reality (AR) technologies, have been widely used in entertainment, education, medical care, military training, and many other fields. During the calibration and verification of head-mounted displays, the alignment between the testing camera and optical components is crucial.

[0003] Currently, a monocular camera is used to align the optical axes of the left and right objects separately, and the distance between the monocular camera and the optical components is adjusted based on experience. However, aligning the optical axes of the left and right objects one by one is a tedious and time-consuming process, especially when high-precision alignment is required. It also depends on the operator's experience and skill level. For inexperienced operators, it may be difficult to achieve the required calibration accuracy. Summary of the Invention

[0004] This application provides a method, system, device, and electronic device for aligning a binocular camera with optical components, in order to improve alignment efficiency and accuracy.

[0005] In a first aspect, embodiments of this application provide a method for aligning a binocular camera with optical components, wherein the optical components are located in a head-mounted display device, the head-mounted display device including two sets of optical components and a first display, the two sets of optical components corresponding one-to-one with two target cameras included in the binocular camera, the method comprising:

[0006] The first current display image corresponding to each of the two target cameras is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera.

[0007] Based on the first current display screen, the pose of the binocular cameras is adjusted so that each target camera is aligned with the optical axis of the corresponding optical component.

[0008] The final distance between the binocular camera and the optical components is determined based on the grayscale brightness of each pixel in the edge region of the two second current display images. The two second current display images are the current display images corresponding to the two target cameras after alignment.

[0009] Optionally, obtain the first currently displayed image corresponding to each of the two target cameras, including:

[0010] Obtain a first distance and adjust the distance between the two target cameras to the first distance, wherein the first distance is the distance between the two optical components;

[0011] Obtain the first currently displayed image corresponding to each of the two target cameras after adjustment.

[0012] Optionally, the first display is used to display a preset image, the preset image having a cross pattern at its center; based on the first currently displayed image, the pose of the binocular camera is adjusted, including:

[0013] Based on the two current first display images, adjust the yaw angle, pitch angle, and roll angle of the binocular camera;

[0014] After the adjustment is completed, the third current display screen corresponding to each of the two target cameras is obtained. When the cross pattern in each third current display screen is aligned with the corresponding physical cross, the attitude adjustment of the binocular camera is confirmed to be complete.

[0015] Based on two third current display screens, the position of the binocular camera in the horizontal and vertical directions is adjusted within the target plane, wherein the target plane is perpendicular to the distance direction between the binocular camera and the optical components;

[0016] After the adjustment is completed, the fourth current display screen corresponding to the two target cameras is obtained respectively; when the difference between the widths of the upper and lower ends of the blank area of ​​each fourth current display screen meets the first preset condition, and the difference between the narrowest end and the widest end of the blank area of ​​the two fourth current display screens respectively meets the second preset condition, it is confirmed that the position of the binocular camera in the target plane has been adjusted, and the fourth current display screen is the second current display screen.

[0017] Optionally, the light projection angle range corresponding to the optical component is 0 to M, and the projection angle range corresponding to the edge region is kM to M, where k is a preset value. The final distance between the binocular camera and the optical component is determined based on the grayscale brightness of each pixel in the edge region of the two second currently displayed images, including:

[0018] Record the current position of the binocular camera;

[0019] For each second currently displayed frame, calculate the first sum of the grayscale brightness of each pixel in the edge region of the second currently displayed frame;

[0020] Add the first sums corresponding to the two second currently displayed screens to obtain the second sum;

[0021] Based on the second sum and the position of the binocular camera corresponding to the second sum, the final distance between the binocular camera and the optical components is determined.

[0022] Optionally, based on the second sum and the position of the stereo camera corresponding to the second sum, the final distance between the stereo camera and the optical components is determined, including:

[0023] Repeat the following step-by-step operation until the preset number of repetitions is reached;

[0024] Based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, determine the final distance between the binocular cameras and the optical components;

[0025] The stepping operation includes:

[0026] Along the distance between the binocular camera and the optical components, move the binocular camera by a preset step length and record the position of the binocular camera after the movement;

[0027] Obtain the updated second current display screen corresponding to the two target cameras after the movement;

[0028] For each updated second current display screen, calculate the first sum of the grayscale brightness of each pixel in the edge region of the updated second current display screen;

[0029] Add the first sums corresponding to the two updated current display screens to obtain the second sum.

[0030] Optionally, based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, the final distance between the binocular cameras and the optical components is determined, including:

[0031] Based on multiple second sums and the positions of the stereo cameras corresponding to each second sum, a functional relationship graph between the second sums and the positions of the stereo cameras is determined; where the second sums are the vertical axis and the positions of the stereo cameras are the horizontal axis.

[0032] If the function graph shows a trend of first increasing and then decreasing, then the final distance between the binocular camera and the optical components is determined as the x-coordinate corresponding to the highest point of the function graph.

[0033] Optionally, once the crosshair pattern in each third currently displayed frame is aligned with the corresponding physical crosshair, confirm that the binocular camera's attitude adjustment is complete, including:

[0034] When the two sets of optical components are parallel, the attitude adjustment of the binocular camera is confirmed to be complete when the cross pattern in each third current display screen is aligned with the corresponding physical cross.

[0035] Accordingly, the method further includes:

[0036] When the two sets of optical components are not parallel, if the cross pattern in one of the third current display screens is located to the left of the corresponding physical cross, and the cross pattern in the other third current display screen is located to the right of the corresponding physical cross, and the distance between the cross patterns in the two third current display screens and the corresponding physical cross is equal, then the attitude adjustment of the binocular camera is confirmed to be complete.

[0037] Secondly, embodiments of this application provide an alignment system for a binocular camera and optical components. The optical components are located in a head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The system includes:

[0038] A clamp is used to secure the head-mounted display device;

[0039] A six-degree-of-freedom turntable is used to support the stereo camera;

[0040] A control unit, connected to the six-degree-of-freedom turntable, adjusts the pose of the binocular cameras by controlling the turntable to align each target camera with the optical axis of its corresponding optical component. Based on the grayscale brightness of each pixel in the edge region of the current display screen corresponding to each of the two target cameras, the control unit adjusts the distance between the binocular cameras and the optical component to achieve alignment. The current display screen is obtained by the optical component projecting light emitted from the first display onto the corresponding target camera.

[0041] Optionally, the first display is used to display a preset image, the preset image having a cross pattern at its center; the control unit, when adjusting the pose of the binocular cameras to align each target camera with the optical axis of its corresponding optical component, is specifically used for:

[0042] Based on the display screens corresponding to the two target cameras, the six-degree-of-freedom turntable is controlled to adjust the yaw angle, pitch angle, and roll angle of the binocular camera, thereby adjusting the attitude of the binocular camera so that after the adjustment, the cross pattern in the display screen corresponding to either target camera is aligned with the corresponding physical cross.

[0043] The six-degree-of-freedom turntable is controlled to adjust the position of the binocular camera in the horizontal and vertical directions within the target plane. After the adjustment is completed, when the difference between the upper and lower widths of the blank area of ​​the display screen corresponding to any target camera satisfies the first preset condition, and the difference between the narrowest end and the widest end of the blank area of ​​the two display screens satisfies the second preset condition, it is confirmed that each target camera is aligned with the optical axis of the corresponding optical component. The target plane is perpendicular to the distance direction between the binocular camera and the optical component.

[0044] Optionally, the six-degree-of-freedom turntable includes a stepper motor, which, when driving the supported binocular camera to perform pose adjustment, is specifically used for:

[0045] The pose of the mounted binocular camera is adjusted by driving a stepper motor.

[0046] Optionally, the system further includes an interpupillary distance adjustment module located on the six-degree-of-freedom turntable, and the binocular camera is carried by the interpupillary distance adjustment module;

[0047] The control unit is connected to the interpupillary distance adjustment module and is used to adjust the distance between the two cameras to a first distance by controlling the interpupillary distance adjustment module, wherein the first distance is the distance between the two optical components.

[0048] Optionally, the interpupillary distance adjustment module is a lead screw adjustment mechanism.

[0049] Thirdly, embodiments of this application provide an alignment device for a binocular camera and optical components. The optical components are located in a head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The device includes:

[0050] The first acquisition module is used to acquire the first current display screen corresponding to the two target cameras respectively. The first current display screen is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera.

[0051] The adjustment module is used to adjust the pose of the binocular cameras based on the first current display screen, so that each target camera is aligned with the optical axis of the corresponding optical component.

[0052] The determination module is used to determine the final distance between the binocular camera and the optical component based on the grayscale brightness of each pixel in the edge region of the two second current display screens, wherein the two second current display screens are the current display screens corresponding to the two target cameras after alignment.

[0053] Fourthly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0054] The memory stores computer-executed instructions;

[0055] The processor executes computer execution instructions stored in the memory to implement the method described in the first aspect above.

[0056] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect above.

[0057] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0058] The present application provides a method, system, device, and electronic device for aligning a binocular camera and an optical component. The method includes: acquiring a first current display screen corresponding to two target cameras, wherein the first current display screen is obtained by the optical component projecting light emitted from the first display screen onto the corresponding target camera; adjusting the pose of the binocular cameras based on the first current display screen, so that each target camera is aligned with the optical axis of the corresponding optical component; and determining the final distance between the binocular camera and the optical component based on the grayscale brightness of each pixel in the edge region of two second current display screens. The two second current display screens are the current display screens corresponding to the two target cameras after alignment, allowing for unified alignment of the left and right optical axes, saving time and improving alignment efficiency. Furthermore, determining the final distance between the binocular camera and the optical component based on the grayscale brightness of each pixel in the edge region of the two second current display screens eliminates reliance on the operator's experience and skill level, accurately determining the distance between the binocular camera and the optical component, thus greatly improving alignment accuracy. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0060] Figure 1 An application scenario diagram provided for an embodiment of this application;

[0061] Figure 2 A schematic flowchart illustrating an alignment method between a binocular camera and optical components provided in an embodiment of this application;

[0062] Figure 3 A schematic diagram illustrating the alignment of a binocular camera and a pancake optical engine, provided for an embodiment of this application;

[0063] Figure 4 A schematic diagram showing the current display screens corresponding to the left and right target cameras after crosshair alignment, as provided in an embodiment of this application.

[0064] Figure 5 An edge field-of-view optical path diagram with an Eye Relief of 20mm is provided as an embodiment of this application;

[0065] Figure 6 An edge field-of-view optical path diagram with an Eye Relief of 10mm is provided as an embodiment of this application;

[0066] Figure 7 This application provides a schematic diagram of edge field illuminance when Eye Relief is 20mm.

[0067] Figure 8 This application provides a schematic diagram of edge field illuminance when Eye Relief is 10mm.

[0068] Figure 9 A schematic diagram of the structure of an alignment device for a binocular camera and optical components provided in an embodiment of this application;

[0069] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0070] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0072] Head-mounted displays, as one of the core devices in Virtual Reality (VR) and Augmented Reality (AR) technologies, have already occupied an important position in modern technological applications. These devices bring users into a completely new digital world through immersive experiences, allowing them to feel as if they are actually there through sight, hearing, and even touch. With the continuous advancement of technology, the application scope of head-mounted displays is also constantly expanding, covering multiple fields such as entertainment, education, healthcare, and military training.

[0073] In the entertainment industry, head-mounted displays provide users with a more realistic and interactive gaming experience, allowing players to fully immerse themselves in the virtual world and experience unprecedented gaming enjoyment. In education, these devices are used to create virtual classrooms and laboratories, helping students better understand complex concepts and theories through interactive learning. In the medical field, head-mounted displays are used for surgical simulations and rehabilitation training, helping doctors improve surgical skills and provide patients with more effective treatment options. In military training, virtual reality technology provides soldiers with a safe and cost-effective training method by simulating real battlefield environments.

[0074] However, for these applications to be efficient and accurate, the calibration and standardization process of the head-mounted display device is crucial. In this process, the alignment between the test camera and optical components is a key factor. Precise alignment ensures that the device delivers clear and accurate images after calibration, thus enhancing the user experience. Improper alignment can lead to problems such as image distortion, parallax mismatch, and user discomfort after calibration.

[0075] Currently, a monocular camera is used to align the optical axes of the left and right objects separately, and the distance between the monocular camera and the optical components is adjusted based on experience. However, aligning the optical axes of the left and right objects one by one is a tedious and time-consuming process, especially when high-precision alignment is required. It also depends on the operator's experience and skill level. For inexperienced operators, it may be difficult to achieve the required calibration accuracy.

[0076] In view of this, this application provides an alignment method for a binocular camera and an optical component. This method acquires a first current display image corresponding to two target cameras, where the first current display image is obtained by the optical component projecting light emitted from a first display onto the corresponding target camera. Based on the first current display image, the pose of the binocular camera is adjusted so that each target camera is aligned with the optical axis of its corresponding optical component. Based on the grayscale brightness of each pixel in the edge region of two second current display images, the final distance between the binocular camera and the optical component is determined. The two second current display images are the current display images corresponding to the two target cameras after alignment. This allows for unified alignment of the left and right optical axes, saving time and improving alignment efficiency. Furthermore, determining the final distance between the binocular camera and the optical component based on the grayscale brightness of each pixel in the edge region of the two second current display images eliminates reliance on the operator's experience and skill level, accurately determining the distance between the binocular camera and the optical component, thus greatly improving alignment accuracy.

[0077] Figure 1 An application scenario diagram provided for an embodiment of this application, such as... Figure 1As shown, it includes: head display fixture 101, binocular camera 102, lead screw slide 103, soft runner 104, base platform 105 and six-degree-of-freedom turntable 106. The head display device is fixed with the head display fixture 101, and the binocular camera 102 is fixed on the six-degree-of-freedom turntable 106.

[0078] The head-mounted display includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera 102. The two target cameras included in the binocular camera 102 communicate with the server via wired or wireless means. The lead screw slide 103 and the six-degree-of-freedom turntable 106 also communicate with the server via wired or wireless means. The lead screw slide 103 adjusts the distance between the two target cameras according to the control commands issued by the server, and the six-degree-of-freedom turntable 106 is used to adjust the pose of the binocular camera 102 according to the control commands issued by the server.

[0079] The server first acquires the first current display images corresponding to the two target cameras respectively. The first current display image is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera. Based on the first current display image, a first control signal is sent to the six-degree-of-freedom turntable 106 to adjust the pose of the binocular camera 102 so that each target camera is aligned with the optical axis of the corresponding optical component. Based on the grayscale brightness of each pixel in the edge region of the two second current display images, the final distance between the binocular camera 102 and the optical component is determined, and the movement of the binocular camera is controlled by the six-degree-of-freedom turntable 106. The two second current display images are the current display images corresponding to the two target cameras respectively after alignment.

[0080] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0081] Figure 2 This is a flowchart illustrating an alignment method between a binocular camera and optical components provided in an embodiment of this application. The optical components are located in a head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The execution entity in this embodiment can be any device with data processing capabilities. This application uses a server as the execution entity for specific description. Figure 2 As shown in the embodiment of this application, an alignment method between a binocular camera and optical components may include:

[0082] Step 201: Obtain the first current display screen corresponding to each of the two target cameras. The first current display screen is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera.

[0083] The optical component may include at least one optical element, which may be a lens, prism, mirror, etc. The function of the optical component is to project the light emitted by the first display in the head-mounted display device onto the corresponding target camera to generate the current display image in the target camera. For example, the optical component may be a pancake (a compact optical design) optical engine.

[0084] Before the server executes step 201, some preparatory work needs to be done, as follows:

[0085] 1. Mount the two target cameras on the bracket respectively, and adjust the distance between the two target cameras to the preset distance;

[0086] 2. Calibrate the intrinsic and extrinsic parameters of the two target cameras respectively;

[0087] 3. Fix the center point of the support to the rotation center of the adjustment platform, wherein the center point is equidistant from the two target cameras; this application does not limit the type of adjustment platform, as long as it can perform six-axis adjustment, wherein the six axes include: X-axis (lateral movement), moving left and right along the horizontal direction in the target plane, wherein the target plane is perpendicular to the distance direction between the binocular camera and the optical component; Y-axis (vertical movement), moving up and down along the vertical direction in the target plane; Z-axis (vertical movement), moving along the distance direction between the binocular camera and the optical component; A-axis (flip or roll), usually referring to rotation around the X-axis; B-axis (pitch or tilt), usually referring to rotation around the Y-axis; C-axis (yaw or rotation), usually referring to rotation around the Z-axis.

[0088] After completing the above preparations, the server obtains the first current display screen corresponding to each of the two target cameras. The first current display screen is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera.

[0089] Optionally, obtain the first currently displayed image corresponding to each of the two target cameras, including:

[0090] Obtain a first distance and adjust the distance between the two target cameras to the first distance, wherein the first distance is the distance between the two optical components;

[0091] Obtain the first currently displayed image corresponding to each of the two target cameras after adjustment.

[0092] Specifically, the server first obtains the first distance, which is the distance between the two optical components, that is, the distance between the optical axes corresponding to the two optical components. Then, the server sends a control command to the lead screw slide to adjust the distance between the two target cameras to the first distance. After the adjustment is completed, the server obtains the first current display screen corresponding to the two target cameras after the adjustment.

[0093] In this way, the distance between the two target cameras is first adjusted to the distance between their optical axes before further adjustments are made. This improves the efficiency and accuracy of the adjustments compared to making direct adjustments.

[0094] Step 202: Based on the first current display screen, adjust the pose of the binocular cameras so that each target camera is aligned with the optical axis of the corresponding optical component.

[0095] Specifically, based on the first currently displayed screen, the server sends control commands to the six-degree-of-freedom turntable, which adjusts the pose of the binocular cameras so that each target camera is aligned with the optical axis of the corresponding optical component.

[0096] Optionally, the first display is used to display a preset image, the preset image having a cross pattern at its center; based on the first currently displayed image, the pose of the binocular camera is adjusted, including:

[0097] Based on the two current first display images, adjust the yaw angle, pitch angle, and roll angle of the binocular camera;

[0098] After the adjustment is completed, the third current display screen corresponding to each of the two target cameras is obtained. When the cross pattern in each third current display screen is aligned with the corresponding physical cross, the attitude adjustment of the binocular camera is confirmed to be complete.

[0099] Based on two third current display screens, the position of the binocular camera in the horizontal and vertical directions is adjusted within the target plane, wherein the target plane is perpendicular to the distance direction between the binocular camera and the optical components;

[0100] After the adjustment is completed, the fourth current display screen corresponding to the two target cameras is obtained respectively. When the difference between the widths of the upper and lower ends of the blank area of ​​each fourth current display screen meets the first preset condition, and the difference between the narrowest end and the widest end of the blank area of ​​the two fourth current display screens respectively meets the second preset condition, it is confirmed that the position of the binocular camera in the target plane has been adjusted, and the fourth current display screen is the second current display screen.

[0101] Specifically, firstly, the server sends a first control command to the six-degree-of-freedom turntable based on the relative position of the crosshair pattern in each first current display screen and the corresponding physical crosshair of the target camera. The first control command instructs the six-degree-of-freedom turntable to adjust the yaw, pitch, and roll angles of the binocular camera. After the adjustment is completed, the server acquires the third current display screens corresponding to the two target cameras respectively. When the crosshair pattern in each third current display screen is aligned with the corresponding physical crosshair, the attitude adjustment of the binocular camera is confirmed to be complete. When the crosshair pattern in any third current display screen is not aligned with the corresponding physical crosshair, the server re-sends the control command to the six-degree-of-freedom turntable based on the relative position of the crosshair pattern in each third current display screen and the corresponding physical crosshair of the target camera to readjust the yaw, pitch, and roll angles of the binocular camera. This process continues until the crosshair pattern in each new current display screen acquired by the server is aligned with the corresponding physical crosshair, confirming that the attitude adjustment of the binocular camera is complete.

[0102] Specifically, for each third currently displayed screen, if the difference between the cross pattern in the third currently displayed screen and the corresponding physical cross is within 5 pixels or 0.3% of a single side pixel, it is confirmed that the cross pattern in the third currently displayed screen and the corresponding physical cross are aligned.

[0103] Subsequently, based on the two third current display screens, the server sends a second control command to the six-degree-of-freedom turntable. The second control command is used to instruct the six-degree-of-freedom turntable to adjust the position of the binocular camera in the horizontal and vertical directions within the target plane. The target plane is perpendicular to the distance direction between the binocular camera and the optical components. After the adjustment is completed, the server obtains the fourth current display screens corresponding to the two target cameras respectively. When the difference in the width of the upper and lower ends of the blank area of ​​each fourth current display screen satisfies the first preset condition, and the difference between the narrowest end and the widest end of the blank area of ​​the two fourth current display screens respectively satisfies the second preset condition, it is confirmed that the position of the binocular camera in the target plane has been adjusted, and the fourth current display screen is the second current display screen. If the difference in width between the upper and lower ends of the blank area of ​​any fourth currently displayed screen does not meet the first preset condition, or if the difference between the narrowest end and the widest end of the blank areas of the two fourth currently displayed screens does not meet the second preset condition, the server re-sends control commands to the six-degree-of-freedom turntable based on each fourth currently displayed screen to readjust the horizontal and vertical positions of the binocular camera in the target plane. This continues until the server obtains that the difference in width between the upper and lower ends of the blank area of ​​each fourth currently displayed screen meets the first preset condition, and the difference between the narrowest end and the widest end of the blank areas of the two fourth currently displayed screens meets the second preset condition. In this case, the position adjustment of the binocular camera in the target plane is confirmed to be complete.

[0104] The first preset condition is that the difference in width between the upper and lower blank areas is within 5 pixels or 0.3% of the width of a single side. The second preset condition is that the difference is within 5 pixels or 0.3% of the width of a single side. The blank areas can be found in [reference needed]. Figure 4 The green part outside the black circle in the middle.

[0105] By using a crosshair pattern at the center of the preset image, a clear reference point can be provided, allowing for precise calibration of the stereo camera's attitude and position during the adjustment process. Furthermore, adjusting the three angles first, followed by horizontal and vertical adjustments within the target plane, improves adjustment efficiency. Finally, each adjustment has clear indicators as a basis for accurate calibration, ensuring the precision of the adjustments.

[0106] Optionally, once the crosshair pattern in each third currently displayed frame is aligned with the corresponding physical crosshair, confirm that the binocular camera's attitude adjustment is complete, including:

[0107] When the two sets of optical components are parallel, the attitude adjustment of the binocular camera is confirmed to be complete when the cross pattern in each third current display screen is aligned with the corresponding physical cross.

[0108] Accordingly, the alignment method between the binocular camera and the optical components provided in this application also includes:

[0109] When the two sets of optical components are not parallel, if the cross pattern in one of the third current display screens is located to the left of the corresponding physical cross, and the cross pattern in the other third current display screen is located to the right of the corresponding physical cross, and the distance between the cross patterns in the two third current display screens and the corresponding physical cross is equal, then the attitude adjustment of the binocular camera is confirmed to be complete.

[0110] Specifically, when the two sets of optical components are parallel (i.e., their corresponding optical axes are parallel), the server confirms the binocular camera's attitude adjustment is complete when the crosshair pattern in each third currently displayed screen is aligned with its corresponding physical crosshair. When the two sets of optical components are not parallel (i.e., their corresponding optical axes are not parallel), the server confirms the binocular camera's attitude adjustment is complete when the following conditions are met: The crosshair pattern in one third currently displayed screen is located to the left of the corresponding physical crosshair, the crosshair pattern in the other third currently displayed screen is located to the right of the corresponding physical crosshair, and the distances between the crosshairs in both third currently displayed screens and their corresponding physical crosshairs are equal.

[0111] This approach considers both parallel and non-parallel optical components, providing a flexible adjustment strategy. This makes the method applicable not only to cases where the optical axes of the components are parallel but also to cases where they are not parallel, thus expanding the applicability of the method.

[0112] Step 203: Based on the grayscale brightness of each pixel in the edge region of the two second current display screens, determine the final distance between the binocular camera and the optical component, wherein the two second current display screens are the current display screens corresponding to the two target cameras after alignment.

[0113] Specifically, the server determines the final distance between the binocular camera and the optical components based on the grayscale brightness of each pixel in the edge region of the two second current display screens. The two second current display screens are the current display screens corresponding to the two target cameras after alignment. They may be the fourth current display screen or the current display screens acquired later.

[0114] The binocular camera and optical component alignment method provided in this application acquires a first current display image corresponding to each of the two target cameras. This first current display image is obtained by the optical component projecting light emitted from the first display onto the corresponding target camera. Based on the first current display image, the pose of the binocular cameras is adjusted so that each target camera is aligned with the optical axis of its corresponding optical component. The final distance between the binocular camera and the optical component is determined based on the grayscale brightness of each pixel in the edge region of the two second current display images, where the two second current display images are the current display images corresponding to the two target cameras after alignment. This method allows for unified alignment of the left and right optical axes, saving time and improving alignment efficiency. Furthermore, determining the final distance between the binocular camera and the optical component based on the grayscale brightness of each pixel in the edge region of the two second current display images eliminates reliance on the operator's experience and skill level, thus accurately determining the distance between the binocular camera and the optical component and significantly improving alignment accuracy.

[0115] Optionally, the light projection angle range corresponding to the optical component is 0 to M, and the projection angle range corresponding to the edge region is kM to M, where k is a preset value. The final distance between the binocular camera and the optical component is determined based on the grayscale brightness of each pixel in the edge region of the two second currently displayed images, including:

[0116] Record the current position of the binocular camera;

[0117] For each second currently displayed frame, calculate the first sum of the grayscale brightness of each pixel in the edge region of the second currently displayed frame;

[0118] Add the first sums corresponding to the two second currently displayed screens to obtain the second sum;

[0119] Based on the second sum and the position of the binocular camera corresponding to the second sum, the final distance between the binocular camera and the optical components is determined.

[0120] Specifically, the light projection angle range corresponding to the optical component is 0 to M, and the projection angle range corresponding to the edge region is kM to M, where k is a preset value between 0 and 1, preferably between 0.8 and 0.9.

[0121] Specifically, the server first records the current position coordinates of the binocular camera. It may record the three-dimensional position coordinates or only the position coordinates along the distance between the binocular camera and the optical component. For each second current display screen, it calculates the first sum of the grayscale brightness of each pixel in the edge region of the second current display screen to obtain the first sum corresponding to the two second current display screens respectively. The two first sums are added together to obtain the second sum. Finally, based on the calculated second sum and the position coordinates of the binocular camera corresponding to the second sum, the final distance between the binocular camera and the optical component is determined.

[0122] Optionally, after recording the current position of the binocular camera, for each second current display screen, the pixel row at the horizontal center of the second current display screen is determined, the grayscale brightness of each pixel in the edge region of the pixel row is read, and the sum of the grayscale brightness of each pixel in the edge region is calculated, i.e., the first sum. The first sums corresponding to the two second current display screens are obtained respectively. The two first sums are added together to obtain the second sum. Finally, based on the calculated second sum and the position coordinates of the binocular camera corresponding to the second sum, the final distance between the binocular camera and the optical components is determined.

[0123] In this way, by calculating the sum of the grayscale brightness of each pixel in the edge region, the calculation process is simplified, avoiding complex image processing algorithms. This simplification helps to improve computational efficiency. By adding the sum of the grayscale brightness of the two second currently displayed images, this method effectively fuses information from the binocular cameras, improving the stability and accuracy of the measurement. This information fusion helps to reduce the impact of measurement errors from a single camera.

[0124] Optionally, based on the second sum and the position of the stereo camera corresponding to the second sum, the final distance between the stereo camera and the optical components is determined, including:

[0125] Repeat the following step-by-step operation until the preset number of repetitions is reached;

[0126] Based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, determine the final distance between the binocular cameras and the optical components;

[0127] The stepping operation includes:

[0128] Along the distance between the binocular camera and the optical components, move the binocular camera by a preset step length and record the position of the binocular camera after the movement;

[0129] Obtain the updated second current display screen corresponding to the two target cameras after the movement;

[0130] For each updated second current display screen, calculate the first sum of the grayscale brightness of each pixel in the edge region of the updated second current display screen;

[0131] Add the first sums corresponding to the two updated current display screens to obtain the second sum.

[0132] Preferably, the preset step size should be less than 1 mm.

[0133] Specifically, the server repeatedly performs the following stepping operation until the number of repetitions reaches a preset number; then, based on the multiple second sums obtained and the positions of the binocular cameras corresponding to each second sum, the final distance between the binocular cameras and the optical components is determined. The stepping operation includes: moving the binocular cameras along the distance direction between the binocular cameras and the optical components by a preset step length, and recording the position of the binocular cameras after the movement; obtaining the updated second current display images corresponding to the two target cameras after the movement; for each updated second current display image, calculating the first sum of the grayscale brightness of each pixel in the edge region of the updated second current display image; and adding the first sums corresponding to the two updated current display images to obtain the second sum.

[0134] Optionally, the stepping operation includes: moving the binocular camera by a preset step length along the distance direction between the binocular camera and the optical component, and recording the position of the binocular camera after the movement; acquiring the updated second current display screen corresponding to the two target cameras respectively after the movement; for each updated second current display screen, determining the horizontally center pixel row in the updated second current display screen, reading the grayscale brightness of each pixel in the edge region of the pixel row, and calculating the sum of the grayscale brightness of each pixel in the edge region, i.e., the first sum, obtaining the first sum corresponding to the two second current display screens respectively, and adding the first sum corresponding to the two updated current display screens respectively to obtain the second sum.

[0135] In this way, by presetting the step size and the number of repetitions, the method can automate the process, reduce manual intervention and operational complexity, and improve efficiency and accuracy.

[0136] Optionally, based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, the final distance between the binocular cameras and the optical components is determined, including:

[0137] Based on multiple second sums and the positions of the stereo cameras corresponding to each second sum, a functional relationship graph between the second sums and the positions of the stereo cameras is determined; where the second sums are the vertical axis and the positions of the stereo cameras are the horizontal axis.

[0138] If the function graph shows a trend of first increasing and then decreasing, then the final distance between the binocular camera and the optical components is determined as the x-coordinate corresponding to the highest point of the function graph.

[0139] Specifically, the server determines the functional relationship between the second sum and the position of the binocular camera based on multiple second sums and the positions of the binocular cameras corresponding to each second sum; wherein the second sum is the vertical axis and the position of the binocular camera is the horizontal axis; if the functional relationship shows a trend of first increasing and then decreasing, then the final distance between the binocular camera and the optical component is determined as the horizontal coordinate corresponding to the highest point of the functional relationship.

[0140] If the function graph does not show a trend of first increasing and then decreasing, the coordinates of the stereo camera along the distance between the stereo camera and the optical components should be redefined, and the above stepping operation should be performed again until the function graph of the redefined second sum and the position of the stereo camera shows a trend of first increasing and then decreasing.

[0141] By analyzing the trends in the function graph, this method can more accurately identify the impact of distance changes on image quality, thus determining the final distance more precisely. This trend analysis reduces random errors in single measurements and improves measurement accuracy.

[0142] Optionally, based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, the final distance between the binocular cameras and the optical components is determined, including:

[0143] The maximum value among the plurality of second sums is determined as the target second sum;

[0144] The position of the binocular camera corresponding to the second sum of the target is determined as the final distance between the binocular camera and the optical components.

[0145] In this way, determining the position of the binocular camera corresponding to the second sum of the targets is the final distance between the binocular camera and the optical components, which can improve the efficiency of determining the final distance.

[0146] This application embodiment also provides an alignment system for a binocular camera and optical components. The optical components are located in a head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The system includes:

[0147] A clamp is used to secure the head-mounted display device;

[0148] A six-degree-of-freedom turntable is used to support the stereo camera;

[0149] A control unit, connected to the six-degree-of-freedom turntable, adjusts the pose of the binocular cameras by controlling the turntable to align each target camera with the optical axis of its corresponding optical component. Based on the grayscale brightness of each pixel in the edge region of the current display screen corresponding to each of the two target cameras, the control unit adjusts the distance between the binocular cameras and the optical component to achieve alignment. The current display screen is obtained by the optical component projecting light emitted from the first display onto the corresponding target camera.

[0150] The display screens corresponding to the two target cameras are the second current display screens in the aforementioned embodiment.

[0151] The specific implementation principle and technical effects of the binocular camera and optical component alignment system provided in this application embodiment are described in the above embodiments and will not be repeated here.

[0152] Optionally, the first display is used to display a preset image, the preset image having a cross pattern at its center; the control unit, when adjusting the pose of the binocular cameras to align each target camera with the optical axis of its corresponding optical component, is specifically used for:

[0153] Based on the display screens corresponding to the two target cameras, the six-degree-of-freedom turntable is controlled to adjust the yaw angle, pitch angle, and roll angle of the binocular camera, thereby adjusting the attitude of the binocular camera so that after the adjustment, the cross pattern in the display screen corresponding to either target camera is aligned with the corresponding physical cross.

[0154] The six-degree-of-freedom turntable is controlled to adjust the position of the binocular camera in the horizontal and vertical directions within the target plane. After the adjustment is completed, when the difference between the upper and lower widths of the blank area of ​​the display screen corresponding to any target camera satisfies the first preset condition, and the difference between the narrowest end and the widest end of the blank area of ​​the two display screens satisfies the second preset condition, it is confirmed that each target camera is aligned with the optical axis of the corresponding optical component. The target plane is perpendicular to the distance direction between the binocular camera and the optical component.

[0155] Optionally, the six-degree-of-freedom turntable includes a stepper motor, which, when driving the supported binocular camera to perform pose adjustment, is specifically used for:

[0156] The pose of the mounted binocular camera is adjusted by driving a stepper motor.

[0157] Optionally, the system further includes an interpupillary distance adjustment module located on the six-degree-of-freedom turntable, and the binocular camera is carried by the interpupillary distance adjustment module;

[0158] The control unit is connected to the interpupillary distance adjustment module and is used to adjust the distance between the two cameras to a first distance by controlling the interpupillary distance adjustment module, wherein the first distance is the distance between the two optical components.

[0159] Specifically, the interpupillary distance adjustment module is located between the six-degree-of-freedom turntable and the binocular camera. The six-degree-of-freedom turntable carries both the interpupillary distance adjustment module and the binocular camera, while the interpupillary distance adjustment module carries the binocular camera.

[0160] Optionally, the interpupillary distance adjustment module is a lead screw adjustment mechanism.

[0161] For example, the interpupillary distance adjustment module can be a lead screw slide.

[0162] This application embodiment also provides another alignment system for a binocular camera and optical components. The optical components are located in a head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The optical components are used to project light emitted from the first display onto the corresponding target cameras. The system includes:

[0163] A clamp is used to secure the head-mounted display device;

[0164] A six-degree-of-freedom turntable is used to receive control commands from the control unit and drive the mounted binocular camera to adjust its pose.

[0165] A control unit is used to perform the method described in any of the above embodiments.

[0166] The specific implementation principle and technical effects of the binocular camera and optical component alignment system provided in this application embodiment are described in the above embodiments and will not be repeated here.

[0167] The following is another method for aligning a binocular camera with optical components, provided in an embodiment of this application. The specific steps are as follows:

[0168] 1. Calibration of internal and external parameters for binocular cameras

[0169] (1) Calibrate the intrinsic and extrinsic parameters of the binocular camera using the initial interpupillary distance.

[0170] 2. Adaptation between binocular cameras and IPD (Inter-pupil Distance)

[0171] (1) Measure the IPD of the pancake;

[0172] (2) Adjust the relative distance between the binocular cameras using the lead screw slide to adapt to Pancake's IPD.

[0173] 3. Align the binocular camera with the Pancake.

[0174] (1) The rigidly connected (fixed X value) binocular camera is placed on a six-degree-of-freedom turntable.

[0175] (2) Adjust Roll / Yaw / Pitch and X / Y in sequence to ensure that the crosshair image captured by the binocular camera is aligned with the physical crosshair of the sensor, and find the optimal ER (Eye Relief) position by using the edge field of view illumination, and adjust the Z axis.

[0176] Figure 3 A schematic diagram illustrating the alignment of a binocular camera and a pancake optical engine is provided for an embodiment of this application, as shown below. Figure 3 As shown, by adjusting the position of the stereo camera on the Z-axis, the stereo camera moves from position 1 to position 2 and then to position 3, thereby determining the optimal distance between the stereo camera and the pancake optical engine.

[0177] Specific steps for aligning a binocular camera with a pancake

[0178] 1. Adjust yaw, pitch, and roll in sequence to align the crosshairs.

[0179] a. The angle difference between the physical crosshair of the sensor and the imaging crosshair should be kept within 5 pixels or 0.3% of the pixel value on one side as much as possible;

[0180] b. If there is a large difference between the left and right eye systems, the system should be adjusted as a whole to ensure that the images of both eyes meet condition a.

[0181] 2. Adjust x and y to make the image symmetrical horizontally and vertically.

[0182] a. The difference in width between the left and right cropping margins should be within 5 pixels or 0.3% of the width of a single side.

[0183] b. The difference in width between the top and bottom margins of the binocular image should also be within 5 pixels or 0.3% of the width of a single side pixel.

[0184] 3. Switch to a white background image, adjust the z-axis, and select an appropriate ER based on the extreme values ​​of illuminance at the edge of the field of view.

[0185] a. A suitable step value can be selected according to the control accuracy of the adjustment table, with a minimum step accuracy of 1mm required.

[0186] Figure 4 This is a schematic diagram showing the current display screens of the left and right target cameras after crosshair alignment, as provided in an embodiment of this application.

[0187] Explanation of selecting a suitable ER step through edge field-of-view illuminance extrema:

[0188] Adjust the Z-axis in 1mm increments, capture images at 1mm intervals, select the pixel rows where y=0 of these images, read the grayscale values ​​of the pixels in that row (showing a trend of high in the middle and low on both sides), calculate the sum of grayscale brightness values ​​of the edge field of view, and the Z-axis position corresponding to the image with the largest sum value is the Eye Relief position.

[0189] Optical path principle

[0190] Using the pancake-designed optical path example, the distance from the entrance pupil to the first lens was adjusted from 20 to 10 mm. The theoretical design value is 15 mm. The relative illumination (RI) performance was obtained from far to near.

[0191] Optical path change: The optical path at the edge of the field of view moves closer to the center of the optical axis. Figure 5 This application provides an edge field-of-view optical path diagram with an EyeRelief of 20mm as an embodiment. Figure 6 An edge field-of-view optical path diagram with an Eye Relief of 10mm is provided for an embodiment of this application.

[0192] Relative illumination change: As the distance increases, the relative illumination at the edge of the field of view first increases and then decreases. Figure 7 This application provides a schematic diagram of edge field of view illumination when Eye Relief is 20mm. Figure 8 This is a schematic diagram of edge field illumination when Eye Relief is 10mm, provided as an embodiment of this application.

[0193] Corresponding to the above-described alignment method between a binocular camera and optical components, this application also provides an alignment device for a binocular camera and optical components. The optical components are located in a head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. Figure 9 This is a schematic diagram of an alignment device for a binocular camera and optical components provided in an embodiment of this application. Figure 9 As shown, the device includes:

[0194] The first acquisition module 901 is used to acquire the first current display screen corresponding to the two target cameras respectively. The first current display screen is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera.

[0195] The adjustment module 902 is used to adjust the pose of the binocular cameras based on the first current display screen, so that each target camera is aligned with the optical axis of the corresponding optical component.

[0196] The determination module 903 is used to determine the final distance between the binocular camera and the optical component based on the grayscale brightness of each pixel in the edge region of the two second current display screens, wherein the two second current display screens are the current display screens corresponding to the two target cameras after alignment.

[0197] Optionally, the first acquisition module 901 is specifically used for:

[0198] Obtain a first distance and adjust the distance between the two target cameras to the first distance, wherein the first distance is the distance between the two optical components;

[0199] Obtain the first currently displayed image corresponding to each of the two target cameras after adjustment.

[0200] Optionally, the first display is used to display a preset image, the preset image having a cross pattern in the center; the adjustment module 902 is specifically used for:

[0201] Based on the two current first display images, adjust the yaw angle, pitch angle, and roll angle of the binocular camera;

[0202] After the adjustment is completed, the third current display screen corresponding to each of the two target cameras is obtained. When the cross pattern in each third current display screen is aligned with the corresponding physical cross, the attitude adjustment of the binocular camera is confirmed to be complete.

[0203] Based on two third current display screens, the position of the binocular camera in the horizontal and vertical directions is adjusted within the target plane, wherein the target plane is perpendicular to the distance direction between the binocular camera and the optical components;

[0204] After the adjustment is completed, obtain the fourth current display screen corresponding to each of the two target cameras;

[0205] When the difference in width between the upper and lower ends of the blank area in each fourth current display screen satisfies the first preset condition, and the difference between the narrowest end and the widest end of the blank areas in the two fourth current display screens satisfies the second preset condition, it is confirmed that the position of the binocular camera in the target plane has been adjusted, and the fourth current display screen is the second current display screen.

[0206] Optionally, the light projection angle range corresponding to the optical component is 0 to M, and the projection angle range corresponding to the edge region is kM to M, where k is a preset value. The determining module 903 is specifically used for:

[0207] Record the current position of the binocular camera;

[0208] For each second currently displayed frame, calculate the first sum of the grayscale brightness of each pixel in the edge region of the second currently displayed frame;

[0209] Add the first sums corresponding to the two second currently displayed screens to obtain the second sum;

[0210] Based on the second sum and the position of the binocular camera corresponding to the second sum, the final distance between the binocular camera and the optical components is determined.

[0211] Optionally, when determining the final distance between the binocular camera and the optical components based on the second sum and the position of the binocular camera corresponding to the second sum, the determining module 903 is specifically used for:

[0212] Repeat the following step-by-step operation until the preset number of repetitions is reached;

[0213] Based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, determine the final distance between the binocular cameras and the optical components;

[0214] The stepping operation includes:

[0215] Along the distance between the binocular camera and the optical components, move the binocular camera by a preset step length and record the position of the binocular camera after the movement;

[0216] Obtain the updated second current display screen corresponding to the two target cameras after the movement;

[0217] For each updated second current display screen, calculate the first sum of the grayscale brightness of each pixel in the edge region of the updated second current display screen;

[0218] Add the first sums corresponding to the two updated current display screens to obtain the second sum.

[0219] Optionally, when determining the final distance between the binocular camera and the optical components based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, the determining module 903 is specifically used for:

[0220] Based on multiple second sums and the positions of the stereo cameras corresponding to each second sum, a functional relationship graph between the second sums and the positions of the stereo cameras is determined; where the second sums are the vertical axis and the positions of the stereo cameras are the horizontal axis.

[0221] If the function graph shows a trend of first increasing and then decreasing, then the final distance between the binocular camera and the optical components is determined as the x-coordinate corresponding to the highest point of the function graph.

[0222] Optionally, when the adjustment module 902 confirms that the attitude adjustment of the binocular camera is complete after the crosshair pattern in each third current display screen is aligned with the corresponding physical crosshair, it is specifically used for:

[0223] When the two sets of optical components are parallel, the attitude adjustment of the binocular camera is confirmed to be complete when the cross pattern in each third current display screen is aligned with the corresponding physical cross.

[0224] Correspondingly, adjustment module 902 is also used for:

[0225] When the two sets of optical components are not parallel, if the cross pattern in one of the third current display screens is located to the left of the corresponding physical cross, and the cross pattern in the other third current display screen is located to the right of the corresponding physical cross, and the distance between the cross patterns in the two third current display screens and the corresponding physical cross is equal, then the attitude adjustment of the binocular camera is confirmed to be complete.

[0226] The specific implementation principle and effect of the alignment device for binocular cameras and optical components provided in this application embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0227] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device in this embodiment may include:

[0228] At least one processor 1001; and

[0229] Memory 1002 communicatively connected to the at least one processor;

[0230] The memory 1002 stores instructions that can be executed by the at least one processor 1001, which, when executed by the at least one processor 1001, cause the electronic device to perform the method described in any of the above embodiments.

[0231] Alternatively, the memory 1002 can be either standalone or integrated with the processor 1001.

[0232] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0233] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the foregoing embodiments.

[0234] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0235] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0236] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0237] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The memory may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0238] The aforementioned storage media can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0239] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0240] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0241] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0242] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0243] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of aligning a binocular camera with an optical assembly, the method comprising: The optical components are located in the head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The method includes: The first current display image corresponding to each of the two target cameras is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera. Based on the first current display screen, the pose of the binocular cameras is adjusted so that each target camera is aligned with the optical axis of the corresponding optical component. The final distance between the binocular camera and the optical components is determined based on the grayscale brightness of each pixel in the edge region of the two second current display images. The two second current display images are the current display images corresponding to the two target cameras after alignment.

2. The method of claim 1, wherein, Obtain the first currently displayed image corresponding to each of the two target cameras, including: Obtain a first distance and adjust the distance between the two target cameras to the first distance, wherein the first distance is the distance between the two optical components; Obtain the first currently displayed image corresponding to each of the two target cameras after adjustment.

3. The method of claim 1, wherein, The first display is used to display a preset image, the preset image having a cross pattern in the center; The step of adjusting the pose of the binocular camera based on the first currently displayed image includes: Based on the two current first display images, adjust the yaw angle, pitch angle, and roll angle of the binocular camera; After the adjustment is completed, the third current display screen corresponding to each of the two target cameras is obtained. When the cross pattern in each third current display screen is aligned with the corresponding physical cross, the attitude adjustment of the binocular camera is confirmed to be complete. Based on two third current display screens, the position of the binocular camera in the horizontal and vertical directions is adjusted within the target plane, wherein the target plane is perpendicular to the distance direction between the binocular camera and the optical components; After the adjustment is completed, the fourth current display screen corresponding to the two target cameras is obtained respectively; when the difference between the widths of the upper and lower ends of the blank area of ​​each fourth current display screen meets the first preset condition, and the difference between the narrowest end and the widest end of the blank area of ​​the two fourth current display screens respectively meets the second preset condition, it is confirmed that the position of the binocular camera in the target plane has been adjusted, and the fourth current display screen is the second current display screen.

4. The method according to any one of claims 1-3, characterized in that, The optical component has a light projection angle range of 0 to M, and the edge region has a projection angle range of kM to M, where k is a preset value; The determination of the final distance between the binocular camera and the optical components based on the grayscale brightness of each pixel in the edge regions of the two second currently displayed images includes: Record the current position of the stereo camera; For each second currently displayed frame, calculate the first sum of the grayscale brightness of each pixel in the edge region of the second currently displayed frame; Add the first sums corresponding to the two second currently displayed screens to obtain the second sum; Based on the second sum and the position of the binocular camera corresponding to the second sum, the final distance between the binocular camera and the optical components is determined.

5. The method according to claim 4, characterized in that, Based on the second sum and the position of the binocular camera corresponding to the second sum, the final distance between the binocular camera and the optical components is determined, including: Repeat the following step-by-step operation until the preset number of repetitions is reached; Based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, determine the final distance between the binocular cameras and the optical components; The stepping operation includes: Along the distance between the binocular camera and the optical components, move the binocular camera by a preset step length and record the position of the binocular camera after the movement; Obtain the updated second current display screen corresponding to the two target cameras after the movement; For each updated second current display screen, calculate the first sum of the grayscale brightness of each pixel in the edge region of the updated second current display screen; Add the first sums corresponding to the two updated current display screens to obtain the second sum.

6. The method according to claim 5, characterized in that, Based on multiple second sums and the positions of the binocular cameras corresponding to each second sum, the final distance between the binocular cameras and the optical components is determined, including: Based on multiple second sums and the positions of the stereo cameras corresponding to each second sum, a functional relationship graph between the second sums and the positions of the stereo cameras is determined; where the second sums are the vertical axis and the positions of the stereo cameras are the horizontal axis. If the function graph shows a trend of first increasing and then decreasing, then the final distance between the binocular camera and the optical components is determined as the x-coordinate corresponding to the highest point of the function graph.

7. The method according to claim 3, characterized in that, Once the crosshair pattern in each of the third currently displayed frames is aligned with the corresponding physical crosshair, confirm that the binocular camera's attitude adjustment is complete, including: When the two sets of optical components are parallel, the attitude adjustment of the binocular camera is confirmed to be complete when the cross pattern in each third current display screen is aligned with the corresponding physical cross. Accordingly, the method further includes: When the two sets of optical components are not parallel, if the cross pattern in one of the third current display screens is located to the left of the corresponding physical cross, and the cross pattern in the other third current display screen is located to the right of the corresponding physical cross, and the distance between the cross patterns in the two third current display screens and the corresponding physical cross is equal, then the attitude adjustment of the binocular camera is confirmed to be complete.

8. An alignment system for a binocular camera and optical components, characterized in that, The optical components are located in the head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The alignment system includes: A clamp is used to secure the head-mounted display device; A six-degree-of-freedom turntable is used to support the stereo camera; A control unit, connected to the six-degree-of-freedom turntable, adjusts the pose of the binocular cameras by controlling the turntable to align each target camera with the optical axis of its corresponding optical component. Based on the grayscale brightness of each pixel in the edge region of the current display screen corresponding to each of the two target cameras, the control unit adjusts the distance between the binocular cameras and the optical component to achieve alignment. The current display screen is obtained by the optical component projecting light emitted from the first display onto the corresponding target camera.

9. The system according to claim 8, characterized in that, The first display is used to display a preset image, the preset image having a cross pattern in the center; the control unit, when adjusting the pose of the binocular cameras to align each target camera with the optical axis of its corresponding optical component, is specifically used for: Based on the display screens corresponding to the two target cameras, the six-degree-of-freedom turntable is controlled to adjust the yaw angle, pitch angle, and roll angle of the binocular camera, thereby adjusting the attitude of the binocular camera so that after the adjustment, the cross pattern in the display screen corresponding to either target camera is aligned with the corresponding physical cross. The six-degree-of-freedom turntable is controlled to adjust the position of the binocular camera in the horizontal and vertical directions within the target plane. After the adjustment is completed, when the difference between the upper and lower widths of the blank area of ​​the display screen corresponding to any target camera satisfies the first preset condition, and the difference between the narrowest end and the widest end of the blank area of ​​the two display screens satisfies the second preset condition, it is confirmed that each target camera is aligned with the optical axis of the corresponding optical component. The target plane is perpendicular to the distance direction between the binocular camera and the optical component.

10. The system according to claim 8, characterized in that, The six-degree-of-freedom turntable includes a stepper motor. When the six-degree-of-freedom turntable drives the supported binocular camera to perform pose adjustment, it is specifically used for: The pose of the mounted binocular camera is adjusted by driving a stepper motor.

11. The system according to claim 8, characterized in that, The system also includes an interpupillary distance adjustment module located on the six-degree-of-freedom turntable, and the binocular camera is carried by the interpupillary distance adjustment module; The control unit is connected to the interpupillary distance adjustment module and is used to adjust the distance between the two cameras to a first distance by controlling the interpupillary distance adjustment module, wherein the first distance is the distance between the two optical components.

12. The system according to claim 11, characterized in that, The interpupillary distance adjustment module is a lead screw adjustment mechanism.

13. An alignment device for a binocular camera and optical components, characterized in that, The optical components are located in the head-mounted display device, which includes two sets of optical components and a first display. The two sets of optical components correspond one-to-one with the two target cameras included in the binocular camera. The device includes: The first acquisition module is used to acquire the first current display screen corresponding to the two target cameras respectively. The first current display screen is obtained by the optical component projecting the light emitted by the first display onto the corresponding target camera. The adjustment module is used to adjust the pose of the binocular cameras based on the first current display screen, so that each target camera is aligned with the optical axis of the corresponding optical component. The determination module is used to determine the final distance between the binocular camera and the optical component based on the grayscale brightness of each pixel in the edge region of the two second current display screens, wherein the two second current display screens are the current display screens corresponding to the two target cameras after alignment.

14. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images