Calibration method, device and electronic device of head-mounted device

Abstract

The present disclosure provides a calibration method and device of a head-mounted device and an electronic device, relates to data processing, and aims to solve the problem that the calibration accuracy of a VR device is low due to the experience of a calibrator in calibrating the VR device in the prior art. The method comprises the following steps: obtaining a mechanical arm script of a mechanical arm for fixing a head-mounted device to be calibrated; obtaining a calibration image based on the mechanical arm script; determining actual distances between each target feature point in the calibration image and each target feature point other than the target feature point according to calibration parameters of the head-mounted device to be calibrated and the calibration image; and determining a calibration result of the head-mounted device to be calibrated according to theoretical distances and the actual distances between each target feature point and each target feature point other than the target feature point.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more particularly to calibration methods, apparatus and electronic devices for head-mounted devices. Background Technology

[0002] Currently, in head-mounted device calibration scenarios, head-mounted recognition devices require calibration at the factory. For example, when the head-mounted device is a Virtual Reality (VR) device, calibration personnel typically calibrate it based on their own experience. This reliance on individual experience among different calibration personnel leads to lower calibration accuracy for the VR device. Summary of the Invention

[0003] In view of this, this disclosure provides a calibration method, apparatus, and electronic device for head-mounted devices to address the problem in the prior art where calibration personnel rely on their experience when calibrating VR devices, resulting in low calibration accuracy of VR devices.

[0004] To achieve the above objectives, the present disclosure provides the following technical solution:

[0005] In a first aspect, this disclosure provides a calibration method for a head-mounted device, comprising: acquiring a robotic arm script for fixing a robotic arm to be calibrated; acquiring a calibration image based on the robotic arm script; determining the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point, according to the calibration parameters of the head-mounted device to be calibrated and the calibration image; and determining the calibration result of the head-mounted device to be calibrated based on the theoretical distance and the actual distance between each target feature point and each target feature point other than the target feature point.

[0006] As an optional embodiment of this disclosure, obtaining the robotic arm script for fixing the head-mounted device to be calibrated includes: obtaining device information of the head-mounted device to be calibrated; and determining the robotic arm script for fixing the head-mounted device to be calibrated based on the device information.

[0007] As an optional implementation of this disclosure, the device information includes the device model; determining the robotic arm script for fixing the head-mounted device to be calibrated based on the device information includes: querying the target script corresponding to the device information in a pre-configured correspondence based on the device model; and determining the robotic arm script for fixing the head-mounted device to be calibrated as the target script based on the target script.

[0008] As an optional implementation of this disclosure, acquiring calibration images based on a robotic arm script includes: controlling the robotic arm to move to at least one target position according to the robotic arm script; at each target position, controlling the head-mounted device to be calibrated to photograph the calibration plate and acquiring the calibration image photographed by the head-mounted device to be calibrated.

[0009] As an optional implementation of this disclosure, determining the actual distance between each target feature point and each other target feature point in the calibration image based on the calibration parameters of the head-mounted device to be calibrated and the calibration image includes: determining the world coordinates corresponding to each target feature point in the calibration image based on the calibration parameters of the head-mounted device to be calibrated and the calibration image; and determining the actual distance between each target feature point in the calibration image and each other target feature point in the calibration image based on the world coordinates.

[0010] As an optional implementation of this disclosure, determining the calibration result of the head-mounted device to be calibrated based on the theoretical distance and actual distance between each target feature point and each other target feature point includes: determining the actual difference between each target feature point and each other target feature point based on the theoretical distance and actual distance between each target feature point and each other target feature point; and determining the calibration result of the head-mounted device to be calibrated based on the actual difference.

[0011] As an optional implementation of this disclosure, determining the calibration result of the head-mounted device to be calibrated based on the actual difference includes: determining the calibration result of the head-mounted device to be calibrated as having normal accuracy when the actual difference is less than or equal to a preset threshold; and determining the calibration result of the head-mounted device to be calibrated as having abnormal accuracy when the actual difference is greater than the preset threshold.

[0012] As an optional implementation of this disclosure, after determining the calibration result of the head-mounted device to be calibrated based on the theoretical distance and actual distance between each target feature point and each target feature point other than the target feature point, the calibration method of the head-mounted device provided in this disclosure further includes: resetting the robotic arm.

[0013] Secondly, this disclosure provides a calibration apparatus for a head-mounted device, comprising: an acquisition unit for acquiring a robotic arm script of a robotic arm that fixes the head-mounted device to be calibrated; a processing unit for acquiring a calibration image based on the robotic arm script acquired by the acquisition unit; the processing unit is further configured to determine, according to the calibration parameters of the head-mounted device to be calibrated and the calibration image, the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point; and the processing unit is further configured to determine the calibration result of the head-mounted device to be calibrated based on the theoretical distance and the actual distance between each target feature point and each target feature point other than the target feature point.

[0014] As an optional implementation of this disclosure, the acquisition unit is specifically used to acquire device information of the head-mounted device to be calibrated; the processing unit is specifically used to determine the robotic arm script for fixing the robotic arm of the head-mounted device to be calibrated based on the device information acquired by the acquisition unit.

[0015] As an optional implementation of this disclosure, the device information includes a device model; a processing unit is specifically used to query a target script corresponding to the device information in a pre-configured correspondence based on the device model obtained by the acquisition unit; and a processing unit is specifically used to determine the robotic arm script for fixing the robotic arm of the head-mounted device to be calibrated as the target script based on the target script.

[0016] As an optional implementation of this disclosure, the processing unit is specifically configured to control the robotic arm to move to at least one target position according to the robotic arm script acquired by the acquisition unit; the processing unit is specifically configured to control the head-mounted device to be calibrated to photograph the calibration plate at each target position and acquire the calibration image photographed by the head-mounted device to be calibrated.

[0017] As an optional implementation of this disclosure, the processing unit is specifically used to determine the world coordinates corresponding to each target feature point in the calibration image based on the calibration parameters of the head-mounted device to be calibrated and the calibration image; the processing unit is specifically used to determine the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point based on the world coordinates.

[0018] As an optional implementation of this disclosure, the processing unit is specifically used to determine the actual difference based on the theoretical distance and actual distance between each target feature point and each target feature point other than the target feature point; the processing unit is specifically used to determine the calibration result of the head-mounted device to be calibrated based on the actual difference.

[0019] As an optional implementation of this disclosure, the processing unit is specifically used to determine that the calibration result of the head-mounted device to be calibrated is of normal accuracy when the actual difference is less than or equal to a preset threshold; the processing unit is specifically used to determine that the calibration result of the head-mounted device to be calibrated is of abnormal accuracy when the actual difference is greater than a preset threshold.

[0020] As an optional embodiment of this disclosure, the processing unit is also used to reset the robotic arm.

[0021] Thirdly, this disclosure provides an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to cause the electronic device to implement the calibration method for the head-mounted device provided in the first aspect above when executing the computer program.

[0022] Fourthly, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a computing device, causes the computing device to implement the calibration method for the head-mounted device as described in the first aspect above.

[0023] Fifthly, this disclosure provides a computer program product, characterized in that, when the computer program product is run on a computer, it enables the computer to implement the calibration method for the head-mounted device as described in the first aspect above.

[0024] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on the first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the calibration device for the head-mounted device, or it may be packaged separately from the processor of the calibration device for the head-mounted device; this disclosure does not limit this.

[0025] The descriptions of the second, third, fourth, and fifth aspects in this disclosure can be referenced to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second, third, fourth, and fifth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0026] In this disclosure, the name of the calibration device for the aforementioned head-mounted device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this disclosure, it falls within the scope of this disclosure and its equivalents.

[0027] These or other aspects of this disclosure will become more readily apparent in the following description.

[0028] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0029] Each time a head-mounted device is calibrated, it can be fixed to a robotic arm, and the robotic arm's script can be acquired, allowing for the quantitative acquisition of calibration images. Then, based on the calibration parameters of the head-mounted device and the calibration images, the actual distance between each target feature point in the calibration images and every other target feature point is determined. Based on the theoretical and actual distances between each target feature point and every other target feature point, the calibration result of the head-mounted device is determined, ensuring the accuracy of the calibration results.

[0030] Furthermore, when the headset to be calibrated is a VR device, it can be fixed to a robotic arm, and calibration images can be acquired based on a robotic arm script. The calibration results can then be obtained by analyzing these images. This avoids relying on the experience of calibration personnel to calibrate VR devices, thus solving the problem of low calibration accuracy caused by the reliance on the experience of calibration personnel in existing technologies. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0032] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A schematic diagram illustrating a scenario for a calibration method for a head-mounted device provided in an embodiment of this disclosure;

[0034] Figure 2 This is one of the flowcharts illustrating a calibration method for a head-mounted device provided in this embodiment of the disclosure;

[0035] Figure 3 A second schematic flowchart illustrating the calibration method for a head-mounted device provided in this embodiment of the present disclosure;

[0036] Figure 4 A third schematic flowchart illustrating the calibration method for a head-mounted device provided in this embodiment of the present disclosure;

[0037] Figure 5 Fourth schematic flowchart of the calibration method for a head-mounted device provided in this embodiment of the disclosure;

[0038] Figure 6 Fifth schematic flowchart of the calibration method for a head-mounted device provided in the embodiments of this disclosure;

[0039] Figure 7 A schematic diagram illustrating the calculation of the actual position in the calibration method for a head-mounted device provided in this embodiment of the disclosure;

[0040] Figure 8 A schematic flowchart of the calibration method for a head-mounted device provided in this embodiment of the present disclosure is shown in Figure 6.

[0041] Figure 9 Seventh schematic flowchart of the calibration method for a head-mounted device provided in this embodiment of the disclosure;

[0042] Figure 10 Eighth schematic flowchart of the calibration method for a head-mounted device provided in this embodiment of the disclosure;

[0043] Figure 11 This is a schematic diagram of the structure of the calibration device for a head-mounted device provided in an embodiment of the present disclosure;

[0044] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure;

[0045] Figure 13 This is a schematic diagram of the structure of a computer program product for a calibration method of a head-mounted device provided in an embodiment of this disclosure. Detailed Implementation

[0046] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0047] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0048] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0049] Figure 1 This is a schematic diagram of a scenario for a calibration method for a head-mounted device provided in an embodiment of the present disclosure, including: a calibration plate 1, a robotic arm 2, a head-mounted device 3 to be calibrated, and a main control device 4. A holder 2-1 is provided on the robotic arm 2, and the head-mounted device 3 to be calibrated is fixed on the holder 2-1.

[0050] In some examples, both the robotic arm 2 and the head-mounted device 3 to be calibrated are connected to the main control device 4 via wired or wireless means. This allows the main control device 4 to control the movement of the robotic arm 2 and simultaneously control the head-mounted device 3 to photograph the calibration board 1. Alternatively, the robotic arm 2 is connected to the main control device 4 via wired or wireless means, and the head-mounted device 3 to be calibrated is connected to the robotic arm 2 via wired or wireless means. In this case, the main control device 4 can control the movement of the robotic arm 2 while simultaneously controlling the head-mounted device 3 to photograph the calibration board 1.

[0051] For example, taking the example of a robotic arm 2 connected to the main control device 4 via a wired or wireless connection, and a head-mounted device 3 to be calibrated connected to the robotic arm 2 via a wired or wireless connection, after fixing the head-mounted device 3 to the fixture 2-1, the head-mounted device 3 needs to be powered on to ensure it is powered on. Then, the main control device 4 obtains the robotic arm script and controls the robotic arm 2 to run according to the script. When the robotic arm 2 reaches the target position, the main control device 4 controls the head-mounted device 3 to photograph the calibration board 1 and acquire the calibration image captured by the head-mounted device 3. After acquiring the calibration images captured by the head-mounted device 3 at each target position, the main control device 4 determines the actual distance between any two target feature points in the calibration image based on the calibration parameters of the head-mounted device and the calibration image, such as the actual distance between target feature point A (corresponding to calibration pattern A in the calibration board) and target feature point B (corresponding to calibration pattern B in the calibration board) in the calibration image. Since the distance between calibration pattern A and calibration pattern B in calibration plate 1 is known, the calibration result of the head-mounted device to be calibrated can be determined based on the theoretical and actual distances between any two target feature points.

[0052] Specifically, robotic arm 2 can move up, down, left, and right, as well as rotate, such as a nine-axis robotic arm. The main control device 4 can be an electronic device or a server, including any of the following: a personal digital assistant (PDA) computer, a tablet computer, or a laptop computer. The head-mounted device 3 to be calibrated can be any of the following: an augmented reality (AR) device, a VR device, a mixed reality (MR) device, a drone, a robot, an autonomous vehicle, or any other device requiring calibration.

[0053] It should be noted that the above example is based on the case where calibration plate 1 contains only 2 target feature points and the head-mounted device 3 to be calibrated contains 2 image acquisition devices for acquiring images of the current environment. In other examples, calibration plate 1 contains 3 or more target feature points and / or the head-mounted device 3 to be calibrated contains 3 or more image acquisition devices for acquiring images of the current environment. In this case, when determining the actual distance between any two target feature points in the calibration image, feature point matching must first be performed on the calibration images captured by any two image acquisition devices. Then, based on the calibration images with matched feature points, the actual distance between any two target feature points is calculated. Finally, the calibration result of the head-mounted device to be calibrated is determined based on the theoretical distance and the actual distance between any two target feature points.

[0054] For example, taking the main control device 4 as the execution subject of the calibration method for the head-mounted device provided in this embodiment of the disclosure, the calibration method for the head-mounted device provided in this embodiment of the disclosure will be described, and the specific implementation process is as follows:

[0055] Figure 2 This is a flowchart illustrating a calibration method for a head-mounted device according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following S11-S14.

[0056] S11. Obtain the robotic arm script for the robotic arm that is fixed to the head-mounted device to be calibrated.

[0057] In some examples, by fixing the head-mounted device to be calibrated onto a robotic arm, the arm can be controlled to move to a specific point, allowing the head-mounted device to capture a calibration image of the calibration plate. This allows the actual distance between any two target feature points in the calibration image to be determined based on the calibration parameters of the head-mounted device and the calibration image. Furthermore, the calibration result of the head-mounted device can be determined based on the theoretical and actual distances between these two target feature points. This provides a quantifiable indicator to reflect the calibration accuracy of the head-mounted device.

[0058] S12. Obtain calibration images based on the robotic arm script.

[0059] In some examples, different robotic arm scripts correspond to different calibration boards. When the robotic arm runs the script, it can move to the corresponding calibration board, allowing the head-mounted device to capture an image of the calibration board and obtain the corresponding calibration image. The calibration image is then retrieved from the head-mounted device's memory (e.g., a Secure Digital Memory Card, SD card).

[0060] S13. Based on the calibration parameters of the head-mounted device to be calibrated and the calibration image, determine the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point.

[0061] In some examples, combined Figure 1 The given scenario diagram shows that the calibration image acquired by the head-mounted device to be calibrated contains only two target feature points, namely target feature point A and target feature point B. At this time, it is only necessary to calculate the actual distance between target feature point A and target feature point B.

[0062] In other examples, when the calibration image acquired by the head-mounted device to be calibrated contains only three target feature points, namely target feature point A, target feature point B, and target feature point C, it is necessary to calculate the actual distance 1 between target feature point A and target feature point B, the actual distance 2 between target feature point A and target feature point C, and the actual distance 3 between target feature point B and target feature point C. Then, based on the theoretical distance 1 between target feature point A and target feature point B, the theoretical distance 2 between target feature point A and target feature point C, the theoretical distance 3 between target feature point B and target feature point C, and the actual distances 1, 2, and 3, the calibration result of the head-mounted device to be calibrated is determined.

[0063] It can be seen that when the calibration image contains three or more target feature points, it is necessary to calculate the actual distance between each target feature point and every other feature point. Based on the theoretical distance between each target feature point and every other feature point, and the actual distance, the calibration result of the head-mounted device to be calibrated is determined.

[0064] Specifically, the calibration image includes at least two target feature points. For example, if the calibration image includes three target feature points, namely target feature point A, target feature point B, and target feature point C, then when target feature point A is selected as the feature point to be calculated, each target feature point other than the target feature point includes target feature point B and target feature point C. When target feature point B is selected as the feature point to be calculated, each target feature point other than the target feature point includes target feature point A and target feature point C. When target feature point C is selected as the feature point to be calculated, each target feature point other than the target feature point includes target feature point A and target feature point B.

[0065] S14. Determine the calibration result of the head-mounted device to be calibrated based on the theoretical distance and actual distance between each target feature point and each target feature point other than the target feature point.

[0066] In some examples, referring to the example given in S13 above, when the calibration image contains only two target feature points, the calibration result of the head-mounted device to be calibrated only needs to be determined based on the theoretical distance and the actual distance corresponding to the two target feature points. For example, the actual difference is determined based on the absolute value of the difference between the theoretical distance (e.g., 2 meters) and the actual distance. Then, the calibration result is determined based on the relationship between the actual difference and a preset threshold. Alternatively, in the pre-configured correspondence between difference intervals and calibration results, the calibration result corresponding to the difference interval into which the actual difference falls is queried, and the calibration result corresponding to that difference interval is used as the calibration result of the head-mounted device to be calibrated.

[0067] In other examples, referring to the example given in S13 above, when the calibration image contains only three or more target feature points, such as target feature point A, target feature point B, and target feature point C, it is necessary to calculate the actual difference 1 between theoretical distance 1 and actual distance 1, the actual difference 2 between theoretical distance 2 and actual distance 2, and the actual difference 3 between theoretical distance 3 and actual distance 3. Then, based on the actual difference 1, actual difference 2, and actual difference 3, the calibration result of the head-mounted device to be calibrated is determined. For example, the calibration result is determined based on the relationship between the actual difference 1, actual difference 2, and actual difference 3 and a preset threshold. Alternatively, in the pre-configured correspondence between difference intervals and calibration results, the calibration result corresponding to the difference interval into which the average value of the actual difference 1, actual difference 2, and actual difference 3 falls is queried, and the calibration result corresponding to that difference interval is used as the calibration result of the head-mounted device to be calibrated.

[0068] It should be noted that the above example is based on the premise that the calibration images of the target feature points contained in the calibration board are the same. In other examples, the calibration images of the target feature points contained in the calibration board may be different, thereby facilitating the identification of the target feature points.

[0069] As described above, the calibration method for a head-mounted device provided in this disclosure can quantitatively acquire calibration images by fixing the head-mounted device to be calibrated onto a robotic arm and obtaining the robotic arm's script. Then, based on the calibration parameters of the head-mounted device and the calibration images, the actual distance between each target feature point in the calibration images and every other target feature point is determined. Based on the theoretical and actual distances between each target feature point and every other target feature point, the calibration result of the head-mounted device is determined, ensuring the accuracy of the calibration results.

[0070] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 3As shown, the above S11 can be implemented by the following S110 and S111.

[0071] S110. Obtain device information for the head-mounted device to be calibrated.

[0072] In some examples, the device information for the head-mounted device to be calibrated can be input by the user into the main control device 4, or the main control device 4 can obtain the device information by capturing images of the head-mounted device from different angles and then importing these images into a pre-configured recognition model. The training process of the pre-configured recognition model is as follows:

[0073] Acquire training sample images and annotation results for the training sample images; wherein, the training sample images include historical product images of the head-mounted devices to be calibrated, and the annotation results include device information of the historical head-mounted devices to be calibrated.

[0074] The training sample images are input into the deep learning model.

[0075] Based on the objective loss function, it is determined whether the prediction comparison results of the deep learning model output on the training sample images match the annotation results.

[0076] When the prediction comparison results do not match the annotation results, the network parameters of the deep learning model are iteratively updated repeatedly until the model converges and a pre-configured recognition model is obtained.

[0077] S111. Based on the equipment information, determine the robotic arm script for fixing the robotic arm of the head-mounted device to be calibrated.

[0078] Specifically, when the robotic arm is used only to fix a head-mounted device with one type of equipment information to be calibrated, then the robotic arm corresponds to only one target script. In this way, each time the head-mounted device to be calibrated is fixed to the robotic arm, the target script can be directly obtained and used as the robotic arm script for fixing the head-mounted device to be calibrated.

[0079] When a robotic arm can be used to fix two or more head-mounted devices with different device information to be calibrated, since the robotic arm scripts corresponding to the head-mounted devices with different device information are different, it is necessary to store the target scripts corresponding to the head-mounted devices with different device information in the memory of the main control device 4 in advance. In this way, after fixing the head-mounted device to be calibrated on the robotic arm, the main control device 4 can find the target script corresponding to the device information in the memory according to the device information of the head-mounted device to be calibrated, and then use the target script as the robotic arm script for fixing the head-mounted device to be calibrated.

[0080] As an optional implementation of this disclosure, the device information includes the device model; combined with Figure 3 ,like Figure 4 As shown, the above S111 can be specifically implemented through the following S1110 and S1111.

[0081] S1110. Query the target script corresponding to the device information in the pre-configured correspondence based on the device model.

[0082] S1111. Based on the target script, determine the robotic arm script for fixing the head-mounted device to be calibrated as the target script.

[0083] It should be noted that the above example uses device information including the device model as an example. In other examples, device information can also be information used to uniquely identify the head-mounted device to be calibrated, such as a device identification code; this is not limited to this example.

[0084] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 5 As shown, the above S12 can be implemented by the following S120 and S121.

[0085] S120. According to the robotic arm script, control the robotic arm to move to at least one target position.

[0086] In some examples, when the head-mounted device to be calibrated is positioned at the target location and photographing the calibration board, all image acquisition devices on the head-mounted device, such as cameras, can simultaneously capture the calibration patterns on the calibration board. Each calibration pattern corresponds to a target feature point. For example, combining... Figure 1 The given scenario diagram shows that calibration board 1 contains two calibration patterns, namely calibration pattern A and calibration pattern B. When the head-mounted device to be calibrated is at the target position and taking pictures of the calibration board, all image acquisition devices set on the head-mounted device to be calibrated, such as cameras, can simultaneously capture calibration pattern A and calibration pattern B in calibration board 1, which is convenient for subsequent calculation of the actual distance.

[0087] For example, in combination Figure 1The given scenario diagram, assuming the head-mounted device to be calibrated contains four image acquisition devices: image acquisition device 0, image acquisition device 1, image acquisition device 2, and image acquisition device 3, can be grouped into one group, denoted as 10; and image acquisition devices 2 and 3 can be grouped into another group, denoted as 32. 10 corresponds to a target position 1, allowing image acquisition devices 0 and 1 to simultaneously acquire calibration patterns A and B on calibration plate 1. Similarly, 32 corresponds to a target position 1, allowing image acquisition devices 2 and 3 to simultaneously acquire calibration patterns A and B on calibration plate 1.

[0088] S121. At each target location, control the head-mounted device to be calibrated to photograph the calibration plate and acquire the calibration image photographed by the head-mounted device to be calibrated.

[0089] In some examples, when the image format of the calibration image captured by the head-mounted device to be calibrated differs from the image format recognizable by the main control device 4, it is necessary to convert the image format of the calibration image captured by the head-mounted device to be calibrated so that the main control device 4 can directly recognize the calibration image.

[0090] Specifically, when photographing the calibration plate using the head-mounted device to be calibrated, multiple photographs of the calibration plate can be taken (e.g., 30 times) to obtain multiple calibration images. Then, the best calibration image among these images is selected for identification, avoiding inaccurate calibration results due to blurry images.

[0091] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 6 As shown, the above S13 can be implemented by the following S130 and S131.

[0092] S130. Based on the calibration parameters of the head-mounted device to be calibrated and the calibration image, determine the world coordinates corresponding to each target feature point in the calibration image.

[0093] Specifically, the calibration parameters include camera intrinsic and extrinsic parameters. The process of determining the world coordinates of each target feature point in the calibration image based on the calibration parameters of the head-mounted device to be calibrated and the calibration image is as follows:

[0094] Combining the example given in S120 above, and Figure 7 The given diagram assumes that the pixel coordinates of the target feature point A in the calibration image captured by cam0 are pixel coordinates A. Based on the camera intrinsic parameters of cam0 and pixel coordinates A, the three-dimensional coordinates of the target feature point A in the camera coordinate system of cam0 are determined to be A-cam0.

[0095] Based on the camera extrinsic parameters between cam0 and cam1, the target feature point A is transformed into its corresponding three-dimensional coordinates in the camera coordinate system of cam1, which is A-cam1.

[0096] Assuming the pixel coordinates of the target feature point A in the calibration image captured by cam1 are pixel coordinates A', then using the camera intrinsic parameters of cam1 and pixel coordinates A', the three-dimensional coordinates of the target feature point A in the camera coordinate system of cam1 are determined to be A'-cam1.

[0097] Line connecting point A-cam0 to the optical center of cam0 forms ray L1, and line connecting point A'-cam1 to the optical center of cam1 forms ray L2. Then, the intersection of ray L1 and ray L2 determines Point_A. Similarly, Point_B can be obtained.

[0098] Next, the Euclidean distance between the world coordinates of point Point_A and point Point_B is calculated, and this Euclidean distance is used as the actual distance between target feature point A and target feature point B in the calibration image captured by cam0.

[0099] As can be seen, this only describes how to calculate the actual distance between target feature point A and target feature point B in the calibration image captured by cam0. The calculation methods for calculating the actual distance between target feature point A and target feature point B in the calibration image captured by cam1, cam2, and cam3 are the same as those for calculating the actual distance between target feature point A and target feature point B in the calibration image captured by cam0, and will not be repeated here.

[0100] It should be noted that when calculating the intersection of light rays L1 and L2, it is necessary to ensure that point A-cam0 and point A'-cam1 are in the same camera coordinate system. For example, here they are both transformed into the cam1 coordinate system to process the intersection of light rays L1 and L2. Therefore, the origin of L1 is the position of the entire camera coordinate system of cam0 in the camera coordinate system of cam1. That is, if you consider cam0 as a point, its coordinates in cam1 are T[0], T[1], T[2]; the origin of L2 is the origin of cam1 itself, because it is now in the camera coordinate system of cam1.

[0101] S131. Based on world coordinates, determine the actual distance between each target feature point in the calibration image and every other target feature point.

[0102] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 8 As shown, the above S14 can be implemented by the following S140 and S141.

[0103] S140. Based on the theoretical distance and actual distance between each target feature point and each target feature point other than the target feature point, determine the actual difference between each target feature point and each target feature point other than the target feature point.

[0104] S141. Determine the calibration result of the head-mounted device to be calibrated based on the actual difference.

[0105] As an optional implementation of this disclosure, combined with Figure 8 ,like Figure 9 As shown, the above S141 can be specifically implemented through the following S1410 and S1411.

[0106] S1410. If the actual difference is less than or equal to the preset threshold, the calibration result of the head-mounted device to be calibrated is determined to be of normal accuracy.

[0107] S1411. If the actual difference is greater than the preset threshold, the calibration result of the head-mounted device to be calibrated is determined to be of abnormal accuracy.

[0108] As can be seen, when calibrating a head-mounted device using the calibration device provided in this embodiment, the quantitative index of the positioning system accuracy of the head-mounted device before it leaves the factory can be calculated, and the calibration result of the head-mounted device can be directly measured, thereby improving the quality of shipment and reducing repair costs.

[0109] As an optional implementation of this disclosure, combined with Figure 2 ,like Figure 10 As shown, the calibration method for the head-mounted device provided in this embodiment of the present disclosure requires execution of S15 after S14 is completed.

[0110] S15. Reset the robotic arm.

[0111] In some examples, in order to continue using the robotic arm next time, S15 needs to be executed after completing S11-S14. In this way, the robotic arm can be reset after obtaining the calibration result of the head-mounted device to be calibrated each time, which facilitates the next calibration of the head-mounted device to be calibrated.

[0112] The foregoing primarily describes the solutions provided by the embodiments of the present invention from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0113] In this embodiment of the invention, the calibration device for a head-mounted device can be divided into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0114] like Figure 11 As shown, an embodiment of the present invention provides a schematic diagram of the structure of a calibration device 10 for a head-mounted device. The calibration device 10 for the head-mounted device includes an acquisition unit 101 and a processing unit 102.

[0115] The acquisition unit is used to acquire the robotic arm script of the robotic arm that fixes the head-mounted device to be calibrated; the processing unit is used to acquire the calibration image based on the robotic arm script acquired by the acquisition unit; the processing unit is also used to determine the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point, according to the calibration parameters of the head-mounted device to be calibrated and the calibration image; the processing unit is also used to determine the calibration result of the head-mounted device to be calibrated according to the theoretical distance and the actual distance between each target feature point and each target feature point other than the target feature point.

[0116] As an optional implementation of this disclosure, the acquisition unit is specifically used to acquire device information of the head-mounted device to be calibrated; the processing unit is specifically used to determine the robotic arm script for fixing the robotic arm of the head-mounted device to be calibrated based on the device information acquired by the acquisition unit.

[0117] As an optional implementation of this disclosure, the device information includes a device model; a processing unit is specifically used to query a target script corresponding to the device information in a pre-configured correspondence based on the device model obtained by the acquisition unit; and a processing unit is specifically used to determine the robotic arm script for fixing the robotic arm of the head-mounted device to be calibrated as the target script based on the target script.

[0118] As an optional implementation of this disclosure, the processing unit is specifically configured to control the robotic arm to move to at least one target position according to the robotic arm script acquired by the acquisition unit; the processing unit is specifically configured to control the head-mounted device to be calibrated to photograph the calibration plate at each target position and acquire the calibration image photographed by the head-mounted device to be calibrated.

[0119] As an optional implementation of this disclosure, the processing unit is specifically used to determine the world coordinates of each target feature point in the calibration image based on the calibration parameters of the head-mounted device to be calibrated and the calibration image; the processing unit is specifically used to determine the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point based on the world coordinates.

[0120] As an optional implementation of this disclosure, the processing unit is specifically used to determine the actual difference based on the theoretical distance and actual distance between each target feature point and each target feature point other than the target feature point; the processing unit is specifically used to determine the calibration result of the head-mounted device to be calibrated based on the actual difference.

[0121] As an optional implementation of this disclosure, the processing unit is specifically used to determine that the calibration result of the head-mounted device to be calibrated is of normal accuracy when the actual difference is less than or equal to a preset threshold; the processing unit is specifically used to determine that the calibration result of the head-mounted device to be calibrated is of abnormal accuracy when the actual difference is greater than a preset threshold.

[0122] As an optional embodiment of this disclosure, the processing unit is also used to reset the robotic arm.

[0123] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and their functions will not be repeated here.

[0124] Of course, the calibration device 10 for the head-mounted device provided in this embodiment of the invention includes, but is not limited to, the modules described above. For example, the calibration device 10 for the head-mounted device may also include a storage unit 103. The storage unit 103 may be used to store the program code of the calibration device 10 for the head-mounted device, and may also be used to store data generated by the calibration device 10 for the head-mounted device during operation, such as data in write requests.

[0125] Figure 12This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 12 As shown, the electronic device may include at least one processor 51, a memory 52, a communication interface 53, and a communication bus 54.

[0126] The following is combined Figure 12 A detailed introduction to each component of the electronic device:

[0127] The processor 51 is the control center of the electronic device. It can be a single processor or a collective term for multiple processing elements. For example, the processor 51 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field-programmable gate arrays (FPGAs).

[0128] In a specific implementation, as one example, processor 51 may include one or more CPUs, for example... Figure 12 CPU0 and CPU1 are shown in the diagram. Furthermore, as one embodiment, the electronic device 10 may include multiple processors, such as... Figure 12 The processors 51 and 56 shown are illustrated. Each of these processors can be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0129] The memory 52 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via the communication bus 54. The memory 52 may also be integrated with the processor 51.

[0130] In a specific implementation, memory 52 is used to store data from this invention and the software program for executing this invention. Processor 51 can perform various functions of the air conditioner by running or executing the software program stored in memory 52 and by calling the data stored in memory 52.

[0131] Communication interface 53, using any transceiver-like device, is used to communicate with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, the cloud, etc. Communication interface 53 may include an acquisition unit to implement acquisition functions.

[0132] The communication bus 54 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0133] As an example, combined Figure 11The acquisition unit 101 in the calibration device 10 of the head-mounted device performs the same function as... Figure 12 The communication interface 53 in the head-mounted device has the same function as the calibration device 10 processing unit 102 in the head-mounted device. Figure 12 The processor 51 in the head-mounted device has the same function as the calibration device 10 storage unit 103 in the head-mounted device. Figure 12 The memory 52 in it has the same function.

[0134] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a computing device, causes the computing device to implement the method shown in the above-described method embodiment.

[0135] In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0136] Figure 13 A conceptual partial view of a computer program product provided in an embodiment of the present invention is shown schematically. The computer program product includes a computer program for executing computer processes on a computing device.

[0137] In one embodiment, the computer program product is provided using signal bearer medium 410. Signal bearer medium 410 may include one or more program instructions that, when executed by one or more processors, can provide the above-mentioned... Figure 2 The described function or part of the function. Therefore, for example, refer to... Figure 2 In the embodiment shown, one or more features of S11-S14 can be provided by one or more instructions associated with the signal carrying medium 410. Furthermore, Figure 13 The program instructions in the document also describe example instructions.

[0138] In some examples, the signal carrying medium 410 may include a computer-readable medium 411, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0139] In some implementations, the signal carrying medium 410 may include a computer recordable medium 412, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0140] In some implementations, the signal carrying medium 410 may include a communication medium 413, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0141] The signal-bearing medium 410 can be transmitted by a wireless communication medium 413 (e.g., a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocols). One or more program instructions can be, for example, computer-executable instructions or logical implementation instructions.

[0142] In some examples, such as targeting Figure 11 The calibration device 10 of the described head-mounted device can be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 411, a computer-recordable medium 412, and / or a communication medium 413.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0144] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0148] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A calibration method for a head-mounted device, characterized in that, include: Obtain the robotic arm script for the robotic arm that is holding the head-mounted device to be calibrated; Based on the robotic arm script, obtain the calibration image; Based on the calibration parameters of the head-mounted device to be calibrated and the calibration image, determine the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point itself; The calibration result of the head-mounted device to be calibrated is determined based on the theoretical distance between each target feature point and each target feature point other than the target feature point and the actual distance.

2. The calibration method for a head-mounted device according to claim 1, characterized in that, The robotic arm script for acquiring the robotic arm that fixes the head-mounted device to be calibrated includes: Obtain the device information of the head-mounted device to be calibrated; Based on the device information, determine the robotic arm script for fixing the robotic arm of the head-mounted device to be calibrated.

3. The calibration method for a head-mounted device according to claim 2, characterized in that, The equipment information includes the equipment model; The step of determining the robotic arm script for fixing the head-mounted device to be calibrated based on the device information includes: Based on the device model, query the target script corresponding to the device information in the pre-configured correspondence; Based on the target script, the robotic arm script for fixing the robotic arm of the head-mounted device to be calibrated is determined as the target script.

4. The calibration method for a head-mounted device according to claim 1, characterized in that, The step of acquiring the calibration image based on the robotic arm script includes: According to the robotic arm script, control the robotic arm to move to at least one target position; At each target location, the head-mounted device to be calibrated is controlled to photograph the calibration plate and acquire the calibration image captured by the head-mounted device to be calibrated.

5. The calibration method for a head-mounted device according to claim 1, characterized in that, The step of determining the actual distance between each target feature point in the calibration image and every other target feature point in the calibration image, based on the calibration parameters of the head-mounted device to be calibrated and the calibration image, includes: Based on the calibration parameters of the head-mounted device to be calibrated and the calibration image, determine the world coordinates corresponding to each target feature point in the calibration image; Based on the world coordinates, determine the actual distance between each target feature point in the calibration image and every other target feature point.

6. The calibration method for a head-mounted device according to claim 1, characterized in that, The step of determining the calibration result of the head-mounted device to be calibrated based on the theoretical distance between each target feature point and each target feature point other than the target feature point and the actual distance includes: Based on the theoretical distance between each target feature point and each target feature point other than the target feature point and the actual distance, determine the actual difference between each target feature point and each target feature point other than the target feature point; The calibration result of the head-mounted device to be calibrated is determined based on the actual difference.

7. The calibration method for a head-mounted device according to claim 6, characterized in that, Determining the calibration result of the head-mounted device to be calibrated based on the actual difference includes: If the actual difference is less than or equal to a preset threshold, the calibration result of the head-mounted device to be calibrated is determined to be of normal accuracy. If the actual difference is greater than the preset threshold, the calibration result of the head-mounted device to be calibrated is determined to be of abnormal accuracy.

8. The calibration method for a head-mounted device according to any one of claims 1-7, characterized in that, After determining the calibration result of the head-mounted device to be calibrated based on the theoretical distance between each target feature point and each target feature point other than the target feature point and the actual distance, the method further includes: Reset the robotic arm.

9. A calibration device for a head-mounted device, characterized in that, include: The acquisition unit is used to acquire the robotic arm script of the robotic arm that fixes the head-mounted device to be calibrated. The processing unit is used to acquire a calibration image based on the robotic arm script acquired by the acquisition unit; The processing unit is further configured to determine, based on the calibration parameters of the head-mounted device to be calibrated and the calibration image, the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point; The processing unit is further configured to determine the calibration result of the head-mounted device to be calibrated based on the theoretical distance between each target feature point and each target feature point other than the target feature point and the actual distance.

10. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to cause the electronic device to implement the calibration method for the head-mounted device according to any one of claims 1-8 when executing the computer program.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computing device, causes the computing device to implement the calibration method for the head-mounted device according to any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when run on a computer, causes the computer to implement the calibration method for the head-mounted device as described in any one of claims 1-8.

Images