Methods, systems, and recording media for accessory pairing

By detecting accessory markings and using image recognition technology, the system automatically adjusts the relative position and orientation of accessories to the user's body parts, solving the problems of manual accessory setting and incorrect wearing in existing technologies, and enabling convenient accessory pairing.

CN116069158BActive Publication Date: 2026-11-10HTC CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210811888.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2022-05-27
Filing Date
2022-07-11
Publication Date
2026-11-10
Estimated Expiration
2042-07-11

AI Technical Summary

Technical Problem

In existing virtual reality systems, pairing accessories with user body parts requires manual setup or can lead to inconvenience due to incorrect wearing, especially fixed accessories which are difficult to reposition.

Method used

By detecting the identification information of accessories and recognizing user body parts through images, the computing device determines the type of accessory and the location of the body part, and automatically adjusts the relative position and orientation of the accessory and the body part to achieve automatic pairing.

Benefits of technology

It simplifies the pairing process between accessories and user body parts, improves convenience and flexibility, and reduces the need for manual operation by the user.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116069158B_ABST
    Figure CN116069158B_ABST
Patent Text Reader

Abstract

This invention provides a method, system, and recording medium for accessory pairing. The method includes: determining the type of an accessory based on its identification information; identifying the type of a user's body part based on an image; in response to the accessory being suitable for the user's body part according to the type of accessory and the type of user's body part, obtaining a first position of the accessory in the environment and obtaining a second position of the user's body part in the environment; and configuring the accessory to pair with the user's body part according to the first position of the accessory and the second position of the user's body part.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method and system for virtual reality, and more particularly to a method, system, and recording medium for accessory pairing. Background Technology

[0002] Virtual reality (VR) is a technology that uses computer simulation to create virtual worlds in three-dimensional (3D) space. VR systems are typically compatible with several accessories (such as controllers, sensors, or speakers) to detect user movement or create an immersive environment. A VR system includes a head-mounted display and a computing device. The computing device is coupled to the head-mounted display and communicates with the accessories. The computing device can collect data from the accessories, enabling the user to interact with virtual objects in the virtual world based on the collected data.

[0003] For wearable or handheld accessories, if the accessory is not initially set to be associated with a specific body part (e.g., left or right hand), it generally needs to be manually set to be associated with the body part operated by the user. Conversely, if the accessory is initially set to be associated with a specific body part but the user wears it in the wrong position, it is necessary for the user to change the wearing position. However, for some accessories that require a secure fit (using a wristband or Velcro), changing the wearing position is particularly inconvenient. Summary of the Invention

[0004] This invention relates to a method, system, and recording medium for accessory pairing, which detects the position of the accessory and the position of a user's body part for pairing the accessory with the user's body part, thereby improving the convenience of accessory setup.

[0005] This invention provides an accessory pairing method. The method includes: determining the type of accessory based on the accessory's identification information; identifying the type of a user's body part based on an image; in response to the accessory being suitable for the user's body part according to the accessory type and the user's body part type, obtaining a first position of the accessory in the environment and obtaining a second position of the user's body part in the environment; and configuring the accessory to pair with the user's body part according to the first position of the accessory and the second position of the user's body part.

[0006] In an embodiment of the present invention, the step of determining the type of accessory based on identification information includes: obtaining accessory type information from the accessory's identification information, wherein the accessory type information indicates the function of the accessory; and determining the type of accessory based on the accessory type information.

[0007] In an embodiment of the present invention, the step of identifying the type of user body part in an image includes: performing image recognition on the image to identify the type of user body part in the image.

[0008] In an embodiment of the present invention, the method further includes: defining candidate regions in an image corresponding to user body parts; and defining reference points based on the candidate regions to indicate the location of the user body parts within the environment.

[0009] In an embodiment of the present invention, the method further includes: converting the coordinates of a reference point in an image into the position of a user's body part within the environment.

[0010] In embodiments of the present invention, the first position of the accessory includes at least one of the accessory's position and the accessory's orientation, and the second position of the user's body part includes at least one of the user's body part's position and the user's body part's orientation.

[0011] In embodiments of the present invention, preset conditions indicate at least one of the relative positional relationship and relative orientation relationship between the accessory and the user's body parts.

[0012] In an embodiment of the present invention, after obtaining the first position of the accessory in the environment and the second position of the user's body part in the environment, the method further includes: calculating the distance between the position of the accessory and the position of the user's body part.

[0013] In an embodiment of the present invention, after obtaining the first position of the accessory in the environment and the second position of the user's body part in the environment, the method further includes: calculating the relative angle between the orientation of the accessory and the orientation of the user's body part.

[0014] This invention provides a display system for accessory pairing, including a head-mounted display (HMD), at least one accessory, and a computing device. The computing device is coupled to the HMD and the accessory. The computing device is configured to determine the type of the accessory based on its identification information. The computing device is configured to identify the type of a user's body part based on an image. In response to the accessory being suitable for the user's body part based on the type of the accessory and the type of the user's body part, the computing device is configured to obtain a first position of the accessory in the environment and a second position of the user's body part in the environment. The computing device is configured to configure the accessory to pair with the user's body part.

[0015] In embodiments of the invention, the computing device is configured to obtain accessory type information from accessory identification information, wherein the accessory type information indicates the function of the accessory. The computing device is configured to determine the type of accessory based on the accessory type information.

[0016] In an embodiment of the invention, the computing device is configured to perform image recognition on an image to identify the type of user body part in the image.

[0017] In embodiments of the invention, the computing device is configured to define candidate regions in an image corresponding to user body parts. The computing device is also configured to define reference points based on these candidate regions to indicate the location of the user body parts within the environment.

[0018] In embodiments of the present invention, the computing device is configured to convert the coordinates of reference points in an image into the position of a user's body part within the environment.

[0019] In an embodiment of the present invention, the display system further includes an image extraction device coupled to a computing device, wherein the image extraction is configured to extract an image and transmit the image to the computing device.

[0020] In embodiments of the present invention, the first position of the accessory includes at least one of the accessory's position and the accessory's orientation, and the second position of the user's body part includes at least one of the user's body part's position and the user's body part's orientation.

[0021] In embodiments of the present invention, preset conditions indicate at least one of the relative positional relationship and relative orientation relationship between the accessory and the user's body parts.

[0022] In an embodiment of the present invention, after obtaining a first position of the accessory in the environment and a second position of the user's body part in the environment, the computing device is configured to calculate the distance between the position of the accessory and the position of the user's body part.

[0023] In an embodiment of the invention, after obtaining a first position of the accessory in the environment and a second position of the user's body part in the environment, the computing device is configured to calculate the relative angle between the orientation of the accessory and the orientation of the user's body part.

[0024] The present invention provides a non-transitory computer-readable recording medium configured to record a program, wherein the program is loaded by a processor into a computing device to perform the following steps: determining the type of an accessory based on the accessory's identification information; identifying the type of a user's body part based on an image; in response to the accessory being suitable for the user's body part according to the type of the accessory and the type of the user's body part, obtaining a first position of the accessory in the environment and obtaining a second position of the user's body part in the environment; and configuring the accessory to be paired with the user's body part according to the first position of the accessory and the second position of the user's body part.

[0025] To make the foregoing better understood, several embodiments with accompanying drawings are described in detail below. Attached Figure Description

[0026] The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0027] Figure 1 This is a block diagram of a display system for accessory pairing according to an embodiment of the present invention.

[0028] Figure 2 This is a flowchart of a method for accessory pairing according to an embodiment of the present invention.

[0029] Figure 3 An example of a user of a wearable head-mounted display and controller according to an embodiment of the present invention.

[0030] Figure 4 This is an example of detecting user body parts in an image according to an embodiment of the present invention.

[0031] To enable a better understanding of the above features and advantages of the present invention, several embodiments, accompanied by accompanying drawings, are described in detail below.

[0032] Explanation of icon numbers

[0033] 32: User;

[0034] 34, 120: HMD;

[0035] 36, 130: Image extraction device;

[0036] 38: Wrist controller;

[0037] 40: Image of a hand;

[0038] 41: Candidate region;

[0039] 50: Wrist controller;

[0040] 100: Display system;

[0041] 110: Computing device;

[0042] 140: Accessories;

[0043] 311, 314: Boundaries;

[0044] 320: Scene;

[0045] d1: Distance;

[0046] D1: First direction;

[0047] P1, P2: Reference points;

[0048] S202, S204, S206, S208, S210, S212: Steps. Detailed Implementation

[0049] Some embodiments of the invention will now be described more fully below with reference to the accompanying drawings, which illustrate some, but not all, of the embodiments of the invention. In fact, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the invention will satisfy applicable legal requirements.

[0050] Figure 1 This is a block diagram of a display system for accessory pairing according to an embodiment of the present invention. (Reference) Figure 1 The display system 100 of this embodiment includes a computing device 110, a head-mounted display (HMD) 120, an image extraction device 130, and accessories 140, the functions of which are described below.

[0051] The computing device 110 may be, for example, a server (e.g., a file server, database server, or application server), a workstation, a personal computer, or a computing-capable laptop computer and includes a processor. The processor may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), or any other general-purpose or special-purpose programmable microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), programmable logic device (PLD), or other similar device, or a combination of these devices.

[0052] HMD 120 is, for example, a liquid-crystal display (LCD), a light-emitting diode (LED) display, or other suitable type of display, manufactured in the form of a helmet or lenses for a user to wear on their head. In this embodiment, HMD 120 is coupled to computing device 110. In an embodiment, display system 100 may further include at least one locator (not shown) for positioning a user wearing HMD 120 within the environment and tracking the user's movement. The locator includes, for example, a laser or infrared (IR) emitter and receiver, used to detect the distance to an object in 3D space, and in cooperation with multiple calibrators disposed on the object (e.g., HMD 120) to locate the position of the object in 3D space. However, display system 100 may use other positioning technologies to locate HMD 120 and other objects in the environment.

[0053] It should be mentioned that, in one embodiment, the computing device 110 is disposed separately from the HMD 120 and is coupled to the HMD 120 in a wired or wireless manner for data transmission. In another embodiment, the computing device 110 may also be integrated into the HMD 120 and connected to the HMD 120 via a transmission line for data transmission, which is not limited to this invention.

[0054] Image extraction device 130 is coupled to computing device 110. Image extraction device 130 is, for example, a camera equipped with a charge-coupled device (CCD) lens, a complementary metal-oxide-semiconductor (CMOS) lens, an infrared lens, or other types of lenses. Alternatively, image extraction device 130 may be a camera with depth information capabilities, such as a depth camera or a stereo camera, which is not limited to this invention. It should be mentioned that, in embodiments, image extraction device 130 may be disposed separately from or integrated into HMD 120, which is not limited to this invention.

[0055] Display system 100 is compatible with several different accessories. Each accessory may have different functions and generate different input and output data in relation to display system 100. For example, accessory 140 may be an input device, such as a handheld or wrist controller equipped with a motion sensor and an infrared (IR) light source (e.g., an infrared LED), a mouse, a joystick, a navigation ball, a touchpad, and / or buttons that allow a user to interact with an environment, object, or avatar in a virtual world. In another embodiment, accessory 140 may be a locator capable of detecting objects in three-dimensional (3D) space and locating the object's position in 3D space using a calibrator placed on the object (e.g., HMD 120). Detailed steps of the accessory pairing method for the present invention are described below.

[0056] Figure 2 A flowchart illustrating a method for accessory mating according to an embodiment of the present invention. (Reference) Figure 2 The method in this embodiment is applicable to Figure 1 The detailed steps of the method for matching accessories according to the present invention are described below with reference to the various components of the display system 100.

[0057] First, the computing device 110 receives identification information from the accessory 140. The computing device 110 determines the type of the accessory 140 based on the identification information (step S202). In an embodiment, the computing device 110 obtains accessory type identification from the identification information of each accessory. The accessory type identification information may indicate the function corresponding to the accessory, such that accessories with different functions will have different accessory type identifications. The computing device 110 may determine the type of the accessory based on the accessory type information. In an embodiment, the identification information may be transmitted from the accessory 140 to the computing device 110 in a standard packet format of the Bluetooth protocol or the Bluetooth Low Energy (BLE) protocol, which is not limited to this invention.

[0058] return Figure 2 In the flowchart above, computing device 110 receives an image. Computing device 110 identifies the type of user's body part based on the image (step S204). Image extraction device 130 can extract an image of the user's surrounding environment. In one embodiment, because image extraction device 130 can be integrated into the HMD 120 worn by the user, image extraction device 130 extracts an image of the user's field of view (FOV). In another embodiment, if image extraction device 130 is separately positioned from HMD 120, then image extraction device 130 can extract an image of the user. The extracted image can be transferred from image extraction device 130 to computing device 110.

[0059] The user's body part can be the user's hands, legs, head, or other body parts. In an embodiment, the computing device 110 can detect and / or track user body part images corresponding to user body parts in received images. Specifically, the computing device 110 reads consecutive images in an image stream and detects and tracks candidate regions corresponding to user body parts in the images. The computing device 110 can perform image recognition on the images to define candidate regions corresponding to user body parts in the images and identify the type of user body part in the images. Image recognition can be performed by a trained deep learning model, neural network model, or similar artificial intelligence model. For example, the deep learning model is implemented by a learning network (e.g., a convolutional neural network (CNN)), which is not limited to this invention.

[0060] For example, Figure 3 An example of a user of a wearable head-mounted display and controller according to an embodiment of the present invention is shown. Figure 4 An example of detecting a user's body part in an image according to an embodiment of the present invention is shown. (Reference) Figure 3 In a real-world environment, a user 32 wears an HMD 34 on their head and a wrist controller 38 on their right hand. The user 32 wears an HMD 34 with an image extraction device 36 having a field of view extending from boundary 311 to boundary 314. In this embodiment, when a connection is established between the wrist controller 38 of the display system and the computing device 110, the computing device 110 can collect identification information from the wrist controller 38 and determine the type of the wrist controller 38 based on the identification information. In this case, the computing device 110 can determine the type corresponding to the wrist controller 38 as "wrist controller" based on the identification information of the wrist controller 38.

[0061] In this embodiment, scene 320 includes a wrist controller 38 and the right hand of user 32, and image extraction device 36 can extract an image corresponding to scene 320. Figure 4 The image VR is illustrated as an example for detailed description. Image VR is an image that can be extracted by image extraction device 36 and can be displayed to user 32 as a virtual reality image. In this embodiment, image VR includes a hand image 40 and a wrist controller image 50. After detecting the hand image 40 in image VR, computing device 110 can define candidate regions corresponding to hand image 40 and, based on the candidate regions, define reference points indicating the position of user body parts, such as the right hand of user 32. The reference point can be the center point of the candidate region, the center point of the upper boundary of the candidate region, or any point within the candidate region, and is not limited to this invention. Figure 4As illustrated, the computing device 110 can define a bounding box as a candidate region, and the size of the candidate region is at least large enough to enclose the user's body part corresponding to the candidate region. The computing device 110 can set the candidate region 41 to be associated with the hand image 40, and set the center point of the candidate region 41 as a reference point P1 indicating the position of the user 32's right hand.

[0062] Additionally, the computing device 110 identifies the type of the hand image 40 and associates the type with the candidate region 41. For example, in this embodiment, the computing device 110 can identify the hand image 40 corresponding to the candidate region 41 and obtain the type "right hand" corresponding to the hand image 40.

[0063] return Figure 2 In the flowchart, the computing device 110 determines whether an accessory is a candidate device for matching a user's body part in an image based on the type of the accessory and the type of the user's body part. Specifically, the computing device 110 determines whether the accessory is suitable for the user's body part based on the type of the accessory and the type of the user's body part (step S206). When the accessory is not suitable for the user's body part, the program returns to step S204, allowing the computing device 110 to continue receiving images from the image extraction device 130. More specifically, multiple types of accessories and the corresponding types of user body parts are stored in a memory (not shown) in the display system 100. The computing device 110 can determine whether an accessory is a candidate device based on the correspondence between the types of accessories and the types of user body parts.

[0064] For example, the computing device 110 can create a lookup table, in which multiple types of accessories and corresponding types of user body parts are recorded. The lookup table can be stored in the memory of the display system 100. Table 1 below is an example of a lookup table in an exemplary embodiment, used to record the correspondence between accessory types and user body part types.

[0065] Table 1

[0066]

[0067] On the other hand, when an accessory is suitable for a user's body part, the accessory is determined to be a candidate device. Specifically, in response to the accessory being suitable for a user's body part based on its type and the type of user's body part, the computing device 110 obtains a first position of the accessory 140 in the environment and a second position of the user's body part in the environment (step S208). For example, according to Figure 3 , Figure 4In Table 1, the computing device 110 can input the type "wrist controller" corresponding to the wrist controller 38 and the type "right hand" corresponding to the hand image 40 into Table 1, and determine whether the wrist controller 38 is a candidate device matching the hand image 40. In Table 1, the type "wrist controller" is recorded as being associated with the type "right hand". Therefore, the computing device 110 will determine that the wrist controller 38 and the hand image 40 have a corresponding relationship, and set the wrist controller 38 as a candidate device.

[0068] In one embodiment, the first position of accessory 140 may include the position of accessory 140 within the environment. For example, the position of accessory 140 within the environment may be obtained by computing device 110 based on the spatial relationship between image extraction device 130 and accessory 140. Image extraction device 130 may detect infrared light emitted from accessory 140, and computing device 110 may calculate the position of accessory 140 within the environment based on the infrared light obtained by image extraction device 130, the position of image extraction device 130, external parameters of image extraction device 130, internal parameters of image extraction device 130, and / or other information. The position of accessory may be a relative position associated with a specific device (e.g., image extraction device 130 or HMD 120) or an absolute position corresponding to a coordinate system in 3D space.

[0069] In another embodiment, the first position of accessory 140 may further include the orientation of the accessory. For example, the orientation of accessory 140 in the environment can be obtained using an accelerometer. The position and orientation of accessory 140 in the environment can be further obtained based on information received by one or more additional sensor components (not shown), such as other position sensors (e.g., motion sensors, GPS sensors, etc.) and / or orientation sensors (e.g., gyroscopes, etc.) that can track the position and / or orientation of accessory 140.

[0070] In one embodiment, the second location of the user's body part may include the location of the user's body part within the environment and / or the orientation of the user's body part. For example, the location of the user's body part within the environment may be obtained by the computing device 110 based on the spatial relationship between the HMD 120 and the user's body part. As mentioned above, the computing device 110 may perform image recognition on the image extracted by the image extraction device 130 to define reference points for the user's body part in the image. In this embodiment, the computing device 110 may further convert the coordinates of the reference points in the image into locations within the environment.

[0071] Additionally, the computing device 110 can perform image recognition on the image extracted by the image extraction device 130 to determine the orientation of the user's body parts. It should be mentioned that in embodiments, position and orientation can be a relative position associated with a particular device (e.g., HMD 120) or an absolute position corresponding to a coordinate system in 3D space.

[0072] In another embodiment, a second location of the user's body part within the environment can be obtained based on skeleton information or other information received by one or more additional sensor components, which is not limited to this invention. Specifically, the computing device 110 can perform skeleton detection or other user body part recognition methods on the received image to detect the user's skeleton information and define candidate regions of the user's body part in the image based on the skeleton information. Subsequently, the computing device 110 can further convert the coordinates of the candidate regions in the image into the location of the user's body part within the environment. Furthermore, the computing device 110 can calculate the orientation of the user's body part based on the skeleton information and inverse kinematics (IK).

[0073] Finally, the computing device 110 determines whether the relative positional relationship between the first position and the second position meets a preset condition (step S210). In response to the determination that the relative positional relationship does not meet the preset condition, the program returns to step S204, so that the computing device 110 continues to receive images from the image extraction device 130.

[0074] On the other hand, in response to determining a relative positional relationship that meets preset conditions, the computing device 110 is configured to pair the accessory 140 with the user's body part based on the first position of the accessory and the second position of the user's body part (step S212). Here, the preset conditions may indicate, for example, the relative positional relationship and / or relative orientation relationship between the accessory 140 and the user's body part. However, the conditions that the relative positional relationship meets are not limited to this. In detail, the computing device 110 may calculate the distance between the position of the accessory 140 and the position of the user's body part to determine the relative positional relationship, and determine whether the relative positional relationship meets the preset conditions. However, the preset conditions may be the distance or a range of distances between the accessory 140 and the user's body part.

[0075] In this embodiment, it is assumed that the preset conditions include "the wrist controller is located 10 cm behind the right hand" and / or "the orientation of the wrist controller is the same as the orientation of the right hand." (Reference) Figure 3 and Figure 4Reference point P1 represents the position of the hand image 40 corresponding to the right hand of user 32, and reference point P2 represents the position of wrist controller 38. The computing device 110 calculates the distance between reference points P1 and P2 to determine their relative positional relationship, and determines whether the relative positional relationship meets preset conditions. Figure 3 and Figure 4 As shown, the distance between reference point P1 and reference point P2 is distance d1. Assume distance d1 is 10 centimeters. Based on the relative positional relationship in this embodiment, it can be determined that, meeting preset conditions, the computing device 110 can pair the wrist controller 38 with the right hand of the user 32.

[0076] In another embodiment, relative orientation is further considered. The computing device 110 can calculate the relative angle between the orientation of the accessory and the orientation of the user's body part. Specifically, the computing device 110 calculates the relative angle between the orientation of the accessory and the orientation of the user to determine the relative orientation between the accessory and the user's body part, and determines whether the relative orientation meets preset conditions. However, the preset conditions may be the relative angle or a range of relative angles between the accessory 140 and the user's body part.

[0077] like Figure 3 and Figure 4 As shown, the computing device 110 can calculate the relative angle to determine the relative position between the hand image 40 corresponding to the right hand of the user 32 and the wrist controller 38, and determine whether the right hand of the user 32 and the wrist controller 38 have the same orientation. Assuming that the two orientations of the wrist controller 38 and the right hand of the user 32 are in the first direction D1, the computing device 110 can pair the wrist controller 38 with the right hand of the user 32.

[0078] The present invention further relates to a non-transitory computer-readable recording medium configured to record a program loaded by a processor into a computing device 110 to execute each step of a method for accessory pairing. The computer program comprises multiple program sections (e.g., a program section for constructing an organization diagram, a program section for approving a list, a program section for setting up, and a program section for deploying, etc.). After the program sections are loaded into the computing device 110 and executed, the steps of the method for accessory pairing can be completed.

[0079] In light of the foregoing description, the proposed method, system, and computer-readable recording medium for accessory pairing allow for pairing of accessories and user body parts based on the type and location of the accessories and user body parts. Based on the results of image tracking or skeletal tracking corresponding to the location where the accessory is worn and the accessory's positioning information, it is inferred on which body part the accessory will be worn. The accessory can then be paired with a body part and worn on that body part. Therefore, it eliminates the need for the user to specify a paired body part associated with each accessory, thereby improving the convenience and flexibility of accessory setup.

[0080] It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments without departing from the scope or spirit of the invention. In view of the foregoing, it is intended that the invention cover modifications and variations, provided that such modifications and variations fall within the scope of the claims and their equivalents.

[0081] Cross-references to related applications

[0082] This application claims priority to U.S. Provisional Application Serial No. 63 / 274,963, filed November 3, 2021. The entire contents of the aforementioned patent applications are hereby incorporated by reference and form part of this specification.

Claims

1. A method for matching accessories, comprising: The type of accessory is determined based on its identification information; Identify the type of user's body part based on the image; In response to determining that the accessory is suitable for the user's body part based on the type of the accessory and the type of the user's body part, a first position of the accessory in the environment and a second position of the user's body part in the environment are obtained; The accessory is configured to pair with the user's body part based on the first position of the accessory and the second position of the user's body part; Define candidate regions in the image corresponding to the user's body parts; A first reference point is defined based on the candidate region to indicate the location of the user's body part within the environment; Convert the coordinates of the first reference point in the image into the position of the user's body part within the environment; A second reference point defining the location of the accessory within the environment; and Pairing is performed based on a comparison of the distance between the first reference point and the second reference point with a preset distance.

2. The accessory pairing method according to claim 1, wherein the step of determining the type of the accessory based on the identification information includes: Accessory type information is obtained from the identification information of the accessory, wherein the accessory type information indicates the function of the accessory; as well as The type of accessory is determined based on the accessory type information.

3. The accessory matching method according to claim 1, wherein the step of identifying the type of the user's body part in the image includes: Image recognition is performed on the image to identify the type of the user's body part in the image.

4. The accessory pairing method according to claim 1, wherein the first position of the accessory includes at least one of the position of the accessory and the orientation of the accessory, and the second position of the user body part includes at least one of the position of the user body part and the orientation of the user body part.

5. The accessory pairing method according to claim 4, further comprising, after the steps of obtaining the first position of the accessory in the environment and obtaining the second position of the user's body part in the environment: Calculate the distance between the location of the accessory and the location of the user's body part.

6. The accessory pairing method according to claim 4, further comprising, after the steps of obtaining the first position of the accessory in the environment and obtaining the second position of the user's body part in the environment: Calculate the relative angle between the orientation of the accessory and the orientation of the user's body part.

7. A display system for accessory pairing, comprising: Head-mounted display (HMD); At least one accessory; as well as Computing device, coupled to the head-mounted display and the accessory, The computing device is configured to determine the type of the accessory based on its identification information. The computing device is configured to identify the type of body part of the user based on the image. In response to determining that the accessory is suitable for the user's body part based on the type of the accessory and the type of the user's body part, the computing device is configured to obtain a first position of the accessory in the environment and a second position of the user's body part in the environment. The computing device is configured to configure the accessory to pair with the user's body part based on the first position of the accessory and the second position of the user's body part. Define candidate regions in the image corresponding to the user's body parts. A first reference point is defined based on the candidate region to indicate the location of the user's body part within the environment. The coordinates of the first reference point in the image are converted into the position of the user's body part within the environment. Define a second reference point for the location of the accessory within the environment, and Pairing is performed based on a comparison of the distance between the first reference point and the second reference point with a preset distance.

8. A non-transitory computer-readable recording medium configured to record a program, wherein the program is loaded into a computing device by a processor to perform the following steps: The type of accessory is determined based on its identification information; Identify the type of user's body part based on the image; In response to determining that the accessory is suitable for the user's body part based on the type of the accessory and the type of the user's body part, a first position of the accessory in the environment and a second position of the user's body part in the environment are obtained; The accessory is configured to pair with the user's body part based on the first position of the accessory and the second position of the user's body part; Define candidate regions in the image corresponding to the user's body parts; A first reference point is defined based on the candidate region to indicate the location of the user's body part within the environment; Convert the coordinates of the first reference point in the image into the position of the user's body part within the environment; A second reference point defining the location of the accessory within the environment; and Pairing is performed based on a comparison of the distance between the first reference point and the second reference point with a preset distance.

Citation Information

Patent Citations

  • Image Capture to Provide Advanced Features for Configuration of a Wearable Device

    US20190369569A1