Device position correction method and device position correction system

Abstract

A device position correction method suitable for a wearable device includes: capturing a wearable device image by a camera in a head-mounted display (HMD); and accessing, by a processor in the HMD, an object detector and a limb detector stored in a storage device, executing the object detector and the limb detector to generate a correction information about the position of the wearable device.

Description

Technical Field

[0001] This invention relates to a calibration method, and more particularly to a device position calibration method and device position calibration system suitable for wearable devices. Background Technology

[0002] Generally speaking, in virtual reality, augmented reality, or mixed reality (MR), users need to wear a head-mounted display and wearable devices to perform corresponding actions in response to prompts or objects in the virtual reality scene.

[0003] Wearable devices, such as bracelets and rings, can be worn on the user's wrist, ankle, fingers, and other body parts. Head-mounted displays can be configured to be worn on specific body parts, such as a left wristband on the user's left hand and a right wristband on the user's right hand.

[0004] However, if the user wears the device in the wrong position, it can cause the head-mounted display to misrecognize the device or make operational errors in virtual reality. For example, if the user mistakenly wears the left wristband on the right hand and the right wristband on the left hand, the user will not receive the correct feedback when experiencing virtual reality scenes.

[0005] Therefore, determining the current wearing position of a wearable device and guiding the user to wear the device in the correct position has become one of the problems to be solved in this field. Summary of the Invention

[0006] This invention provides a device position correction method applicable to a wearable device. The device position correction method includes: receiving an image of a wearable device; detecting the position of a wearable device in the image using an object detector; detecting multiple skeletal points and a limb position in the image using a limb detection model to determine a body part corresponding to the limb; and generating correction information about the position of the wearable device based on the multiple skeletal points and the limb position.

[0007] In one embodiment, a processor in a head-mounted display device is used to access the object detection model and the limb detection model stored in a storage device, so as to execute the object detection model and the limb detection model.

[0008] In one embodiment, the processor calculates an interval distance based on the size of a limb or the base of a limb. When the distance between the wearable device and the base of the limb is equal to the interval distance, and the body part is within the preset range, it indicates that the wearable device is worn correctly.

[0009] In one embodiment, when the distance between the wearable device and the root of the limb is less than the interval distance, the processor issues a downward correction message for the wearable device via a prompting audio or a prompting video.

[0010] In one embodiment, when the distance between the wearable device and the base of the limb is greater than the interval distance, the processor sends an upward correction message for the wearable device via a prompting audio or a prompting video.

[0011] In one embodiment, the limb detection model determines that the body part corresponding to the limb is a left limb, and then the processor determines whether the position of a left limb wearable device is located on the left limb; wherein, when the processor determines that the position of the left limb wearable device is located on the left limb and the left limb is within the preset range, a correct wearing information is issued; wherein, when the processor determines that the position of the left limb wearable device is not located on the left limb, a wearing error information is issued.

[0012] In one embodiment, when the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first identification code corresponding to the left limb wearable device with a second identification code of the right limb wearable device in a replacement procedure.

[0013] In one embodiment, when the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first feature code corresponding to the left limb wearable device with a second feature code of a right limb wearable device in a replacement program; wherein the first feature code records a right limb feature and the second feature code records a left limb feature.

[0014] In one embodiment, when the processor determines that the left limb wearable device is located on a right limb, the processor issues a left / right device wearing error prompt audio message or a left / right device wearing error prompt video.

[0015] In one embodiment, when the processor determines that the left limb wearable device is located on a left limb, but the left limb wearable device is inverted, the processor issues a device inverted prompt audio message or a device inverted prompt video.

[0016] In one embodiment, after the limb detection model detects the plurality of skeletal points of the limb in the image of the wearable device, the processor determines whether the distance between the position of the wearable device and a specific skeletal point is less than a specific distance; wherein, when the processor determines that the distance between the position of the wearable device and the specific skeletal point is not less than the specific distance, the processor issues a wearable error prompt audio message or a wearable error prompt video.

[0017] This invention provides a device position correction system applicable to a wearable device. The device position correction system includes a head-mounted display (HMD). The HMD includes a camera, a storage device, and a processor. The camera captures an image of the wearable device. The processor accesses an object detector model and a limb detection model stored in the storage device, and executes the object detector model and the limb detection model, wherein the processor performs the following: detecting the position of the wearable device in the image using the object detector model; detecting multiple skeletal points and a limb position in the image using the limb detection model to determine a body part corresponding to the limb; and generating correction information about the position of the wearable device based on the multiple skeletal points and the limb position.

[0018] In one embodiment, the processor calculates an interval distance based on the size of a limb or the base of a limb. When the distance between the wearable device and the base of the limb is equal to the interval distance, and the body part is within the preset range, it indicates that the wearable device is worn correctly.

[0019] In one embodiment, when the distance between the wearable device and the root of the limb is less than the interval distance, the processor issues a downward correction message for the wearable device via a prompting audio or a prompting video.

[0020] In one embodiment, when the distance between the wearable device and the base of the limb is greater than the interval distance, the processor sends an upward correction message for the wearable device via a prompting audio or a prompting video.

[0021] In one embodiment, the limb detection model determines that the body part corresponding to the limb is a left limb, and then the processor determines whether the position of a left limb wearable device is located on the left limb; wherein, when the processor determines that the position of the left limb wearable device is located on the left limb and the left limb is within the preset range, a correct wearing information is issued; wherein, when the processor determines that the position of the left limb wearable device is not located on the left limb, a wearing error information is issued.

[0022] In one embodiment, when the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first identification code corresponding to the left limb wearable device with a second identification code of the right limb wearable device in a replacement procedure.

[0023] In one embodiment, when the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first feature code corresponding to the left limb wearable device with a second feature code of a right limb wearable device in a replacement program; wherein the first feature code records a right limb feature and the second feature code records a left limb feature.

[0024] In one embodiment, when the processor determines that the left limb wearable device is located on a right limb, the processor issues a left / right device wearing error prompt audio message or a left / right device wearing error prompt video.

[0025] In one embodiment, when the processor determines that the left limb wearable device is located on a left limb, but the left limb wearable device is inverted, the processor issues a device inverted prompt audio message or a device inverted prompt video.

[0026] In one embodiment, after the limb detection model detects the plurality of skeletal points of the limb in the image of the wearable device, the processor determines whether the distance between the position of the wearable device and a specific skeletal point is less than a specific distance; wherein, when the processor determines that the distance between the position of the wearable device and the specific skeletal point is not less than the specific distance, the processor issues a wearable error prompt audio message or a wearable error prompt video.

[0027] In summary, the embodiments of the present invention provide a device position correction method and a device position correction system. By using an object detection model and a limb detection model, correction information about the position of the wearable device can be generated based on multiple skeletal points and limb positions. In this way, the processor can determine whether the current wearing position of the wearable device is correct based on the correction information. When the processor determines that the current wearing position of the wearable device needs to be adjusted, it guides the user to wear the wearable device in the correct position based on the correction information. This enables the device position correction method and device position correction system to achieve more precise control and provide a better user experience when applied to virtual reality, augmented reality and / or mixed reality systems. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of a device position correction system according to an embodiment of the present invention.

[0029] Figure 2This is a flowchart illustrating a device position correction method according to an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram illustrating a device position correction method according to an embodiment of the present invention.

[0031] Figures 4A-4E This is a schematic diagram illustrating a device position correction method according to an embodiment of the present invention.

[0032] Figures 5A-5E This is a schematic diagram illustrating a device position correction method according to an embodiment of the present invention.

[0033] Symbol explanation: 100: Device Position Correction System CAM: Camera 10: Processor 20: Storage device HMD: Head-mounted display device 30: Wearable devices 200: Device Position Correction Method 210~240: Steps RL: Preset range for the left hand RP: Left hand position IMG: Images from Wearable Devices GL: Upcalibrate wearable device information DL: Interval distance L30: Left wristband R30: Right wristband 310~330: Screen RR: Preset range RP: Limb Position HR: Right hand HL: Left hand GR, DR: Indicators SR31~SR33, SR41~SR43, SL31~SL33, SL41~SL43: Skeleton points Detailed Implementation

[0034] The following description is a preferred embodiment of the invention and is intended to describe the basic spirit of the invention, but is not intended to limit the invention. The actual scope of the invention must be understood by referring to the claims that follow.

[0035] It must be understood that the terms "comprising" and "including" used in this specification are used to indicate the presence of specific technical features, values, method steps, work processes, elements and / or components, but do not preclude the addition of more technical features, values, method steps, work processes, elements, components, or any combination thereof.

[0036] The use of terms such as "first," "second," and "third" in the claims is to modify elements in the claims and is not intended to indicate a priority order, a prior relationship, or that one element precedes another, or the chronological order of the execution of method steps. It is only used to distinguish elements with the same name.

[0037] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating a device position correction system 100 according to an embodiment of the present invention. In one embodiment, the device position correction system 100 includes a head-mounted display (HMD). In one embodiment, the HMD includes a camera (CAM), a processor 10, and a storage device 20.

[0038] In one embodiment, the head-mounted display device (HMD) includes multiple cameras (CAMs). In another embodiment, the multiple cameras (CAMs) in the HMD are positioned at different locations within the HMD and can be used to capture environmental images.

[0039] In one embodiment, the storage device 20 in the head-mounted display device (HMD) can be a read-only memory, flash memory, floppy disk, hard disk, optical disk, USB flash drive, magnetic tape, a database accessible by a network, or a storage medium that can be easily conceived and has the same function by those skilled in the art.

[0040] In one embodiment, the processor 10 in the head-mounted display device (HMD) may be implemented by a microcontroller, a microprocessor, a digital signal processor, an application-specific integrated circuit (ASIC), or a logic circuit.

[0041] In one embodiment, the camera (CAM) may consist of at least one charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) sensor.

[0042] In one embodiment, a processor 10 in a head-mounted display device (HMD) is used to access an object detector model and a limb detection model stored in a storage device 20, and to execute the object detector model and the limb detection model.

[0043] In one embodiment, the object detection model may be an execution thread, which is accessed and executed by the processor 10 in the head-mounted display device 10 from the storage device 20.

[0044] In one embodiment, the limb detection model may be an execution thread, accessed and executed by the processor 10 in the head-mounted display device 10 from the storage device 20.

[0045] In one embodiment, the object detection model may be implemented by a region-based convolutional neural network (R-CNN) algorithm or other deep learning methods capable of selecting specific objects.

[0046] In one embodiment, the limb detection model can be implemented by a region-based convolutional neural network algorithm or other deep learning methods capable of outputting skeleton points and limb positions.

[0047] In one embodiment, the object detection model is trained on images of multiple pre-set objects, such as wearable devices 30, controllers, motion trackers, etc., before the head-mounted display device (HMD) leaves the factory. Training is considered complete when the training results converge, and the trained object detection model is stored in the storage device 20 of the HMD. However, this embodiment is not limited to this; the object detection model can also be updated or trained after leaving the factory according to user needs.

[0048] In one embodiment, the trained object detection model can also be burned into a chip to become a piece of hardware in a head-mounted display device (HMD).

[0049] In one embodiment, the object detection model can also be implemented by a hardware circuit.

[0050] In one embodiment, after the head-mounted display device (HMD) leaves the factory, the user can view the detected object positions on the display of the head-mounted display device (HMD). For example, the head-mounted display device (HMD) detects the position of the wearable wristband 30 through an object detection model.

[0051] In one embodiment, the device position correction system 100 includes a wearable device 30 (e.g., a wearable wristband 30). In one embodiment, the head-mounted display (HMD) and the wearable wristband 30 can establish a communication connection via wired or wireless means.

[0052] In one embodiment, the limb detection model is trained before the head-mounted display device (HMD) leaves the factory using images and / or skeletal feature points of multiple pre-set human body parts, such as the left palm, right palm, left wrist, right wrist, left ankle, right ankle, waist, left arm, right arm, left thigh, right thigh, each finger of the left hand, each finger of the right hand, etc. Training is considered complete when the results converge, and the trained limb detection model is stored in the storage device 20 of the HMD. However, this embodiment is not limited to this; the limb detection model can also be updated or trained after leaving the factory according to user needs.

[0053] In one embodiment, the trained limb detection model can identify human skeleton points and human body parts in an image.

[0054] In one embodiment, the trained limb detection model can also be burned into a chip to become a piece of hardware in a head-mounted display device (HMD).

[0055] In one embodiment, the limb detection model can also be implemented by a hardware circuit.

[0056] In one embodiment, after the head-mounted display device (HMD) leaves the factory, the user can view the detected human body on the display of the HMD. For example, the HMD detects the position of the left palm through a limb detection model.

[0057] Please refer to the above as well. Figures 1-3 , Figure 2 This is a flowchart illustrating a device position correction method 200 according to an embodiment of the present invention. The device position correction method 200 can be derived from... Figure 1 The device position correction system 100 shown is implemented. Figure 3 This is a schematic diagram of a device position correction method 200 according to an embodiment of the present invention.

[0058] In step 210, a camera (CAM) of a head-mounted display (HMD) captures an image (IMG) of the wearable device.

[0059] In step 220, the processor 10 detects the position of a wearable device in the wearable device image IMG using an object detection model.

[0060] like Figure 3 As shown in image 310, the object detection model detects the wearable wristband 30 in the image IMG of the wearable device.

[0061] In one embodiment, the wearable bracelet 30 is pre-set to be worn on the left wrist, and is therefore marked with the symbol L30. Conversely, when the wearable bracelet 30 is pre-set to be worn on the right wrist, it is marked with the symbol R30. Thus, the object detection model can also obtain the pre-set wearing position (e.g., left or right wrist) of the wearable bracelet 30 by capturing images of it.

[0062] In one embodiment, the processor 10 can also use any conventional image recognition algorithm to identify the symbols on the wearable bracelet 30 in order to obtain the preset wearing position of the wearable bracelet 30 (e.g., left hand or right hand).

[0063] In step 230, the limb detection model detects multiple skeletal points (e.g., skeletal points SL31~SL34 of the left middle finger) and a limb position (e.g., left hand position RP) of a limb in the wearable device image IMG to determine the body part corresponding to the limb.

[0064] In one embodiment, such as Figure 3 As shown in screen 320, the limb position RP can optionally include the position of the wearable wristband 30.

[0065] In one embodiment, the limb detection model detects a limb in the screen 320 and detects the relative position of the limb in the screen 320, such as the limb position RP, and detects the skeletal points SL31~SL34 of the middle finger of the limb.

[0066] In one embodiment, the limb detection model can directly identify the human body part corresponding to the limb in the output image of the screen 320 as the left hand, or the processor 10 can also use traditional image recognition technology to compare the limb in the image with multiple human body parts to identify the human body part corresponding to the limb in the image as the left hand.

[0067] In one embodiment, the limb detection model can output at least 20 skeletal points of the left hand, and the human body part corresponding to the limb can also be determined to be the left hand based on the multiple skeletal points.

[0068] In step 240, the processor 10 generates correction information about the position of the wearable device based on these skeletal points (e.g., skeletal points SL31~SL34 of the left middle finger) and limb positions (e.g., left hand position RP).

[0069] In one embodiment, the correction information may be a preset range and / or correction information, such as a preset range RL for the left hand, upward correction of wearable device information GL, etc.

[0070] In one embodiment, the processor 10 determines whether the current wearing position of the wearable device is correct based on the calibration information. When the processor 10 determines that the current wearing position of the wearable device needs to be adjusted, it guides the user to wear the wearable device in the correct position based on the calibration information.

[0071] In this way, the processor 10 can apply calibration information and display the preset range and / or calibration information on the display, so that the user knows the relationship between the body parts and the position of the wearable device (such as the wearable bracelet 30), and can adjust the position of the wearable device according to the preset range and / or calibration information.

[0072] In one embodiment, such as Figure 3 As shown in screen 330, when the processor 10 determines that the position of the wearable bracelet 30 is too far from the root of the left hand, it will display correction information on the display, such as "correct wearable device information GL".

[0073] In one embodiment, such as Figure 3 As shown in screen 330, processor 10 calculates an interval distance based on the size of a limb (e.g., the size of a palm) or the base of a limb (e.g., the base of a finger) (in subsequent calculations). Figures 4A-4E and Figures 5A-5E (Indicated by DR and / or DL), when the distance between the wearable device and the base of the limb is equal to the interval distance, and the body part is within the preset range RL, it means that the wearable device is worn correctly.

[0074] In one embodiment, when the processor 10 determines that the limb is the left hand, it can determine from known information that the wearable bracelet 30 should be worn approximately two centimeters below the base of the left palm. In one embodiment, if the wearable bracelet 30 is exactly at the base of the left palm, it may affect the user's dexterity. In one embodiment, if the wearable bracelet 30 is too far from the base of the left palm, it may cause the head-mounted display (HMD) to be less accurate in judging the user's movements.

[0075] In one embodiment, the limb detection model can output the position of the base of the hand, for example, by marking it on the output image or by indicating it with a coordinate.

[0076] In one embodiment, the processor 10 can determine the position where the wearable bracelet 30 should be worn based on information output from the limb detection model, or read from the storage device 20 the position where the wearable bracelet 30 should be worn based on multiple palm sizes.

[0077] Please see Figures 4A-4E This is a schematic diagram of a device position correction method 200 according to an embodiment of the present invention.

[0078] In one embodiment, the device position correction method 200 can support multiple wearable devices (e.g., wearable left wristband L30, wearable right wristband R30).

[0079] In one embodiment, such as Figure 4A As shown, when the distance between the wearable device (e.g., wearable left wristband L30 and wearable right wristband R30) and the root of the limb (e.g., the root of the palm of the left hand HL and the root of the palm of the right hand HR) is less than the interval distance, the processor 10 issues a downward correction information for the wearable device (e.g., indicators GL and GR) through a prompting tone or a prompting image.

[0080] In addition, in this example, the processor 10 will also detect that the left hand HL is not within the preset range RL and the right hand RL is not within the preset range RR. Therefore, the processor 10 will determine that the user has not yet put on the wearable device.

[0081] In one embodiment, such as Figure 4B As shown, when the distance between the wearable device (e.g., wearable left wristband L30 and wearable right wristband R30) and the base of the limb (e.g., the base of the palm of the left hand HL and the base of the palm of the right hand HR) is greater than the interval distance (e.g., interval distance DL and interval distance DR), the processor 10 issues an upward correction information for the wearable device (e.g., indicator GL and indicator GR) through a prompting tone or a prompting image.

[0082] In one embodiment, such as Figure 4C As shown, if the limb detection model determines that the body part corresponding to the limb is a left limb (such as the left hand HL), then the processor 10 determines whether a left limb wearable device (such as a wearable left hand bracelet L30) is located on the left limb. In one embodiment, when the processor 10 determines that the left limb wearable device is located on the left limb and the left limb is within a preset range RL, then a correct wearing information is issued.

[0083] In one embodiment, such as Figure 4C As shown, if the limb detection model determines that the body part corresponding to the limb is a right limb (e.g., the right hand HR), then the processor 10 determines whether a right limb wearable device (e.g., a wearable right hand bracelet R30) is located on the right limb. In one embodiment, when the processor 10 determines that the right limb wearable device is located on the right limb and the right limb is within a preset range RL, then it issues a correct wearing information.

[0084] In one embodiment, such as Figure 4DAs shown, when the processor 10 determines that the wearable device for the left limb (e.g., a wearable left wristband L30) is not located on the left limb (e.g., the left hand HL), it issues a wear error message. For example, if the wearable left wristband L30 is located on the right hand HR, the processor 10 issues a wear error message.

[0085] In one embodiment, such as Figure 4D As shown, when the processor 10 determines that the wearable device for the right limb (e.g., the wearable right wristband R30) is not located on the right limb (e.g., the right hand HR), it issues a wear error message. For example, if the wearable right wristband R30 is located on the left hand HL, the processor 10 issues a wear error message.

[0086] In one embodiment, such as Figure 4D As shown, when the processor 10 determines that the position of the left limb wearable device (e.g., wearable left wristband L30) is located on a right limb (e.g., right hand HR), the processor 10 swaps a first identification code corresponding to the left limb wearable device (e.g., wearable left wristband L30) with a second identification code of a right limb wearable device (e.g., wearable right wristband R30) in a replacement procedure.

[0087] For example, the processor 10 changes the first identification code from ID_R to ID_L and the second identification code from ID_R to ID_L. When the first identification code ID_L is called later, it will correspond to the left limb wearable device, and when the second identification code ID_R is called, it will correspond to the right limb wearable device.

[0088] By swapping the identification codes of wearable devices on the software side, users can avoid removing and swapping the wearable devices before putting them back on, and can continue to use the updated identification codes to identify each wearable device in the future.

[0089] In one embodiment, when the processor 10 determines that the position of a left-limb wearable device (e.g., a wearable left wristband L30) is located on a right limb (e.g., the right hand HR), the processor 10 swaps a first feature code corresponding to the left-limb wearable device (e.g., the wearable left wristband L30) with a second feature code of a right-limb wearable device (e.g., the wearable right wristband R30) in a replacement procedure. The first feature code records a right-limb feature, and the second feature code records a left-limb feature.

[0090] In one embodiment, limb features include information such as the location of each skeletal point, limb size, limb skin color, limb shape, limb position, etc.

[0091] In one embodiment, the original first feature code corresponds to the skeleton point features of the left hand and the position information of the left hand (e.g., position information in a preset range RL), and the original second feature code corresponds to the skeleton point features of the right hand and the position information of the right hand (e.g., position information in a preset range RR).

[0092] By swapping the feature codes of wearable devices in the software, users can identify each wearable device without having to remove the device, swap its position, and put it back on.

[0093] In one embodiment, when the processor 10 determines that the left limb wearable device (e.g., a wearable left wristband L30) is located on a right limb (e.g., the right hand HR), the processor 10 issues a left / right device wear error prompt audio message or a left / right device wear error prompt video.

[0094] In one embodiment, such as Figure 4E As shown, when the processor 10 determines that the left limb wearable device (e.g., wearable left wristband L30) is located on a left limb (e.g., left hand HL), but the position of the left limb wearable device is inverted, i.e. upside down, the processor 10 issues a device inverted prompt sound or a device inverted prompt image (e.g., indicators GL and GR).

[0095] Figure 3 and Figures 4A-4E The example used is the hand; however, this application is not limited to this, and the above method can be adapted to any body part.

[0096] Furthermore, the indicators mentioned in this invention are all used to prompt users to calibrate or adjust the position of the wearable device.

[0097] Please see Figures 5A-5E This is a schematic diagram of a device position correction method 200 according to an embodiment of the present invention. Figures 5A-5E The wearable device is a ring, such as a left ring L30 and a right ring L30. Generally speaking, the processor 10 pre-sets the correct wearing position of the left ring L30 and the right ring L30. The correct wearing position must be at a specific distance below the third skeletal point SL33 from the top of the left middle finger and at a specific distance below the third skeletal point SR33 from the top of the right middle finger.

[0098] In one embodiment, such as Figure 5AAs shown, the processor 10 determines that the left ring L30 and the right ring L30 are respectively worn on the skeletal point SL33 and the skeletal point SR33, and not at a specific distance below the third skeletal point SL33 from the top of the left middle finger and the third skeletal point SR33 from the top of the right middle finger. Therefore, the processor 10 issues a wearing error prompt audio message or a wearing error prompt image (for example, the indicator GR, which indicates that the user should move the right ring R30 down a distance DR, or the indicator GL, which indicates that the user should move the left ring L30 down a distance DL).

[0099] In one embodiment, after the limb detection model detects multiple skeletal points of a limb in the image of a wearable device, the processor 10 determines whether the distance between the wearable device position of a wearable device and a specific skeletal point is less than a specific distance; wherein, when the processor 10 determines that the distance between the wearable device position of the wearable device and the specific skeletal point is not less than the specific distance, the processor 10 issues a wearable error prompt audio message or a wearable error prompt image.

[0100] For example, Figure 5B As shown, the left ring L30 is positioned below the skeletal point SL43, but not below SL33. In this case, the processor 10 determines that the distance between the wearable device position of the left ring L30 and the specific skeletal point SL33 is not less than a specific distance (e.g., the distance between the skeletal point SL33 of the middle finger and the skeletal point SL43 of the ring finger). Then, the processor 10 issues a wear error prompt audio or a wear error prompt image (e.g., the indicator GL). Similarly, the right ring L30 is positioned below the skeletal point SR43, but not below SR33. In this case, the processor 10 determines that the distance between the wearable device position of the right ring R30 and the specific skeletal point SR33 is not less than a specific distance (e.g., the distance between the skeletal point SR33 of the middle finger and the skeletal point SR43 of the ring finger). Then, the processor 10 issues a wear error prompt audio or a wear error prompt image (e.g., the indicator GR).

[0101] In one embodiment, such as Figure 5C As shown, if the processor 10 determines that the position of the wearable device on the left limb (e.g., the left ring L30) is on the left limb (e.g., at a specific distance below the skeletal point SL33) and the left limb is within a preset range RL, then it issues a wear correct information message (indicating that the position correction of the wearable device on the left hand HL is complete). In one embodiment, if the processor 10 determines that the position of the wearable device on the right limb (e.g., the right ring R30) is on the right limb (e.g., at a specific distance below the skeletal point SR33) and the right limb is within a preset range RR, then it issues a wear correct information message (indicating that the position correction of the wearable device on the right hand HR is complete).

[0102] In one embodiment, such as Figure 5D As shown, the left ring L30 is worn on the right hand HR, and the right ring R30 is worn on the left hand HK. If the processor 10 determines that the two rings need to be swapped, the processor 10 will issue a wearing error prompt audio message or a wearing error prompt video to prompt the user to swap the positions of the left ring L30 and the right ring R30.

[0103] In one embodiment, when the processor 10 determines that the position of the left ring L30 is on the right HR, the processor 10 swaps a first identification code corresponding to the left ring L30 with a second identification code of the right ring R30 in a swapping procedure.

[0104] For example, the processor 10 changes the first identification code from ID_R to ID_L and the second identification code from ID_R to ID_L. When the first identification code ID_L is called later, it will correspond to the left limb wearable device (such as the left ring L30), and when the second identification code ID_R is called, it will correspond to the right limb wearable device (such as the right ring R30).

[0105] In one embodiment, when the processor 10 determines that the left ring L30 is located on the right hand HR, the processor 10 swaps a first feature code corresponding to a left limb wearable device (e.g., left ring L30) with a second feature code of a right limb wearable device (e.g., right ring R30) in a replacement procedure. The first feature code records a right limb feature, and the second feature code records a left limb feature.

[0106] By swapping the identification codes or feature codes of wearable devices on the software side, users can identify each wearable device without having to remove the device, swap its position, and then put it back on.

[0107] In one embodiment, such as Figure 5E As shown, when the processor 10 determines that the left ring L30 is located on the left hand HL, but the position of the left ring L30 is inverted, the processor 10 issues a device inversion prompt sound or a device inversion prompt image to prompt the user to adjust the direction of the left ring L30 by 180 degrees to correct the position of the left ring L30. In one embodiment, when the processor 10 determines that the right ring R30 is located on the right hand HR, but the position of the right ring R30 is inverted, the processor 10 issues a device inversion prompt sound or a device inversion prompt image to prompt the user to adjust the direction of the right ring R30 by 180 degrees to correct the position of the right ring R30.

[0108] In summary, the embodiments of the present invention provide a device position correction method and a device position correction system. By using an object detection model and a limb detection model, correction information about the position of the wearable device can be generated based on multiple skeletal points and limb positions. In this way, the processor can determine whether the current wearing position of the wearable device is correct based on the correction information. When the processor determines that the current wearing position of the wearable device needs to be adjusted, it guides the user to wear the wearable device in the correct position based on the correction information. This enables the device position correction method and device position correction system to achieve more precise control and provide a better user experience when applied to virtual reality, augmented reality and / or mixed reality systems.

[0109] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the scope of the invention. Any person skilled in the art can make some modifications without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A device position correction method, applicable to a wearable device, the device position correction method comprising: Receives images from a wearable device; The location of a wearable device in the image of the wearable device is detected by an object detection model. A limb detection model is used to detect multiple skeletal points and the position of a limb in the image of the wearable device in order to determine the body part corresponding to the limb. as well as Based on the multiple skeletal points and the position of the limb, a correction information regarding the position of the wearable device is generated; This calibration information guides the wearable device to be worn in the correct position; If the limb detection model determines that the body part corresponding to the limb is a left limb, then a processor determines whether the position of a left limb wearable device is located on the left limb. When the processor determines that the wearable device for the left limb is located on the left limb and the left limb is within a preset range, it issues a correct wearing message. When the processor determines that the left limb wearable device is located on a right limb, the processor issues a left / right device wearing error prompt audio message or a left / right device wearing error prompt video. Specifically, when the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first identification code corresponding to the left limb wearable device with a second identification code of the right limb wearable device in a replacement procedure; and / or When the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first feature code corresponding to the left limb wearable device with a second feature code of the right limb wearable device in a replacement program. The first feature code records a right limb feature, and the second feature code records a left limb feature.

2. The device position correction method as described in claim 1, further comprising: A processor in a head-mounted display device accesses the object detection model and the limb detection model stored in a storage device, and executes the object detection model and the limb detection model.

3. The device position correction method as described in claim 1, wherein, A processor calculates an interval distance based on the size of a limb or the base of a limb. When the distance between the wearable device and the base of the limb is equal to the interval distance, and the body part is within the preset range, it means that the wearable device is worn correctly.

4. The device position correction method as described in claim 3, wherein, When the distance between the wearable device and the root of the limb is less than the interval distance, the processor sends a downward correction message for the wearable device through a prompting sound or a prompting image.

5. The device position correction method as described in claim 3, wherein, When the distance between the wearable device and the base of the limb is greater than the interval distance, the processor sends an upward correction message for the wearable device through a prompting audio or a prompting image.

6. The device position correction method as described in claim 1, wherein, If the processor determines that the wearable device for the left limb is not located on the left limb, it will issue a wear error message.

7. The device position correction method as described in claim 6, wherein, When the processor determines that the wearable device for the left limb is located on the left limb, but the wearable device for the left limb is inverted, the processor will issue a device inverted warning sound or a device inverted warning image.

8. The device position correction method as described in claim 1, wherein, After the limb detection model detects the multiple skeletal points of the limb in the image of the wearable device, a processor determines whether the distance between the position of the wearable device and a specific skeletal point is less than a specific distance. When the processor determines that the distance between the wearable device and the specific skeletal point is not less than the specific distance, the processor issues a wearable error prompt audio message or a wearable error prompt video.

9. A device position correction system, applicable to a wearable device, the device position correction system comprising: A head-mounted display device, comprising: A camera used to capture images of a wearable device; A storage device; and A processor is configured to access an object detection model and a limb detection model stored in the storage device, and to execute the object detection model and the limb detection model, wherein the processor executes: The object detection model is used to detect the position of a wearable device in the image of the wearable device. The limb detection model detects multiple skeletal points and the position of a limb in the image of the wearable device to determine the corresponding body part; and Based on the multiple skeletal points and the position of the limb, a correction information regarding the position of the wearable device is generated; This calibration information guides the wearable device to be worn in the correct position; If the limb detection model determines that the body part corresponding to the limb is a left limb, then the processor determines whether the position of a left limb wearable device is located on the left limb. When the processor determines that the wearable device for the left limb is located on the left limb and the left limb is within a preset range, it issues a correct wearing message. When the processor determines that the left limb wearable device is located on a right limb, the processor issues a left / right device wearing error prompt audio message or a left / right device wearing error prompt video. Specifically, when the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first identification code corresponding to the left limb wearable device with a second identification code of the right limb wearable device in a replacement procedure; and / or When the processor determines that the left limb wearable device is located on a right limb, the processor swaps a first feature code corresponding to the left limb wearable device with a second feature code of the right limb wearable device in a replacement program. The first feature code records a right limb feature, and the second feature code records a left limb feature.

10. The device position correction system as described in claim 9, wherein, The processor calculates an interval distance based on the size of a limb or the base of a limb. When the distance between the wearable device and the base of the limb is equal to the interval distance, and the body part is within the preset range, it means that the wearable device is worn correctly.

11. The device position correction system as described in claim 10, wherein, When the distance between the wearable device and the root of the limb is less than the interval distance, the processor sends a downward correction message for the wearable device through a prompting sound or a prompting image.

12. The device position correction system as described in claim 10, wherein, When the distance between the wearable device and the base of the limb is greater than the interval distance, the processor sends an upward correction message for the wearable device through a prompting audio or a prompting image.

13. The device position correction system as described in claim 9, wherein, If the processor determines that the wearable device for the left limb is not located on the left limb, it will issue a wear error message.

14. The device position correction system as described in claim 13, wherein, When the processor determines that the wearable device for the left limb is located on the left limb, but the wearable device for the left limb is inverted, the processor will issue a device inverted warning sound or a device inverted warning image.

15. The device position correction system as described in claim 9, wherein, After the limb detection model detects the multiple skeletal points of the limb in the image of the wearable device, the processor determines whether the distance between the position of the wearable device and a specific skeletal point is less than a specific distance. When the processor determines that the distance between the wearable device and the specific skeletal point is not less than the specific distance, the processor issues a wearable error prompt audio message or a wearable error prompt video.

Images