A wearable projection system and projection method for motion capture

Abstract

The present invention relates to the technical field of projection image processing, and discloses a wearable projection system and method for motion capture. The wearable projection system comprises a hardware device cluster and a software node cluster. The hardware device cluster comprises a display module, a projection module and a gesture capture module. The display module comprises a carrier, and the carrier is provided with an inner lens and an outer lens spaced upward at the level of a user's eyes. A liquid storage cavity is provided between the inner lens and the outer lens. The software node cluster comprises a rendering node, a data processing node, a standby node, a load balancing node, a storage node, a cache node and a monitoring node. The present invention controls the shape of the liquid in the liquid storage cavity by changing the output voltage of the electrode, thereby adjusting the focal length of the transmission between the inner lens and the outer lens instead of acting on the image output level, thereby meeting the personalized viewing needs of different people, reducing the overall burden of the system, and eliminating the learning obstacles of users for complex technical means.

Description

Technical Field

[0001] The present invention relates to the technical field of projection image processing, and more specifically, to a wearable projection system and method for motion capture. Background Art

[0002] With the rapid development of virtual reality (VR) and augmented reality (AR) technologies, traditional display and input devices are gradually unable to meet more complex interaction needs. Existing projection devices are mostly desktop or fixed installations, which limits the user's ability to interact with the virtual world. Although motion capture technology can accurately record user movements, it still has high costs and complex installation problems. In order to break through these limitations, the market urgently needs a device that can integrate projection and motion capture technology and achieve a more natural and smooth interactive experience through wearable devices.

[0003] Generally, due to the wearable nature of the projection system that integrates projection and motion capture, the frequent movements of the user during use will cause the projection output device to follow the disturbance. In this way, when the projected image is distorted, the correction of the projection output device itself alone will cause the following disadvantages in the interactive projection system:

[0004] 1. Lack of personalization: The calibration of traditional projection output devices is generally fixed for the entire image. Different people will face different visual experiences when watching projection, especially at different viewing angles or distances. The calibration method with fixed settings for the image cannot meet the needs of adjusting the focal length according to personal circumstances.

[0005] 2. Heavy equipment burden: Traditional projection output devices need to use software or complex hardware mechanisms, such as geometric correction or dynamic adjustment, to correct image distortion, which increases the burden on the equipment and leads to loss of image quality.

[0006] 3. There are technical learning barriers: The calibration of traditional projection output devices requires relatively complex technical means to adjust. Users need to learn the adjustment operations during use, which is quite troublesome. Summary of the invention

[0007] The present invention provides a wearable projection system and method for motion capture, which solve the technical problems of lack of personalization, heavy equipment burden and technical learning barriers in the related art when correcting image distortion.

[0008] The present invention provides a wearable projection system for motion capture, comprising a hardware device cluster and a software node cluster;

[0009] The hardware device cluster includes a display module, a projection module and a gesture capture module. The projection module is fixedly mounted on the display module. The projection module outputs an image by interacting with the gesture capture module. The display module is arranged between the user's eyes and the image.

[0010] The display module comprises a carrier, an inner lens and an outer lens are arranged on the carrier at an interval upward from the user's eye level, a liquid storage cavity is arranged between the inner lens and the outer lens, a transparent colorless liquid is stored in the liquid storage cavity, a pair of electrodes are arranged in the carrier and are connected to the liquid storage cavity, and the shape of the liquid changes with the voltage applied to the electrodes;

[0011] The software node cluster includes rendering nodes, data processing nodes, standby nodes, load balancing nodes, storage nodes, cache nodes and monitoring nodes, and each of the nodes cooperates alternately to instruct the hardware device cluster to run.

[0012] As a further solution of the present invention: the gesture capture module is in the shape of a glove, and includes a plurality of sensors arranged at the finger joints.

[0013] As a further solution of the present invention: the rendering node is equipped with a GPU to generate 3D rendering content and perform real-time image processing.

[0014] As a further solution of the present invention: the data processing node receives and processes the data from the gesture capture module and provides real-time feedback.

[0015] As a further solution of the present invention: the standby node cooperates with the rendering node and the data processing node to take over the task and continue to run when the rendering node and the data processing node fail.

[0016] As a further solution of the present invention: the storage node stores static images, video materials and real-time interactive data.

[0017] As a further solution of the present invention: the cache node reduces data access delay when the system is running.

[0018] As a further solution of the present invention: the monitoring node collects and analyzes the resource usage of each node in real time.

[0019] A projection method, using the wearable projection system for motion capture, comprises the following steps:

[0020] S1. Initialization and system startup: When the system starts, the display module, projection module and gesture capture module in the hardware device cluster perform self-checks to ensure normal operation. The monitoring system begins to collect the usage of each node and establish a preliminary load benchmark.

[0021] S2. Image content output: The image content pre-stored in the system is projected onto an opaque background via a projection module, and the user wears the display module to view the image content;

[0022] S3, User Interaction: Users interact with the system through gesture changes. The sensor in the gesture capture module captures the user's gesture data and uploads it to the computing server. The data processing node analyzes the data and generates corresponding images or effects. Then, the rendering node renders and processes the data, regenerates the image content, and then outputs it through the projection module and feeds it back to the user, forming an interactive process.

[0023] S4, distortion compensation: when the output image of the projection module is distorted, the display module changes the focal length of the user's observation to adjust and adapt, thereby compensating for the distortion and blur of the image;

[0024] S5, Load Management Intervention: According to the load benchmark in S1, load management is achieved through resource monitoring, dynamic calculation of resource allocation, confirmation of task priority, establishment of system fault tolerance mechanism, and feedback and adjustment.

[0025] As a further solution of the present invention: the objects of resource monitoring in step S5 include CPU and GPU load monitoring, memory usage monitoring and network bandwidth.

[0026] The beneficial effects of the present invention are:

[0027] The present invention controls the shape of the liquid in the liquid storage chamber by changing the output voltage of the electrode, thereby adjusting the focal length of the transmission between the inner lens and the outer lens, rather than acting on the image output level. The adjustment operation is positioned on the user side, which can meet the personalized viewing needs of different people.

[0028] The present invention allows the entire projection system to focus on outputting the best image by adjusting the focus on the display module, and the audience can then make fine adjustments by themselves, which helps to reduce the working pressure of the projection device without considering integrating it into the projection output device with the help of software or complex hardware mechanisms.

[0029] The present invention adjusts the focal length by changing the voltage, thereby correcting image distortion. Users can adjust their visual experience more simply, especially in temporary occasions or when a problem needs to be solved quickly. It provides a convenient solution and users no longer need to learn complex technical means, eliminating usage obstacles.

[0030] The present invention introduces load management at the software level, and combines resource monitoring, task scheduling, load balancing, failover, and adaptive tuning mechanisms to enable the system to effectively manage hardware resources and provide an efficient and stable user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the hardware device cluster structure of a wearable projection system for motion capture proposed by the present invention;

[0032] Figure 2 Schematic diagram of projection effect of a wearable projection system for motion capture proposed by the present invention;

[0033] Figure 3 It is a schematic diagram of the structure of a display module in a wearable projection system for motion capture proposed by the present invention;

[0034] Figure 4 It is a schematic diagram of a software node cluster in a wearable projection system for motion capture proposed by the present invention;

[0035] Figure 5 It is a schematic diagram of the steps of a projection method proposed by the present invention.

[0036] In the figure: 100, display module; 110, carrier; 120, inner lens; 130, outer lens; 140, liquid storage chamber; 150, electrode; 200, projection module; 300, gesture capture module. DETAILED DESCRIPTION

[0037] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that the discussion of these implementations is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Various examples may omit, replace, or add various processes or components as needed. In addition, the features described in some examples may also be combined in other examples.

[0038] In at least one embodiment of the present invention, a wearable projection system for motion capture is disclosed, such as Figure 1 - Figure 5 As shown, it includes a hardware device cluster and a software node cluster;

[0039] The hardware device cluster includes a display module 100, a projection module 200 and a gesture capture module 300. The projection module 200 is fixedly mounted on the display module 100. The projection module 200 outputs an image by interacting with the gesture capture module 300. The display module 100 is arranged between the user's eyes and the image.

[0040] The display module 100 includes a carrier 110, wherein an inner lens 120 and an outer lens 130 are arranged at intervals upward from the user's eye level on the carrier 110, a liquid storage cavity 140 is arranged between the inner lens 120 and the outer lens 130, and a transparent colorless liquid is stored in the liquid storage cavity 140. A pair of electrodes 150 are arranged in the carrier 110 and are connected to the liquid storage cavity 140, and the shape of the liquid changes with the voltage applied to the electrodes 150;

[0041] The software node cluster includes rendering nodes, data processing nodes, standby nodes, load balancing nodes, storage nodes, cache nodes and monitoring nodes, and each of the nodes cooperates alternately to instruct the hardware device cluster to run.

[0042] The gesture capture module 300 is in the shape of a glove, and includes a plurality of sensors disposed at finger joints for real-time perception of user gestures.

[0043] In order to deal with the image distortion problem caused by the disturbance of the projection module 200 when in use, the present invention is different from the traditional projection system correction method on the projection device itself, and adopts the method of adding a wearable sensing intermediate device (display module 100) for correction. In actual implementation, the shape of the liquid in the liquid storage chamber 140 is controlled by changing the output voltage of the electrode 150, so as to adjust the focal length of the transmission between the inner lens 120 and the outer lens 130, rather than acting on the image output level. The advantages of this are:

[0044] Different people will have different visual experiences when watching projection. Positioning the adjustment operation on the user side can meet the personalized viewing needs of different people.

[0045] By adjusting the focus on the display module 100, the entire projection system can focus on outputting the best image, and then the audience can make fine adjustments by themselves, which helps to reduce the working pressure of the projection device, and there is no need to consider integrating it into the projection output device with the help of software or complex hardware mechanisms;

[0046] By changing the voltage to adjust the focal length and thus correct image distortion, users can more easily adjust their visual experience. This provides a convenient solution, especially in temporary occasions or when a problem needs to be solved quickly. Users no longer need to learn complex technical means, eliminating usage barriers.

[0047] In the specific implementation of the software node cluster, the rendering node is equipped with a GPU to generate 3D rendering content and perform real-time image processing. The data processing node receives and processes data from the gesture capture module 300 and provides real-time feedback. The standby node cooperates with the rendering node and the data processing node to take over the task and continue to run when the rendering node and the data processing node fail. The storage node stores static images, video materials and real-time interactive data. The cache node reduces data access delay during system operation. The monitoring node collects and analyzes the resource usage of each node in real time.

[0048] In the above implementation, the importance of simplifying the structure of the projection device and facilitating adjustment when interacting with motion capture has been reflected. At the software level, the present invention also introduces load management based on the principle of "burden reduction", specifically:

[0049] A projection method, using the wearable projection system for motion capture, comprises the following steps:

[0050] S1. Initialization and system startup: When the system starts, the display module 100, the projection module 200 and the gesture capture module 300 in the hardware device cluster perform self-checks to ensure normal operation, and the monitoring system begins to collect the usage of each node and establish a preliminary load benchmark;

[0051] S2, image content output: the image content pre-stored in the system is projected onto an opaque background via the projection module 200, and the user wears the display module 100 to view the image content;

[0052] S3, user interaction: the user interacts with the system through the change of gestures. The sensor in the gesture capture module 300 captures the user's gesture data and uploads it to the computing server. The data processing node generates a corresponding image or effect after analyzing the data. Then, the rendering node performs rendering processing, regenerates the image content, and then outputs it through the projection module 200 and feeds it back to the user, forming an interactive process.

[0053] S4, distortion compensation: when the image output by the projection module 200 is distorted, the display module 100 is used to adjust and adapt by changing the focal length of the user's observation, thereby compensating for the distortion and blur of the image;

[0054] S5, Load Management Intervention: According to the load benchmark in S1, load management is achieved through resource monitoring, dynamic calculation of resource allocation, confirmation of task priority, establishment of system fault tolerance mechanism, and feedback and adjustment.

[0055] Resource monitoring means that during the entire process, the system will monitor the usage of hardware resources in real time, including CPU and GPU load monitoring, memory usage monitoring and network bandwidth monitoring. First of all, when rendering images, the GPU load is very critical. The system monitors the computing power of the GPU in real time to avoid overload. Secondly, it monitors memory usage in real time to avoid memory leaks or excessive memory usage that causes system performance degradation. Finally, the system needs to transmit data over the network, so it monitors the network bandwidth to prevent delays or packet loss during transmission.

[0056] Dynamic computing resource allocation can effectively promote load balancing. When the system load is low, the use of computing resources can be reduced, such as reducing the resolution of image rendering or turning off some non-real-time background tasks. When the load is high, the system automatically allocates computing tasks to other idle computing nodes or cloud servers. When the system contains multiple GPUs, tasks can be allocated to multiple GPUs to reduce the burden of a single GPU. For example, when the image rendering task is more complex, the computing task can be split and allocated to different GPUs to achieve load balancing. In addition, container technology (such as Docker, Kubernetes, etc.) can be used to manage the system in a containerized manner to ensure that resources can be dynamically expanded when the load changes to ensure performance.

[0057] Confirming the task priority means that the system will set the priority according to the type and urgency of the task. For example, the response task of user interaction should have a higher priority, image rendering should ensure real-time performance, and some background logging tasks can be delayed. Tasks can also be divided into multiple subtasks and executed asynchronously to avoid long-term blocking. For example, tasks such as image rendering and user location calculation can be processed in parallel to improve the system response speed. Under high load conditions, the system automatically adjusts the order of task execution, giving priority to critical tasks and delaying low-priority tasks.

[0058] Establishing a system fault-tolerant mechanism means that when a hardware component (such as a GPU or sensor) fails, the system can automatically transfer tasks to a backup device to avoid system crashes. Through load balancing and fault-tolerant mechanisms, the system can quickly detect hardware failures and automatically transfer tasks to ensure that the user experience is not interrupted.

[0059] During the load management process, the system will regularly provide administrators with load feedback, including CPU / GPU usage, memory usage, etc. Administrators can make manual adjustments based on these data. The system can also automatically optimize computing resource allocation strategies based on historical load data and real-time monitoring, reduce unnecessary resource waste, and improve the overall efficiency of the system.

[0060] In order to realize the above-mentioned interactive projection system architecture and effectively manage the load, the present invention provides corresponding algorithm design, covering resource monitoring, task scheduling, priority management, load balancing, fault tolerance and other aspects.

[0061] 1. Algorithm: Resource Monitoring and Update

[0062] class ResourceMonitor:

[0063] def __init__(self):

[0064] self.cpu_usage = 0 # CPU usage

[0065] self.gpu_usage = 0 #GPU usage

[0066] self.memory_usage = 0 # Memory usage

[0067] self.network_usage = 0 # Network bandwidth usage

[0068] def update_resource_usage(self):

[0069] # Get the current resource usage status

[0070] self.cpu_usage = self.get_cpu_usage()

[0071] self.gpu_usage = self.get_gpu_usage()

[0072] self.memory_usage = self.get_memory_usage()

[0073] self.network_usage = self.get_network_usage()

[0074] def get_cpu_usage(self):

[0075] # Returns CPU usage (simulated data)

[0076] return random.uniform(0, 100)

[0077] def get_gpu_usage(self):

[0078] # Return GPU usage (simulated data)

[0079] return random.uniform(0, 100)

[0080] def get_memory_usage(self):

[0081] # Return memory usage (simulated data)

[0082] return random.uniform(0, 100)

[0083] def get_network_usage(self):

[0084] # Returns network bandwidth usage (simulated data)

[0085] return random.uniform(0, 100)

[0086] The algorithm monitors the system load in real time by periodically updating the utilization of hardware resources. In order to optimize performance, a scheduled task is usually used once every second or every minute.

[0087] 2. Algorithm: Task Scheduling and Priority Management

[0088] import heapq

[0089] class TaskScheduler:

[0090] def __init__(self):

[0091] # Use the minimum heap to manage the task queue, and sort the tasks from high to low priority

[0092] self.task_queue = []

[0093] def schedule_task(self, task, priority):

[0094] # Add tasks to the queue based on priority (lower numbers mean higher priority)

[0095] heapq.heappush(self.task_queue, (priority, task))

[0096] def execute_next_task(self):

[0097] if self.task_queue:

[0098] priority, task = heapq.heappop(self.task_queue)

[0099] task.execute() # Execute task

[0100] else:

[0101] print("No tasks to execute")

[0102] class Task:

[0103] def __init__(self, name):

[0104] self.name = name

[0105] def execute(self):

[0106] print(f"Executing task: {self.name}")

[0107] In the above task scheduling algorithm, TaskScheduler uses the minimum heap to manage the task queue. The priority of the task is managed through the heapq library. Tasks with higher priority (such as user interaction or image rendering) will be executed first.

[0108] 3. Algorithm: Dynamic Resource Allocation and Load Balancing

[0109] class LoadBalancer:

[0110] def __init__(self, nodes):

[0111] self.nodes = nodes # Node list, each node contains the current load information

[0112] def balance_load(self):

[0113] # Select the node with the lightest load to assign tasks

[0114] lightest_node = min(self.nodes, key=lambda node: node.get_load())

[0115] lightest_node.assign_task() # Assign tasks

[0116] def update_node_loads(self):

[0117] # Update the load status of each node

[0118] for node in self.nodes:

[0119] node.update_load()

[0120] class Node:

[0121] def __init__(self, id):

[0122] self.id = id

[0123] self.load = 0 # Node load

[0124] def get_load(self):

[0125] return self.load

[0126] def assign_task(self):

[0127] # Simulate task allocation

[0128] print(f"Task assigned to Node {self.id}")

[0129] self.load += 1 # Load increases after assigning tasks

[0130] def update_load(self):

[0131] # Simulate load update

[0132] self.load = random.randint(0, 10)

[0133] The load balancing algorithm ensures load balancing by selecting the computing node with the lightest load to allocate tasks. The system will regularly update the node load information and allocate tasks to the node with the lightest current load.

[0134] 4. Algorithms: Fault Detection and Recovery

[0135] import time

[0136] class FaultTolerance:

[0137] def __init__(self, nodes):

[0138] self.nodes = nodes

[0139] def monitor_nodes(self):

[0140] # Check the status of each node regularly

[0141] for node in self.nodes:

[0142] if not node.is_healthy():

[0143] self.recover_from_failure(node)

[0144] def recover_from_failure(self, failed_node):

[0145] print(f"Node {failed_node.id} failed. Initiating recovery.")

[0146] # Start the standby node

[0147] backup_node = self.get_backup_node()

[0148] backup_node.activate()

[0149] def get_backup_node(self):

[0150] # Get the backup node (simplified here)

[0151] return Node(999)

[0152] class Node:

[0153] def __init__(self, id):

[0154] self.id = id

[0155] self.healthy = True

[0156] def is_healthy(self):

[0157] # Simulate node health status

[0158] return random.choice([True, False]) # Randomly select healthy or faulty

[0159] def activate(self):

[0160] # Start the standby node

[0161] print(f"Backup Node {self.id} activated.")

[0162] When hardware fails, the system should be able to automatically perform failover to ensure that the task can continue to be executed. Through heartbeat detection and failover mechanisms, automatic replacement of nodes can be achieved. By regularly monitoring the status of nodes, when a node failure is detected, the backup node is automatically started to continue processing the task.

[0163] 5. Algorithm: Adaptive Tuning

[0164] class AdaptiveTuning:

[0165] def __init__(self):

[0166] self.load_history = []

[0167] def adjust_resources(self, current_load):

[0168] # Record historical load data

[0169] self.load_history.append(current_load)

[0170] # Adjust resource allocation based on historical data

[0171] if current_load < 50:

[0172] print("Low load detected. Reducing resource usage.")

[0173] self.reduce_resources()

[0174] elif current_load > 80:

[0175] print("High load detected. Increasing resources.")

[0176] self.increase_resources()

[0177] def reduce_resources(self):

[0178] # Simulate reducing resources

[0179] print("Reducing rendering resolution.")

[0180] def increase_resources(self):

[0181] #Simulate adding resources

[0182] print("Increasing rendering resolution.")

[0183] The adaptive tuning algorithm automatically adjusts system resources according to the current load and historical data. When the load is low, the system can reduce the rendering resolution and reduce computing requirements. When the load is high, it can increase resource allocation.

[0184] The above algorithms work together to ensure that the interactive projection system can run stably and optimize performance under dynamic loads. Combined with resource monitoring, task scheduling, load balancing, failover, and adaptive tuning mechanisms, the system can effectively manage hardware resources and provide an efficient and stable user experience.

[0185] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation mode. The above-mentioned specific implementation mode is merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make more forms of equivalent embodiments, all of which are within the protection of this embodiment.

Claims

1. A wearable projection system for motion capture, characterized in that: Includes hardware device clusters and software node clusters; The hardware device cluster comprises a display module (100), a projection module (200) and a gesture capture module (300); the projection module (200) is fixedly mounted on the display module (100); the projection module (200) outputs an image by interacting with the gesture capture module (300); and the display module (100) is arranged between the user's eyes and the image; The display module (100) comprises a carrier (110), wherein the carrier (110) is provided with an inner lens (120) and an outer lens (130) spaced apart from each other upward at the level of the user's eyes, wherein a liquid storage chamber (140) is provided between the inner lens (120) and the outer lens (130), wherein a transparent colorless liquid is stored in the liquid storage chamber (140), wherein a pair of electrodes (150) which are in electrical communication with the liquid storage chamber (140) are provided in the carrier (110), wherein the shape of the liquid changes with the voltage supplied to the electrodes (150), and the shape of the liquid in the liquid storage chamber (140) is driven to change by changing the output voltage of the electrodes (150), thereby realizing adjustment of the transmission focal length between the inner lens (120) and the outer lens (130); The software node cluster includes a rendering node, a data processing node, a standby node, a load balancing node, a storage node, a cache node and a monitoring node. The rendering node is equipped with a GPU to generate 3D rendering content and perform real-time image processing. The data processing node receives and processes data from a gesture capture module (300) and provides real-time feedback. The standby node cooperates with the rendering node and the data processing node to take over the task and continue to run when the rendering node and the data processing node fail. The storage node stores static images, video materials and real-time interactive data. The cache node reduces data access delay during system operation. The monitoring node collects and analyzes resource usage of each node in real time. The nodes cooperate alternately to instruct the hardware device cluster to operate.

2. A wearable projection system for motion capture according to claim 1, characterized in that: The gesture capturing module (300) is in the shape of a glove and comprises a plurality of sensors arranged at finger joints.

3. A projection method, using a wearable projection system for motion capture according to any one of claims 1 to 2, comprising the following steps: S1, initialization and system startup: When the system starts, the display module (100), projection module (200) and gesture capture module (300) in the hardware device cluster perform self-checks to ensure normal operation, and the monitoring system begins to collect usage information of each node to establish a preliminary load benchmark; S2, image content output: the image content pre-stored in the system is projected onto an opaque background via the projection module (200), and the user wears the display module (100) to view the image content; S3, user interaction: the user interacts with the system through changes in gestures. The sensor in the gesture capture module (300) captures the user's gesture data and uploads it to the computing server. The data processing node analyzes the data and generates a corresponding image or effect. The image content is then regenerated through rendering processing by the rendering node and outputted through the projection module (200) and fed back to the user, forming an interactive process. S4, distortion compensation: when the image output by the projection module (200) is distorted, the display module (100) is used to change the focal length of the user's observation to adjust and adapt, thereby compensating for the distortion and blur of the image; S5, Load Management Intervention: According to the load benchmark in S1, load management is achieved through resource monitoring, dynamic calculation of resource allocation, confirmation of task priority, establishment of system fault tolerance mechanism, and feedback and adjustment.

4. A projection method according to claim 3, characterized in that: The objects of resource monitoring in step S5 include CPU and GPU load monitoring, memory usage monitoring and network bandwidth.

Images