Wearable motion capture data collection method, apparatus, device and medium

Abstract

Embodiments of the present application relate to the field of data processing, and provide a wearable motion capture data acquisition method, device, equipment and medium, comprising: in response to a wake-up interaction operation, detecting the running state of a plurality of acquisition nodes, and determining a target acquisition mode when each acquisition node meets a preset acquisition condition; in response to an acquisition start interaction operation, issuing a synchronization reference and an acquisition trigger signal according to the target acquisition mode, receiving multi-source data collected by each acquisition node at a unified sampling time; fusing and processing the multi-source data to obtain a motion capture acquisition result and perform validity verification; during acquisition operation, in response to an acquisition operation interaction operation, executing a corresponding acquisition control instruction, generating acquisition feedback information according to the instruction execution result, the node running state and the verification result, and sending the acquisition feedback information to the wearable device. The embodiments of the present application can enable the wearer to timely master the operation execution situation and the motion capture data acquisition state, and improve the acquisition interaction efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically to a wearable motion capture data acquisition method, apparatus, device, and medium. Background Technology

[0002] With the development of fields such as robot skill learning, digital human driving, film and animation production, and motion analysis, the demand for simultaneous collection of human motion data and environmental perception data is gradually increasing. Existing wearable motion capture data acquisition solutions typically set up multiple acquisition nodes at different parts of the human body, acquire human motion data through each acquisition node, and combine this with visual acquisition devices to acquire environmental data. The acquired data is then transmitted to processing equipment for processing and storage.

[0003] Existing wearable motion capture data acquisition solutions mostly use wireless transmission, which is easily affected by the electromagnetic environment, resulting in transmission delays or data loss. Some solutions use different acquisition nodes to perform data processing and attitude calculation separately. The lack of a unified time reference between the acquisition nodes makes it difficult to accurately align data from different sources, and is also easily limited by the computing power of the acquisition nodes, affecting the fusion processing effect of human motion data and environmental perception data.

[0004] Furthermore, existing solutions typically require external display devices or host computers to set acquisition parameters, monitor operational status, and troubleshoot anomalies. Wearers often struggle to monitor the operational status of each acquisition node and the quality of the acquired data during the acquisition process. Existing troubleshooting methods also fail to promptly detect issues such as time asynchrony, data jumps, missing node data, or insufficient visual information. This can lead to some acquired data being unusable, increasing the cost of repeated acquisitions and anomaly troubleshooting, and ultimately impacting the efficiency of motion capture data acquisition. Summary of the Invention

[0005] This application provides a wearable motion capture data acquisition method, device, equipment, and medium to solve the problems of inconsistent acquisition timing of multiple acquisition nodes and difficulty for wearers to obtain timely information on the operating status of acquisition nodes and the quality of acquired data in the existing wearable motion capture data acquisition process.

[0006] In a first aspect, embodiments of this application provide a wearable motion capture data acquisition method, the method comprising: In response to the input wake-up interaction, the running status of multiple acquisition nodes is detected, and the target acquisition mode is determined when the running status of the multiple acquisition nodes meets the preset acquisition conditions. In response to the input acquisition initiation interactive operation, a synchronization reference and acquisition trigger signal are sent to the multiple acquisition nodes according to the target acquisition mode, so that the multiple acquisition nodes can acquire multi-source data at a unified sampling time, and receive the multi-source data returned by the multiple acquisition nodes; The multi-source data is fused to obtain motion capture acquisition results, and the motion capture acquisition results are verified to obtain corresponding data validity judgment results; During the data acquisition process, in response to the input data acquisition operation interaction, the system executes the data acquisition control command corresponding to the data acquisition operation interaction, and generates data acquisition feedback information based on the execution result of the data acquisition control command, the operating status of the multiple data acquisition nodes and the data validity determination result. The system then sends the data acquisition feedback information to the wearable device, so that the wearable device can provide feedback on the motion capture data acquisition status to the wearer.

[0007] Secondly, embodiments of this application provide a wearable motion capture data acquisition device, which has the function of implementing the wearable motion capture data acquisition method corresponding to the first aspect described above. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function, and the modules can be software and / or hardware.

[0008] In one embodiment, the wearable motion capture data acquisition device includes: The input / output module is configured to receive wake-up interaction operations, acquisition start interaction operations, and / or acquisition run interaction operations; send synchronization references and acquisition trigger signals to multiple acquisition nodes, and receive multi-source data returned by the multiple acquisition nodes; and send the acquisition feedback information to the wearable device, so that the wearable device can provide feedback on the motion capture acquisition status to the wearer. The processing module is configured to, in response to an input wake-up interaction, detect the operating status of the multiple acquisition nodes and determine the target acquisition mode if the operating status of the multiple acquisition nodes meets the preset acquisition conditions; in response to an input acquisition start interaction, send a synchronization reference and acquisition trigger signal to the multiple acquisition nodes through the input / output module according to the target acquisition mode, so that the multiple acquisition nodes acquire the multi-source data at a unified sampling time; perform fusion processing on the multi-source data to obtain motion capture acquisition results, and verify the motion capture acquisition results to obtain corresponding data validity judgment results; during acquisition operation, in response to an input acquisition operation interaction, execute the acquisition control command corresponding to the acquisition operation interaction, and generate acquisition feedback information based on the execution result of the acquisition control command, the operating status of the multiple acquisition nodes, and the data validity judgment result.

[0009] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the wearable motion capture data acquisition method as described in the first aspect.

[0010] Fourthly, embodiments of this application provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the wearable motion capture data acquisition method described in the first aspect.

[0011] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to execute the wearable motion capture data acquisition method provided in the first aspect.

[0012] Compared to existing technologies, in this embodiment, the operating status of multiple acquisition nodes is detected in response to the wake-up interaction operation. The target acquisition mode is determined after the operating status of multiple acquisition nodes meets the preset acquisition conditions, which reduces the occurrence of acquisition nodes that are not ready or are malfunctioning participating in data acquisition. Furthermore, in response to the acquisition start interaction operation, synchronization references and acquisition trigger signals are sent to multiple acquisition nodes according to the target acquisition mode, enabling multiple acquisition nodes to acquire multi-source data at a unified sampling time, which helps improve the time consistency between data from different sources. Next, by fusing the multi-source data and verifying the motion capture acquisition results, the valid status of the acquired data can be determined. During the acquisition operation, the execution results of the acquisition control commands, the operating status of multiple acquisition nodes, and the data validity determination results are generated into acquisition feedback information and sent to the wearable device. This allows the wearer to promptly grasp the execution status of the acquisition operation, the operating status of the acquisition nodes, and the quality of the acquired data through the wearable device, without frequently relying on external display devices or host computers to check the acquisition status. This facilitates timely detection of acquisition anomalies and corresponding adjustments, reduces invalid data and duplicate acquisition, and improves the interactive efficiency and data availability of wearable motion capture data acquisition. Attached Figure Description

[0013] The objectives, features, and advantages of the embodiments of this application will become readily understood by referring to the accompanying drawings and the detailed description of the embodiments. Wherein: Figure 1 This is a schematic diagram of the structure of a wearable motion capture data acquisition device provided in an embodiment of this application; Figure 2 A flowchart illustrating a wearable motion capture data acquisition method provided in this application embodiment; Figure 3 A schematic diagram of a wearable motion capture data acquisition process based on tapping interaction is provided for an embodiment of this application; Figure 4This is a schematic diagram of the structure of the wearable motion capture data acquisition device provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application; Figure 6 This is a schematic diagram of another computing device provided in an embodiment of this application.

[0014] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0015] The wearable motion capture data acquisition method provided in this application can be applied to scenarios such as robot imitation learning, robot dexterity operation, digital human driving, film and animation production, motion analysis, and real-world operation data acquisition. In these scenarios, the wearer can simultaneously collect human motion information and environmental information of the wearer's environment through wearable devices and multiple acquisition nodes distributed on different parts of the body, and the computing device can uniformly control and provide status feedback for the acquisition process.

[0016] Optionally, the wearable motion capture data acquisition method is applied to a computing device. The computing device is communicatively connected to the wearable device and multiple acquisition nodes, including a master acquisition node and multiple slave acquisition nodes distributed across multiple body parts of the wearer. The master acquisition node is used to acquire the wearer's head movement information and environmental information, while the multiple slave acquisition nodes are used to acquire movement information of corresponding body parts. The computing device can perform status detection, synchronization control, and acquisition scheduling of the multiple acquisition nodes based on interactive operations input by the wearer through the wearable device, and receive multi-source data returned by the multiple acquisition nodes.

[0017] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a wearable motion capture data acquisition device provided in an embodiment of this application is shown. Figure 1 The device includes a helmet, a head-up display (HUD), arm acquisition units, hand acquisition units, and leg acquisition units. The HUD is positioned within the wearer's head field of vision. The arm, hand, and leg acquisition units are positioned on their respective body parts. Figure 1 This illustration only shows the placement of some wearable components and data acquisition units and does not constitute a limitation on the number, installation location, shape, or connection method of data acquisition nodes. The computing device and its transmission links with each data acquisition node are... Figure 1 It is not shown separately in the text.

[0018] In one specific embodiment, a helmet can constitute the wearable device, and the main acquisition node can be integrated into the helmet. The main acquisition node may include a panoramic vision acquisition unit, a depth sensing unit, and a head inertial measurement unit. The panoramic vision acquisition unit is used to acquire environmental images of the wearer's surroundings, the depth sensing unit is used to acquire corresponding environmental depth information, and the head inertial measurement unit is used to acquire the wearer's head movement information. After the environmental images, environmental depth information, and head movement information acquired by the main acquisition node are respectively added with corresponding node identifiers and sampling time information, they can be sent to a computing device. Optionally, the head-mounted wearable device can also be equipped with an eye-tracking module to acquire eye movement data such as the wearer's gaze point, gaze direction, blink frequency, or pupil diameter. Furthermore, after adding timestamps to the eye movement data, it can be spatiotemporally aligned with the action information and environmental information, expanding the application scope of the dataset.

[0019] The helmet can also be equipped with a HUD, interactive components, and vibration components. The wearer can input wake-up, acquisition mode selection, acquisition start, acquisition run, or acquisition end interactive operations through the interactive components. The computing device can execute corresponding acquisition control commands based on these interactive operations and send the operating status of the acquisition nodes, the execution results of the acquisition control commands, and the motion capture data acquisition status to the helmet. The HUD is used to display corresponding acquisition feedback information to the wearer, and the vibration components are used to output corresponding tactile feedback based on different interactive operations. The interactive components can be, for example, a damped knob, a touch-sensitive area, or a vibration sensor for detecting tapping operations.

[0020] The multiple acquisition nodes can be installed on the motion capture suit worn by the wearer. Specifically, the multiple acquisition nodes can be distributed across the wearer's torso, upper arm, forearm, thigh, calf, hand, and fingers. Each acquisition node can include an inertial measurement unit (IMU) for acquiring inertial motion information such as acceleration, angular velocity, and magnetic field information for the corresponding body part. Each acquisition node can perform data acquisition based on the synchronization reference, acquisition trigger signal, and acquisition parameters issued by the computing device, and add a node identifier and local timestamp to the acquired data so that the computing device can identify the source of different data and perform time alignment of the data returned by different acquisition nodes.

[0021] In one specific embodiment, the computing device can be a portable, small-scale chemical control host, equipped with a processor, a computing acceleration unit, storage media, and multiple communication interfaces. The helmet and motion capture suit can be connected to the computing device via wired transmission links. Multiple slave acquisition nodes on the motion capture suit can aggregate data via an internal wired bus and then connect to the computing device via a backbone cable; the master acquisition node can be connected to the computing device via an independent wired transmission link. Thus, environmental information collected by the master acquisition node and motion information collected by multiple slave acquisition nodes can be transmitted through different transmission links, reducing bandwidth contention and mutual interference between different types of data during transmission.

[0022] In this embodiment, the computing device can perform unified clock synchronization, acquisition triggering, and parameter configuration for multiple acquisition nodes, and centrally process the multi-source data returned by the multiple acquisition nodes. The computing device can also monitor the connection status, synchronization status, data frame rate, and data validity status of each acquisition node, generate acquisition feedback information based on the monitoring results, and send the feedback information to the helmet's HUD for display. Therefore, the wearer can input acquisition operations through the wearable device during data acquisition and promptly obtain the execution results of the acquisition operations, the operating status of each acquisition node, and the current motion capture data acquisition status.

[0023] Figure 2 This is a flowchart illustrating a wearable motion capture data acquisition method provided in an embodiment of this application. The method can be executed by a wearable motion capture data acquisition device. The wearable motion capture data acquisition device can be deployed on a computing device that is communicatively connected to a wearable device and multiple acquisition nodes. This method can be applied to scenarios requiring simultaneous acquisition of human motion information and environmental information, such as robot skill learning, robot dexterity operation, digital human driving, film and animation production, motion analysis, and real-world scene operation data acquisition. (Refer to...) Figure 2 The method includes steps 101-104: Step 101: In response to the input wake-up interaction operation, detect the running status of multiple acquisition nodes, and determine the target acquisition mode if the running status of the multiple acquisition nodes meets the preset acquisition conditions.

[0024] The wake-up interaction can be input by the wearer through the wearable device to wake up the computing device and / or multiple data acquisition nodes, and trigger pre-data acquisition state detection. The wearable device can be, for example, a head-mounted wearable device equipped with interactive components, a heads-up display, and a vibration component. The interactive components may include damped knobs, touch-sensitive areas, or vibration sensors, and the wearer can input the wake-up interaction by rotating, pressing, sliding, clicking, or tapping.

[0025] When detecting the operational status of multiple acquisition nodes, the computing device can send wake-up messages and node query messages to the multiple acquisition nodes, and receive node identifiers and status information returned by each acquisition node. The operational status may include at least one of connection status, synchronization status, and pending acquisition status. The operational status of multiple acquisition nodes all meeting preset acquisition conditions can mean that each acquisition node is in a pending acquisition state that is connected, synchronized, and capable of performing data acquisition.

[0026] When the operating states of multiple acquisition nodes all meet the preset acquisition conditions, the computing device can respond to the wearer's input of an acquisition mode selection interaction and determine the target acquisition mode from multiple candidate acquisition modes. Candidate acquisition modes may include, for example, a daily mode, a high-precision mode, and a low-power mode. Different acquisition modes may correspond to different node activation ranges, sampling rates, or other acquisition parameters. The computing device can also send the determined target acquisition mode to the wearable device, which will then display the target acquisition mode to the wearer and / or output corresponding haptic feedback. If the operating state of any acquisition node does not meet the preset acquisition conditions, the computing device can generate abnormal feedback information based on the abnormal acquisition node and its operating state, allowing the wearer to check the corresponding acquisition node.

[0027] In this embodiment, the multiple acquisition nodes include a master acquisition node and multiple slave acquisition nodes. Optionally, the master acquisition node is located on a head-mounted wearable device, and the multi-source data acquired by the master acquisition node includes environmental images, environmental depth information, and head inertial information. The multiple slave acquisition nodes are respectively located on multiple body parts of the wearer's torso, limbs, hands, and fingers, and the multi-source data acquired by each slave acquisition node includes inertial motion information of the corresponding body part. The multiple acquisition nodes add node identifiers and local timestamps to the acquired multi-source data to form multi-source data with the aforementioned unified sampling time relationship.

[0028] For reference Figure 1 Taking the device shown as an example, the figure illustrates a helmet worn by the wearer, a head-up display (HUD), and data acquisition units located on the arms, hands, and legs. The helmet can serve as a head-mounted wearable device, integrating the main data acquisition node and the HUD. Figure 1 The arm acquisition unit, hand acquisition unit, and leg acquisition unit shown can be used as slave acquisition nodes to acquire motion information of the corresponding body parts. Figure 1 This illustration only shows some of the data acquisition nodes and their locations. Data acquisition nodes can also be set on parts of the body such as the torso, upper arm, forearm, thigh, calf, palm, or fingers that are not separately marked in the figure. This application does not limit the specific number or location of data acquisition nodes.

[0029] For example, after receiving a wake-up interaction from the wearer, the computing device can send wake-up information to the main acquisition node in the helmet and multiple slave acquisition nodes located on the torso, limbs, hands, and fingers. The main acquisition node can return a node identifier used to identify the helmet nodes, as well as the current status of the panoramic vision acquisition unit, depth sensing unit, and head inertial measurement unit; each slave acquisition node can return its corresponding node identifier and the current status of the inertial measurement unit located on the corresponding body part. The computing device completes the acquisition node registration based on the returned information and establishes a node topology according to the acquisition locations such as head, torso, upper limbs, hands, and lower limbs. The computing device can also send the node topology and the operating status of each node to the wearable device, where the HUD displays whether each acquisition node is properly connected according to different acquisition locations.

[0030] Alternatively, preset acquisition conditions can be set separately according to the node type, acquisition object, and data channel configuration of the acquisition nodes. Considering that the acquisition objects and data channels of the master acquisition node and the slave acquisition nodes are different, the preset acquisition conditions may include common acquisition conditions applicable to multiple acquisition nodes, as well as node acquisition conditions corresponding to different node types.

[0031] For example, common acquisition conditions may include node registration completion, normal communication connection, clock synchronization completion, and entry into the acquisition state. For the master acquisition node, node acquisition conditions may also include the environmental image acquisition channel, environmental depth information acquisition channel, and head inertial information acquisition channel being in an acquireable state. For slave acquisition nodes, node acquisition conditions may also include the corresponding inertial measurement unit being in an acquireable state. The computing device compares the operating status of each acquisition node with its corresponding preset acquisition conditions. When all acquisition nodes participating in the current acquisition task meet their respective preset acquisition conditions, it is determined that the overall operating status of multiple acquisition nodes meets the preset acquisition conditions. Thus, different acquisition nodes can use preset acquisition conditions adapted to their respective node types, acquisition objects, and data channels, without having to use completely identical status indicators.

[0032] In this application embodiment, the phrase "the operating status of multiple acquisition nodes all meets preset acquisition conditions" can mean that each acquisition node meets common acquisition conditions, and the master acquisition node and each slave acquisition node also meet node acquisition conditions corresponding to their respective node types. Only when all multiple acquisition nodes participating in the current acquisition task have normal acquisition capabilities will the computing device be allowed to enter the acquisition mode selection and subsequent acquisition startup phase.

[0033] When the operating states of multiple acquisition nodes all meet the preset acquisition conditions, the computing device can respond to the wearer's input of an acquisition mode selection interaction and determine the target acquisition mode from multiple candidate acquisition modes. Candidate acquisition modes may include, for example, a daily mode, a high-precision mode, and a low-power mode. Different acquisition modes may correspond to different node sampling rates, data channel configurations, or other acquisition parameters. For example, in high-precision mode, the sampling rate of critical acquisition nodes such as head and hand acquisition nodes can be increased; in low-power mode, the sampling rate of non-critical acquisition nodes can be reduced; and in daily mode, each acquisition node can perform data acquisition according to preset conventional sampling parameters.

[0034] After determining the target acquisition mode, the computing device can generate mode feedback information based on the target acquisition mode and send the mode feedback information to the wearable device. The wearable device can display the name of the target acquisition mode, the corresponding node activation status, and / or acquisition parameters through a HUD, and can also output corresponding tactile feedback through vibration components so that the wearer can confirm the currently selected acquisition mode.

[0035] For example, after all acquisition nodes have been successfully connected and synchronized, the HUD can display that all acquisition nodes are in a ready-to-acquire state. The wearer can rotate the damped knob to cycle through daily mode, high-precision mode, and low-power mode. Each time a candidate acquisition mode is switched, the wearable device can output a step-like vibration and update the currently displayed candidate acquisition mode in the HUD. When the wearer switches to high-precision mode and briefly presses the damped knob, the computing device can determine high-precision mode as the target acquisition mode, and the wearable device will display confirmation information for the target acquisition mode, providing a basis for subsequently starting data acquisition from multiple acquisition nodes according to high-precision mode.

[0036] As an optional embodiment, in step 101, in response to the input wake-up interaction operation, the running status of multiple acquisition nodes is detected, and a target acquisition mode is determined when the running status of the multiple acquisition nodes all meet the preset acquisition conditions. This includes: in response to the wake-up interaction operation, sending wake-up information and node query information to the multiple acquisition nodes; receiving node identifiers and initial status information returned by each acquisition node, registering the multiple acquisition nodes and establishing a node topology; determining the running status of the multiple acquisition nodes based on their connection status and synchronization status; if the multiple acquisition nodes are all in a connected and synchronized pending acquisition state, then it is determined that the running status of the multiple acquisition nodes all meet the preset acquisition conditions; in response to the input acquisition mode selection interaction operation, determining the target acquisition mode from multiple candidate acquisition modes, and sending mode feedback information to the wearable device to provide feedback on the target acquisition mode.

[0037] Specifically, the wearer can initiate a wake-up interaction through the wearable device when it is in sleep mode or has not yet entered data acquisition mode. This wake-up interaction can be input via a damped knob, a touch-sensitive area, or a vibration sensor. For example, the wearer can press and hold a damped knob, tap a touch-sensitive area, or strike the head-mounted wearable device to trigger the wake-up interaction. After detecting the wake-up interaction, the wearable device sends it to the computing device.

[0038] In response to the wake-up interaction, the computing device first sends wake-up information to multiple acquisition nodes. The wake-up information is used to control the acquisition nodes in the dormant state to start their local processing and acquisition components and enter a working state capable of receiving status queries, synchronization control, and acquisition control.

[0039] The multiple acquisition nodes may include a main acquisition node installed on a head-mounted wearable device, and multiple slave acquisition nodes installed on different body parts of the motion capture suit. The computing device can send wake-up information through transmission links connected to the main acquisition node and the multiple slave acquisition nodes. Alternatively, the multiple slave acquisition nodes can first receive a unified wake-up message through a wired bus within the motion capture suit, and then each perform its own local wake-up processing.

[0040] After sending the wake-up message, the computing device sends node query information to multiple acquisition nodes. The node query information is used to query the acquisition nodes currently connected to the computing device and the initial working status of each acquisition node. The node query information may include, for example, node address query commands and status query commands, enabling each acquisition node to return information that characterizes its identity and current status. By waking up the acquisition nodes first and then querying their identities and initial statuses, the device avoids directly querying nodes in a dormant state and obtaining no response, and provides input for subsequent node registration, node topology establishment, and operational status determination.

[0041] After receiving the node query information, each acquisition node can perform local status detection and return a node identifier and initial status information to the computing device. The node identifier is used to uniquely or relatively uniquely distinguish different acquisition nodes in the current wearable motion capture data acquisition process. The node identifier can be associated with the node type and location of the acquisition node. For example, the node identifier of the master acquisition node can correspond to the head position; the node identifiers of multiple slave acquisition nodes can correspond to body parts such as the torso, upper arm, forearm, thigh, calf, hand, or fingers. The initial status information indicates the current status of the acquisition node after wake-up and can include at least one of the following: hardware model, firmware version, initial battery level, connection status, and local self-test result. The initial status information returned by the master acquisition node can also indicate whether the environmental image acquisition channel, environmental depth information acquisition channel, and head inertial information acquisition channel are functioning normally. The initial status information returned by the slave acquisition nodes can indicate whether the corresponding inertial measurement unit is functioning normally.

[0042] The computing device can record the response status of each acquisition node according to the order in which the node identifier and initial status information are received. For acquisition nodes that have returned node identifiers and initial status information, it can be determined that the acquisition node has responded to the node query. For acquisition nodes that have not returned information within a preset query time, they can be temporarily identified as unconnected nodes or unresponsive nodes, and corresponding abnormal feedback information will be generated subsequently.

[0043] The computing device can determine the connection status of each data acquisition node based on the response to node query information, node registration results, and transmission link status. When the computing device can communicate with a data acquisition node through the corresponding transmission link and has received the node identifier and initial status information returned by the data acquisition node, the data acquisition node can be determined to be in a connected state. When the computing device does not receive information returned by a data acquisition node within a preset query time, or when the corresponding transmission link cannot communicate normally, the data acquisition node can be determined to be in a disconnected state.

[0044] For multiple slave acquisition nodes connected via the wired bus within the motion acquisition suit, the computing device can also determine whether each slave acquisition node is properly connected to the service bus based on the node access information returned by the motion acquisition suit's aggregation interface. This allows for the differentiation between different situations, such as the motion acquisition suit as a whole not being connected, and a specific slave acquisition node within the motion acquisition suit not being connected.

[0045] The connection status determination result can be associated with node identifiers and node topology. For example, if the computing device determines that the right-hand acquisition node is not connected, it can locate the connection anomaly to the wearer's right hand based on the node topology, so as to provide the wearer with targeted inspection prompts in the future.

[0046] After the nodes are properly connected, the computing device can perform clock synchronization processing on multiple acquisition nodes and determine the synchronization status of each acquisition node based on the results of the clock synchronization processing. For example, the computing device can first send a reference timestamp to each acquisition node, enabling each acquisition node to adjust its local clock according to the reference timestamp, thus completing preliminary time alignment. The computing device can also continuously send multiple sets of synchronization pulses and determine the corresponding clock drift and link transmission delay based on the clock feedback values ​​returned by each acquisition node, and then send the corresponding clock calibration parameters to each acquisition node.

[0047] When a data acquisition node completes local clock adjustment according to clock calibration parameters, and its time deviation relative to the computing device's reference clock meets preset synchronization requirements, the acquisition node can be identified as being in a synchronized complete state. When a data acquisition node has not yet completed clock adjustment, has not returned a clock feedback value, or its time deviation does not meet preset synchronization requirements, the acquisition node can be identified as being in a non-synchronized state. Connection status indicates whether the computing device can communicate normally with the corresponding acquisition node, while synchronization status indicates whether the corresponding acquisition node can perform data acquisition according to a unified time reference. By separately detecting connection status and synchronization status, the direct equation of "able to communicate" with "able to synchronize data acquisition" can be avoided. The computing device can determine the operating status of each acquisition node based on its connection status and synchronization status.

[0048] When a data acquisition node is connected and clock synchronization is complete, the computing device can further determine whether the node has completed local self-test and is able to receive acquisition parameters and acquisition trigger signals. If all the above conditions are met, the node can be designated as a data acquisition target. When a data acquisition node is not connected, its operating status can be designated as disconnected; when a data acquisition node is connected but clock synchronization is not complete, its operating status can be designated as synchronization abnormal; when a data acquisition node is connected and synchronization is complete, but the corresponding data acquisition channel is not working properly, its operating status can be designated as acquisition abnormal.

[0049] The computing device can traverse multiple acquisition nodes participating in the current acquisition task and compare the operating status of each acquisition node with the corresponding preset acquisition conditions. If multiple acquisition nodes are all in a connected and synchronized state ready for acquisition, and each meets the node acquisition conditions corresponding to its respective node type, then the computing device determines that the operating status of the multiple acquisition nodes meets the preset acquisition conditions. At this point, the acquisition mode selection stage can be entered.

[0050] If any acquisition node is not connected, fails to synchronize, or the data acquisition channel is not functioning properly, the computing device determines that the operating status of multiple acquisition nodes does not meet the preset acquisition conditions and suspends the acquisition startup function. The computing device can also generate abnormal feedback information based on the acquisition nodes that do not meet the preset acquisition conditions, prompting the wearer to check the corresponding nodes.

[0051] For example, if the current data acquisition task requires acquisition nodes for the head, torso, arms, hands, and legs, and all acquisition nodes have completed registration, connection, and clock synchronization, the computing device can mark all acquisition nodes as pending acquisition and determine that the operating status of multiple acquisition nodes meets the preset acquisition conditions. If the right-hand acquisition node is not connected, even if other nodes are normal, the computing device can still determine that multiple acquisition nodes as a whole do not meet the preset acquisition conditions.

[0052] When the operating status of multiple acquisition nodes meets the preset acquisition conditions, the computing device can enable the acquisition mode selection function and prompt the wearer to select an acquisition mode via the wearable device. The wearer can input the acquisition mode selection interaction operation through a damped knob, a touch-sensitive area, or tapping. For example, the wearer can rotate the damped knob or slide up and down on the touch-sensitive area to switch candidate acquisition modes sequentially; or they can tap the wearable device twice consecutively to cycle through the candidate acquisition modes in a preset order. The candidate acquisition modes can include a daily mode, a high-precision mode, and a low-power mode. The daily mode corresponds to the normal acquisition parameters of each acquisition node; the high-precision mode can increase the sampling rate of key acquisition nodes such as the head and hands; the low-power mode can reduce the sampling rate of non-critical acquisition nodes to reduce the power consumption required for data acquisition, transmission, and processing. In this way, the computing device can determine the currently selected candidate acquisition mode based on the acquisition mode selection interaction operation, and after receiving a confirmation operation, determine the candidate acquisition mode as the target acquisition mode.

[0053] After determining the target acquisition mode, the computing device can generate mode feedback information based on the target acquisition mode. The mode feedback information may include the name of the target acquisition mode, the range of acquisition nodes corresponding to the target acquisition mode, the sampling rate configuration, and / or the acquisition parameter level. The computing device can send the mode feedback information to the wearable device. After receiving the mode feedback information, the wearable device can display the currently determined target acquisition mode through a heads-up display and output corresponding tactile feedback through a vibration component. For example, the wearer rotates a damping knob to sequentially switch candidate acquisition modes from the daily mode to the high-precision mode. Each time the damping knob is rotated one level, the wearable device can output a short, stepped vibration and update the current candidate acquisition mode in the heads-up display. After the wearer briefly presses the damping knob to confirm the high-precision mode, the computing device determines the high-precision mode as the target acquisition mode, the wearable device displays mode feedback information such as "High-precision mode confirmed," and outputs a confirmation vibration.

[0054] The mode feedback information allows the wearer to confirm the target acquisition mode and understand the current acquisition parameter configuration before starting data acquisition, reducing errors in acquisition mode selection. At this point, the computing device has completed node wake-up, node networking, operational status assessment, and target acquisition mode determination before acquisition, providing a foundation for subsequent distribution of synchronization benchmarks, acquisition trigger signals, and acquisition parameters.

[0055] Optionally, if any acquisition node is not connected or has not completed synchronization, it is determined that the operating status of the plurality of acquisition nodes does not meet the preset acquisition conditions. An anomaly feedback message is generated based on the first abnormal acquisition node that is not connected or has not completed synchronization, and this message is sent to the wearable device. After the operating status of the first abnormal acquisition node is updated, the operating status of the plurality of acquisition nodes is re-checked until the operating status of all the plurality of acquisition nodes meets the preset acquisition conditions. The first abnormal acquisition node is the acquisition node that currently does not meet the preset acquisition conditions. The anomaly feedback message may include at least one of the following: the node identifier of the first abnormal acquisition node, the corresponding body part, the connection status, the synchronization status, and an anomaly handling prompt. By feeding back the abnormal acquisition nodes and their operating status to the wearer, the wearer can specifically check the corresponding acquisition nodes and avoid initiating data acquisition when acquisition nodes are missing or have not completed synchronization.

[0056] Optionally, when any acquisition node is not connected or has not completed synchronization, the computing device pauses data acquisition and re-acquires the connection and synchronization status of that acquisition node according to a preset detection cycle. Alternatively, after receiving the status update information reported by that acquisition node, the device re-evaluates whether the operating status of multiple acquisition nodes meets the preset acquisition conditions until all acquisition nodes have entered a connected and synchronized ready-to-acquire state before allowing the wearer to perform acquisition mode selection and acquisition initiation operations.

[0057] For example, re-acquiring the operating status according to a preset detection cycle is a periodic detection method, suitable for situations where the acquisition nodes cannot actively report status changes. Re-judging based on updated status information is an event-triggered method, which can update the judgment result promptly after the acquisition node restores its connection or completes synchronization. Both methods allow the acquisition startup operation to be constrained by the actual operating status of multiple acquisition nodes, reducing the loss of multi-source data or inconsistencies in time relationships caused by abnormal node status.

[0058] Step 102: In response to the input acquisition start interactive operation, a synchronization reference and acquisition trigger signal are sent to the multiple acquisition nodes according to the target acquisition mode, so that the multiple acquisition nodes can acquire multi-source data at a unified sampling time and receive the multi-source data returned by the multiple acquisition nodes.

[0059] The data acquisition initiation interaction can be initiated by the wearer through the wearable device after confirming the target acquisition mode and the operating status of multiple acquisition nodes. In response to the data acquisition initiation interaction, the computing device can determine the acquisition parameters of each acquisition node according to the target acquisition mode, and send synchronization references, acquisition trigger signals, and corresponding acquisition parameters to multiple acquisition nodes.

[0060] The synchronization reference enables multiple acquisition nodes to perform data acquisition based on a unified time reference, while the acquisition trigger signal controls the multiple acquisition nodes to start data acquisition at a unified sampling time. Acquisition parameters may include at least one of sampling rate, acquisition channels, and acquisition accuracy. For example, in high-precision mode, the sampling rate of critical acquisition nodes such as head and hand acquisition nodes can be increased; in low-power mode, the sampling rate of non-critical acquisition nodes can be reduced.

[0061] As an optional embodiment, in step 102 above, sending a synchronization reference and acquisition trigger signal to multiple acquisition nodes according to the target acquisition mode, so that the multiple acquisition nodes acquire multi-source data at a unified sampling time, includes: sending a reference timestamp to the multiple acquisition nodes, so that the multiple acquisition nodes adjust their local clocks according to the reference timestamp; sending multiple sets of synchronization pulses to the multiple acquisition nodes, and determining the corresponding clock drift and link transmission delay according to the clock feedback value returned by each acquisition node; generating clock calibration parameters according to the clock drift and the link transmission delay, and sending the clock calibration parameters to the corresponding acquisition nodes; determining the acquisition parameters corresponding to each acquisition node according to the target acquisition mode, and sending the acquisition trigger signal and the corresponding acquisition parameters to the multiple acquisition nodes, so that the multiple acquisition nodes start data acquisition at the unified sampling time according to the calibrated local clock.

[0062] Specifically, after multiple acquisition nodes complete registration and initial network formation, the computing device can first perform coarse synchronization processing. The computing device uses its own clock as the reference clock and sends a reference timestamp to each acquisition node; after receiving the reference timestamp, each acquisition node adjusts its local clock to the time corresponding to the reference timestamp, completing the initial time alignment between different acquisition nodes.

[0063] After coarse synchronization is completed, the computing device can perform fine synchronization. The computing device continuously sends multiple sets of hardware synchronization pulses to multiple acquisition nodes and reads the clock feedback values ​​returned by each acquisition node for each set of hardware synchronization pulses. Based on the transmission time of the hardware synchronization pulses, the clock feedback values, and the corresponding reception time, the computing device can determine the clock drift of each acquisition node relative to the reference clock and the transmission delay of the corresponding transmission link, and then generate clock calibration parameters for different acquisition nodes. After receiving the corresponding clock calibration parameters, each acquisition node can adjust its local clock frequency and / or local time to reduce the time deviation caused by crystal oscillator drift and link transmission delay.

[0064] It should be noted that the clock drift, link transmission delay, and clock calibration parameters can differ for each acquisition node. The computing device can generate corresponding clock calibration parameters for each acquisition node based on the clock feedback values ​​returned by each node, rather than using the same calibration parameters to adjust all acquisition nodes. After clock calibration is completed, multiple acquisition nodes can enter a synchronized, ready-to-acquire state, and the computing device can also send the synchronization status of each acquisition node to the wearable device for display.

[0065] After determining the target acquisition mode, the computing device can read acquisition parameters such as sampling rate, acquisition channels, and acquisition accuracy of each acquisition node from the acquisition parameter configuration corresponding to the target acquisition mode. In response to the wearer's input of an acquisition initiation interaction, the computing device verifies the acquisition initiation interaction; upon successful verification, it sends acquisition trigger signals and corresponding acquisition parameters to multiple acquisition nodes. Each acquisition node, responding to the same acquisition trigger signal, initiates data acquisition at the unified sampling time indicated by the acquisition trigger signal according to its calibrated local clock, and maintains a corresponding acquisition rhythm according to the received acquisition parameters.

[0066] For example, when a wearer selects a target acquisition mode and inputs an interactive operation to initiate acquisition via the wearable device, the computing device can send a unified acquisition start trigger pulse to the main acquisition node in the helmet and multiple slave acquisition nodes on the motion acquisition suit. Responding to the acquisition start trigger pulse, the main acquisition node can simultaneously activate the panoramic vision acquisition unit, the depth perception unit, and the head inertial measurement unit to acquire environmental images, environmental depth information, and head inertial information, respectively. Each slave acquisition node, responding to the same acquisition start trigger pulse, acquires inertial motion information of corresponding body parts such as the torso, limbs, hands, and fingers.

[0067] The master acquisition node and multiple slave acquisition nodes can add node identifiers and local timestamps to the acquired data and encapsulate it according to a preset data frame format. Multiple slave acquisition nodes on the motion capture suit can first aggregate inertial motion information to the suit's aggregation interface via the suit's wired bus, and then send it to the computing device via the corresponding wired transmission link. The environmental image, environmental depth information, and head inertial information acquired by the master acquisition node can be sent to the computing device via another wired transmission link. The computing device thus receives multi-source data returned by multiple acquisition nodes, can identify the data source based on the node identifier, and determine the sampling time relationship between each data stream based on the local timestamp.

[0068] The above synchronization process first uses a reference timestamp to complete the initial time alignment between different acquisition nodes, then compensates for clock drift and link transmission delay based on the clock feedback value of each acquisition node, and controls each acquisition node to start data acquisition through a unified acquisition trigger signal. This can reduce the sampling time deviation caused by local clock differences and transmission path differences between different acquisition nodes, and provide a unified time basis for subsequent frame ordering, timestamp remapping and fusion processing of multi-source data.

[0069] Further optionally, in the above embodiments, determining the acquisition parameters corresponding to each acquisition node according to the target acquisition mode includes: when the target acquisition mode is a high-precision mode, increasing the sampling rate of the head acquisition node and hand acquisition node among the plurality of acquisition nodes; when the target acquisition mode is a low-power mode, reducing the sampling rate of non-critical acquisition nodes among the plurality of acquisition nodes; when the target acquisition mode is a daily mode, controlling the plurality of acquisition nodes to acquire the multi-source data according to the corresponding preset sampling rate.

[0070] Specifically, the computing device can pre-establish a correspondence between candidate acquisition modes and the acquisition parameters of each acquisition node. After determining the target acquisition mode, the computing device can determine the target sampling rate and / or the acquisition channels to be enabled for each acquisition node based on the correspondence, and then send the determined acquisition parameters to the corresponding acquisition nodes. Each acquisition node performs data acquisition according to the acquisition parameters sent by the computing device, without changing the acquisition parameters itself, so that the acquisition rhythm of multiple acquisition nodes is consistent with the target acquisition mode.

[0071] When the target acquisition mode is high-precision, the computing device can identify the head and hand acquisition nodes as key acquisition nodes and increase the sampling rate of these key nodes. The head inertial information and environmental information acquired by the head acquisition node can be used to determine the wearer's global pose and correct cumulative drift caused by inertial calculations. The inertial motion information acquired by the hand acquisition node can be used to reconstruct the fine movements of the hand and finger joints. Therefore, increasing the sampling rate of the head and hand acquisition nodes increases the number of samples of corresponding body parts and environmental information per unit time, providing input data with higher temporal resolution for global pose determination and fine hand movement calculations.

[0072] For example, in a robot operation data acquisition scenario, the wearer can select a high-precision mode through the wearable device. After the computing device determines the high-precision mode, it can increase the sampling rate of the head acquisition node, the palm acquisition nodes, and the finger acquisition nodes, and send the corresponding acquisition parameters to the aforementioned key acquisition nodes. After the wearer inputs the acquisition to initiate the interactive operation, the head acquisition node and the hand acquisition node acquire head and hand movement information according to the increased sampling rate, while other acquisition nodes synchronously acquire the movement information of the corresponding body parts according to the acquisition parameters corresponding to the high-precision mode. Thus, while maintaining the acquisition of full-body movements, the acquisition accuracy of head posture and fine hand movements can be improved.

[0073] Alternatively, when the synchronization signal of any acquisition node is briefly lost, the computing device can control the acquisition node to maintain a local clock based on historical clock calibration parameters and report the abnormal status. If the duration of the synchronization signal loss reaches a preset duration threshold, the acquisition node is marked as failed and an alarm is triggered to prevent a single node synchronization failure from affecting other acquisition nodes.

[0074] When the target acquisition mode is low-power mode, the computing device can determine non-critical acquisition nodes based on the current acquisition task and reduce the sampling rate of these nodes. For example, if the current acquisition task has high requirements for fine hand movements and head posture, but lower requirements for the movement changes of certain limb parts, the acquisition nodes for those limb parts can be determined as non-critical acquisition nodes, and their sampling rates reduced. Therefore, by reducing the sampling rate of non-critical acquisition nodes, the number of data acquisitions, data transmission volume, and the amount of data that the computing device needs to process are reduced, thereby reducing the power consumption of both non-critical acquisition nodes and the computing device.

[0075] It should be noted that critical and non-critical acquisition nodes can be determined based on the acquisition task, target action type, or target acquisition mode, and are not limited to fixed acquisition nodes. For example, in an acquisition task primarily involving hand operations, the hand acquisition node can be considered a critical acquisition node; in an acquisition task primarily involving gait movements, the leg acquisition node can be considered a critical acquisition node. Accordingly, the computing device can adjust the acquisition parameters of different acquisition nodes in high-precision mode or low-power mode according to the current acquisition task.

[0076] When the target acquisition mode is normal mode, the computing device can control multiple acquisition nodes to acquire multi-source data according to corresponding preset sampling rates. The preset sampling rate can be a regular sampling rate configured for each acquisition node during device initialization or parameter configuration. In normal mode, the sampling rate boosting process for critical acquisition nodes in high-precision mode can be omitted, and the sampling rate reduction process for non-critical acquisition nodes in low-power mode can also be omitted, allowing each acquisition node to perform data acquisition according to regular sampling parameters.

[0077] For example, in a typical full-body motion capture scenario, the wearer can select the daily mode. The computing device reads the preset sampling rates for each of the capture nodes (head, torso, limbs, hands, and fingers) from the capture parameter configuration corresponding to the daily mode, and sends the corresponding capture parameters to each capture node. Each capture node responds to a unified capture trigger signal and begins capturing data according to its own preset sampling rate, enabling the computing device to obtain multi-source data that meets the needs of conventional motion capture.

[0078] By configuring different acquisition parameters for daily, high-precision, and low-power modes, the computing device can adjust the sampling rate and acquisition channels of each acquisition node according to the different requirements of the acquisition task for motion accuracy, temporal resolution, and device power consumption. On the one hand, the number of samples of key body parts and environmental information can be increased in high-precision mode. On the other hand, the data acquisition and transmission burden of non-critical acquisition nodes can be reduced in low-power mode, thereby adapting between motion capture data acquisition accuracy, processing load, and wearable device battery life.

[0079] Step 103: Perform fusion processing on the multi-source data to obtain motion capture acquisition results, and verify the motion capture acquisition results to obtain corresponding data validity judgment results.

[0080] After receiving multi-source data, the computing device can determine the corresponding data source based on the node identifier carried by each data point, and perform time alignment on the data returned by different acquisition nodes based on the sampling time information. The computing device can also perform spatial coordinate transformation, human kinematics calculation, and multi-source fusion on the time-aligned multi-source data to obtain motion capture results. These motion capture results may include full-body skeletal motion capture data carrying global spatial pose and three-dimensional environmental perception data.

[0081] Alternatively, the computing device can also monitor the data transmission rate and buffer occupancy of each transmission channel. When data congestion occurs, the data packaging strategy is dynamically adjusted to prioritize the transmission and processing of high-priority data, thereby reducing the probability of data loss.

[0082] The computing device can verify the validity of motion capture results and the corresponding multi-source data. For example, it can determine the validity of the current frame data based on the sampling time deviation between different acquisition nodes, the number of valid nodes participating in the acquisition of the current frame data, whether the human joint movements conform to human motion constraints, and whether the environmental image meets the visual positioning requirements.

[0083] When all checks pass, the corresponding data frame can be marked as a valid frame. When any check fails, the corresponding data frame can be marked as an invalid frame, and the reason for the failure and the corresponding acquisition node are recorded. The computing device can obtain a data validity determination result based on the valid or invalid marking of the data frame, the reason for the failure, and the corresponding acquisition node. The data validity determination result can be used to indicate the data validity status of a single data frame or a continuous acquisition segment, and provide a basis for generating subsequent acquisition feedback information.

[0084] As an optional embodiment, step 103 above, which involves fusing the multi-source data to obtain the motion capture acquisition result, includes: receiving multi-source data sent by the main acquisition node and multi-source data sent by multiple slave acquisition nodes through physically isolated wired transmission links; parsing and verifying the integrity of each data frame in the multi-source data, discarding data frames that fail verification and recording lost frame information; reordering the data frames of each channel according to the local timestamp of each data frame, and remapping the local timestamps of each acquisition node to a unified reference time axis; filtering out noise from the multi-source data after reordering and timestamp remapping, and reconstructing it into data to be processed according to time slices; and performing human kinematics calculation and multi-source fusion on the data to be processed to obtain the motion capture acquisition result.

[0085] Specifically, the master acquisition node and multiple slave acquisition nodes can send multi-source data to the computing device via physically isolated wired transmission links. Inertial motion information acquired by the multiple slave acquisition nodes can first be aggregated to the waist backbone interface via a flexible bus within the motion capture suit, and then transmitted to the computing device via a shielded backbone cable. Environmental images, environmental depth information, and head inertial information acquired by the master acquisition node can be directly transmitted to the computing device via independent shielded cables. This allows for the separate transmission of large-volume environmental information and high-frequency inertial motion information, reducing bandwidth contention and signal crosstalk between different data types.

[0086] After receiving data frames from each acquisition node, the computing device can parse the header, node identifier, local timestamp, payload, and checksum of the data frame according to a preset communication protocol. It can then check whether any errors occurred during transmission based on the checksum. For data frames that fail the checksum, the computing device can discard the data frame and record the corresponding node identifier, local timestamp, and reason for loss as frame loss information. For data frames that pass the checksum, the device can retain their payload and proceed to the subsequent frame reordering process.

[0087] Because the data transmission paths, data volumes, and transmission delays between the master acquisition node and multiple slave acquisition nodes may differ, data frames within the same data channel may be out of order. The computing device can reorder the data frames in each data channel according to the local timestamp carried by each data frame to obtain a temporally continuous sequence of data frames. Subsequently, the computing device can remap the local timestamps of each acquisition node to a unified reference time axis corresponding to the computing device, based on the clock calibration parameters corresponding to each acquisition node, thereby establishing a unified time relationship for the data returned by different acquisition nodes.

[0088] Furthermore, the computing device can perform preliminary noise filtering on the multi-source data that has undergone frame ordering and timestamp remapping. For example, a sliding window filtering method can be used to identify and remove abnormal data points in the inertial motion information that clearly exceed the physically reasonable range. It can also perform invalid frame detection and preliminary denoising on environmental images, filtering out blurry, missing, or unusable data frames for subsequent visual positioning. After noise filtering, the computing device can divide the data into multiple fixed time slices according to a unified reference time axis, and organize the environmental images, environmental depth information, head inertial information, and inertial motion information of different body parts within the same time slice into a set of data to be processed.

[0089] For example, for any target time slice, the computing device can select environmental images, environmental depth information, and head inertial information corresponding to that target time slice from the data returned by the main acquisition node, based on the timestamp mapped to a unified reference time axis. It can also select inertial motion information of the torso, limbs, hands, and fingers corresponding to that target time slice from multiple data returned from acquisition nodes. The computing device organizes this information according to node identifiers and data types to form the data to be processed corresponding to that target time slice, and then passes this data to be processed to the subsequent fusion processing.

[0090] Through the above processing, different types of multi-source data can be stably transmitted using physically isolated wired transmission links. Integrity verification, frame order reordering, timestamp remapping, and noise filtering reduce the impact of transmission errors, out-of-order data, node clock deviations, and abnormal data points on subsequent calculations, ensuring that the data entering the fusion process has a clear data source, continuous data frame order, and a unified time reference.

[0091] Further optionally, in the above embodiments, performing human kinematics calculation and multi-source fusion on the data to be calculated to obtain the motion capture acquisition result includes: according to pre-stored calibration parameters, converting the inertial motion information in the data to be calculated from the sensor coordinate system to the local coordinate system of the corresponding skeleton, converting the environmental image and the environmental depth information from the acquisition coordinate system to the coordinate system of the head-mounted wearable device, and mapping the converted inertial motion information, environmental image, and environmental depth information to the global coordinate system; performing kinematic calculation on the inertial motion information based on the human skeletal kinematic chain to obtain the relative pose of each skeletal node; performing visual positioning based on the environmental image to obtain the global pose of the main acquisition node; using the global pose of the main acquisition node as a constraint, correcting the cumulative drift corresponding to the relative pose of each skeletal node to obtain whole-body skeletal motion data; and performing multi-source fusion on the whole-body skeletal motion data, the environmental image, and the environmental depth information based on human motion constraints and spatial observation constraints to obtain whole-body skeletal motion capture data carrying global spatial pose and environmental three-dimensional perception data, as the motion capture acquisition result.

[0092] Specifically, the computing device can pre-store calibration parameters corresponding to multiple acquisition nodes, the visual acquisition component in the main acquisition node, and the depth sensing component. These calibration parameters can include intrinsic sensor parameters and extrinsic installation parameters between different acquisition components. The intrinsic sensor parameters can be used to correct image distortion of the environmental image and the zero bias of the inertial measurement unit, while the extrinsic installation parameters can be used to characterize the transformation relationships between the sensor coordinate system of each acquisition node, the local coordinate system of the corresponding skeleton, the coordinate system of the main acquisition node, and the global coordinate system.

[0093] The computing device can transform inertial motion information from the sensor coordinate system of the corresponding acquisition node to the local coordinate system of the corresponding skeleton based on the installation extrinsic parameters of each acquisition node. It can also transform environmental perception data corresponding to environmental images and depth information from the acquisition coordinate system to the coordinate system of the head-mounted wearable device based on the calibration parameters of the visual acquisition component and depth perception component in the main acquisition node. Subsequently, the computing device can further map the inertial motion information in the skeleton's local coordinate system and the environmental perception data in the head-mounted wearable device's coordinate system to the global coordinate system, placing human motion information and environmental information under a unified spatial reference.

[0094] After completing the spatial coordinate transformation, the computing device can perform forward kinematic calculations on the inertial motion information corresponding to each body part based on a preset human skeletal kinematic chain. For example, using the trunk root node as a reference, along the kinematic chains from the trunk to the upper limbs, from the upper limbs to the hands, and from the trunk to the lower limbs, from the lower limbs to the feet, the position and rotation state of each skeletal node relative to the previous level skeletal node can be determined sequentially. For hand acquisition nodes, the bending angle and spatial position of each finger joint can be determined based on the kinematic chains corresponding to the palm and multiple finger joints to form the relative pose of each skeletal node.

[0095] The computing device can also perform visual localization on environmental images. For example, environmental features can be extracted and tracked from continuous environmental images, and combined with environmental depth information to determine the position and orientation of the main acquisition node in the global coordinate system, thus obtaining the global pose of the main acquisition node. Since the main acquisition node moves with the wearer's head, its global pose can provide global spatial constraints for the whole-body skeletal motion data. The computing device can use this global pose to correct the pose drift accumulated during the inertial motion information processing, converting the relative poses of each skeletal node into whole-body skeletal motion data in the global coordinate system.

[0096] Furthermore, the computing device can construct multi-source fusion relationships that include human motion constraints and spatial observation constraints. Human motion constraints can define the connection relationships between adjacent bones, the range of motion of joints, and the relationships within the human kinetic chain. Spatial observation constraints can define the spatial positional relationship between the human body and its environment based on environmental images, environmental depth information, and the global pose of the master acquisition node. The computing device can combine the data validity or confidence levels of inertial motion information, environmental images, and environmental depth information to determine the corresponding data fusion weights, and thereby solve for the human pose and 3D environmental perception results in the global coordinate system.

[0097] In one specific implementation, when the visual features in the environmental image are sufficient and the visual positioning is stable, the computing device can increase the fusion weight corresponding to the environmental image, thereby improving the global pose accuracy of the whole-body skeletal motion data using the visual positioning results. When the environmental image is occluded, lacks sufficient visual features, or has reduced tracking confidence, the computing device can decrease the fusion weight corresponding to the environmental image and increase the fusion weight corresponding to the inertial motion information to maintain the continuity of the motion data in the time dimension. It should be noted that the specific method for determining the fusion weight can be based on the validity judgment results of each data source, tracking confidence, or preset fusion parameter settings; this application embodiment does not limit this.

[0098] For example, during a wearer's hand movements, slave acquisition nodes located on the palm and fingers can provide inertial motion information of the palm and finger joints, while the master acquisition node on the head-mounted device can provide environmental images, environmental depth information, and head inertial information. The computing device first reconstructs the relative motion of each finger joint based on the human skeletal kinematic chain, then determines the global pose of the master acquisition node based on the environmental image, and uses the environmental depth information to establish the spatial relationship between human movements and the environment, ultimately obtaining full-body skeletal motion capture data carrying global spatial pose and 3D environmental perception data.

[0099] Through the above processing, the data returned by different acquisition nodes can be unified to the same spatial reference, and the motion correlation between different body parts can be established by using human kinematics. At the same time, the cumulative drift of inertial calculation is constrained by visual positioning results, and spatial observation constraints are supplemented by environmental depth information, so that the obtained motion capture acquisition results take into account the dynamic continuity of inertial measurement, the global position accuracy of visual positioning, and the three-dimensional spatial relationship of environmental information.

[0100] As an optional embodiment, step 103 above, verifying the motion capture acquisition results to obtain the corresponding data validity determination results, includes: performing time synchronization verification, data integrity verification, and visual validity verification on multi-source data corresponding to the same sampling time, and performing motion rationality verification on the corresponding motion capture acquisition results; marking data frames that pass all verifications in the multi-source data as valid frames; marking data frames that fail any verification in the multi-source data as invalid frames, and recording the failure reason and the corresponding acquisition node; obtaining the data validity determination result based on the data frame marking, the failure reason, and the corresponding acquisition node.

[0101] It is worth noting that in the above embodiments, the computing device can determine the multi-source data and motion capture acquisition results corresponding to the same sampling time or the same time slice as the data frame to be verified, and verify them from four dimensions: time synchronization, data integrity, motion rationality and visual effectiveness.

[0102] For time synchronization verification, the computing device can obtain the sampling time information corresponding to each acquisition node in the current data frame, determine the maximum time deviation between each sampling time, and compare the maximum time deviation with a preset synchronization error threshold. When the maximum time deviation does not exceed the preset synchronization error threshold, it can be determined that the current data frame passes the time synchronization verification; when the maximum time deviation exceeds the preset synchronization error threshold, it can be determined that the time synchronization of the current data frame fails, and the acquisition node that caused the time deviation is recorded.

[0103] For data integrity verification, the computing device can count the number of valid acquisition nodes participating in the acquisition of the current data frame based on the node identifiers contained in the current data frame, and compare the number of valid acquisition nodes and node identifiers with the node configuration corresponding to the current acquisition task. If the current data frame contains data from all acquisition nodes required by the current acquisition task, it can be determined that the current data frame passes the data integrity verification; if data from the head, torso, hands, or other key acquisition nodes is missing, it can be determined that the current data frame is incomplete, and the missing acquisition nodes and corresponding body parts are recorded.

[0104] For motion rationality verification, the computing device can determine the motion angle, velocity, and / or acceleration of the corresponding joints based on the pose of each skeletal node in the motion capture results, and determine whether they meet the preset human kinematic constraints. For example, it can determine whether the joint motion angle exceeds the physiological range of motion of the corresponding joint, or whether there are abnormal jumps in motion velocity or acceleration between adjacent data frames that do not conform to the laws of human motion. If there are no cases of exceeding the physiological range of motion or abnormal jumps, then the corresponding motion capture results can be determined to have passed the motion rationality verification.

[0105] For visual validity verification, the computing device can determine the number of visual feature points and tracking confidence in the current environmental image, and judge whether it meets the visual positioning requirements. If the number of visual feature points reaches the preset number and the tracking confidence reaches the preset confidence, the current data frame can be determined to have passed the visual validity verification. If the number of visual feature points is insufficient or the tracking confidence is reduced due to environmental occlusion, insufficient lighting, or insufficient environmental texture, the current data frame can be determined to have failed the visual validity verification, and the corresponding failure reason is recorded.

[0106] When a data frame passes all the above checks, the computing device can mark it as a valid frame. If any check fails, the computing device can mark it as an invalid frame and associate the failed check type, the specific reason for the failure, and the corresponding acquisition node with the invalid frame. For example, if the right-hand acquisition node does not return data, the current data frame can be marked as invalid, and "Data integrity check failed" and "Right-hand acquisition node data missing" can be recorded. When the number of visual feature points in the environmental image is insufficient, "Visual validity check failed" and "Insufficient visual features" can be recorded.

[0107] The computing device can generate a data validity determination result for the current data frame based on the valid or invalid flag, the cause of failure, and the corresponding acquisition node. The data validity determination result can be transmitted to the acquisition feedback process along with the corresponding motion capture acquisition result, and can also be associated with the corresponding data frame as validity metadata for subsequent storage, output, or filtering of valid data segments.

[0108] By verifying data frames from dimensions such as time synchronization, data integrity, motion rationality, and visual effectiveness, it is possible to identify issues such as sampling time deviation between nodes, missing data at key nodes, abnormal jumps in human movements, and insufficient visual positioning conditions. The cause of failure can be located to the corresponding verification dimension and acquisition node, enabling the wearer or subsequent processing to know the specific reason for the invalid data and reducing the situation where the entire data segment is found to be unusable only after the acquisition is completed.

[0109] Optionally, in the above embodiments, after verifying the motion capture acquisition results to obtain the corresponding data validity determination results, the proportion of valid frames, the length of consecutive invalid frames, and the anomaly frequency of each acquisition node in the multi-source data can also be statistically analyzed. Then, based on the proportion of valid frames, the length of consecutive invalid frames, and the anomaly frequency, the data validity status of the corresponding acquisition segment is determined. If the length of consecutive invalid frames reaches a preset length, the corresponding acquisition segment is marked as a failed acquisition segment, and abnormal handling information is generated to prompt the wearer to adjust their movements, check the environment, or inspect the acquisition nodes.

[0110] For example, after completing the validity verification of a single frame of data, the computing device can set a sliding statistical window according to a unified reference time axis, and select the most recent preset number of consecutive data frames for statistical analysis within the sliding statistical window. Whenever a new data validity determination result is generated, the computing device can add the new data frame to the sliding statistical window and remove historical data frames that exceed the sliding statistical window, thereby continuously updating the overall data quality of the current acquisition process.

[0111] The computing device can count the number of valid frames, the number of invalid frames, and the length of consecutive invalid frames based on the valid or invalid flags of each data frame within the sliding statistical window. Furthermore, based on the acquisition nodes associated with each invalid frame, it can count the anomaly frequency of each acquisition node within the sliding statistical window. The percentage of valid frames can be determined based on the relationship between the number of valid frames and the total number of data frames within the sliding statistical window; the length of consecutive invalid frames represents the number of consecutive data frames marked as invalid; and the anomaly frequency represents the number of times the same acquisition node is recorded as an anomaly node in multiple invalid frames.

[0112] The computing device can compare the percentage of valid frames, the length of consecutive invalid frames, and the frequency of anomalies at each acquisition node with corresponding preset judgment conditions to determine the validity status of the target acquisition segment covered by the sliding statistical window. For example, when the percentage of valid frames meets the corresponding requirements, the length of consecutive invalid frames does not reach the preset length, and there are no acquisition nodes with excessively high anomaly frequencies, the target acquisition segment can be marked as a valid acquisition segment. When the percentage of valid frames is low or a certain acquisition node experiences consecutive anomalies, the target acquisition segment can be marked as an abnormal acquisition segment. When the length of consecutive invalid frames reaches the preset length, the target acquisition segment can be marked as a failed acquisition segment.

[0113] When a target acquisition segment is marked as an abnormal or failed acquisition segment, the computing device can generate abnormal handling information based on the failure cause corresponding to the invalid frame, the abnormal acquisition node, and the corresponding body part. For example, if the joint motion angle or acceleration exceeds human motion constraints, abnormal handling information can be generated to prompt the wearer to adjust their movements; if there are insufficient visual feature points or reduced tracking confidence, abnormal handling information can be generated to prompt the wearer to adjust their environment, lighting conditions, or head orientation. In cases of missing node data, continuous frame loss, or synchronization anomalies, abnormal handling information can be generated to prompt the wearer to check the corresponding acquisition node and its connecting lines.

[0114] For example, if the data acquisition node located on the wearer's right hand fails to return data for multiple consecutive data frames within the sliding statistics window, causing all corresponding data frames to fail data integrity verification and be marked as invalid frames, the computing device can mark the current acquisition segment as a failed acquisition segment when the length of consecutive invalid frames reaches a preset length. The computing device can also generate an anomaly handling message, "Right-hand acquisition node data is continuously missing; please check node connections," based on the node identifier and node topology of the right-hand acquisition node, and send this anomaly handling message to the wearable device for display.

[0115] For example, when data from multiple acquisition nodes are returned normally, but the number of visual feature points in the environmental image is continuously insufficient due to occlusion, the computing device can determine that multiple consecutive data frames have failed the visual validity check, mark the corresponding acquisition segment as an abnormal acquisition segment or a failed acquisition segment, and generate abnormal handling information to prompt the wearer to check for environmental occlusion or adjust the head perspective.

[0116] Furthermore, the computing device can append the data validity determination result of a single frame, the data validity status of the acquisition segment, and the anomaly handling information as validity metadata to the corresponding data frame or motion capture acquisition result, and output and store them together with the motion capture acquisition result. Subsequent data processing can automatically filter valid data segments based on the validity metadata, or locate the time interval and body part that needs to be re-acquired based on the cause of failure.

[0117] By statistically analyzing the percentage of valid frames, the length of consecutive invalid frames, and the frequency of anomalies at each acquisition node based on single-frame verification, it is possible to avoid directly determining the failure of an entire data segment based on a single, occasional anomaly, and to identify persistent and concentrated data quality issues. Simultaneously, by providing feedback on the acquisition segment status, anomaly causes, and anomaly nodes to the wearer, the wearer can promptly adjust their actions, investigate the environment, or check the equipment during the acquisition process, reducing the continuous accumulation of invalid data and improving the usability of the acquired data and the efficiency of subsequent filtering.

[0118] Step 104: During the data acquisition process, in response to the input data acquisition operation interaction, the data acquisition control command corresponding to the data acquisition operation interaction is executed, and data acquisition feedback information is generated based on the execution result of the data acquisition control command, the operating status of the multiple data acquisition nodes and the data validity determination result. The data acquisition feedback information is then sent to the wearable device, so that the wearable device can provide feedback on the motion capture data acquisition status to the wearer.

[0119] The data acquisition and operation interactive operations may include at least one of the following: display brightness adjustment operation, preview channel switching operation, acquisition parameter adjustment operation, keyframe marking operation, and acquisition start / stop operation. The computing device can identify the operation type corresponding to the data acquisition and operation interactive operation, generate corresponding acquisition control commands, and control the wearable device or at least one acquisition node to execute the operation indicated by the acquisition control commands.

[0120] For example, in response to a display brightness adjustment operation, the computing device can adjust the display brightness of the wearable device; in response to a preview channel switching operation, it can switch between preview channels such as environmental image, environmental depth information, and the running status of the acquisition node; in response to an acquisition parameter adjustment operation, it can adjust the sampling rate or other acquisition parameters of at least one acquisition node; in response to a keyframe marking operation, it can add keyframe markers to the multi-source data or motion capture acquisition results corresponding to the current sampling time.

[0121] The computing device can generate acquisition feedback information by comprehensively considering the execution results of acquisition control commands, the operating status of each acquisition node, and the data validity determination results. This feedback information may include at least one of the following: whether the acquisition control command was executed successfully, whether each acquisition node is operating normally, whether the current data frame or acquisition segment is valid, the cause of the anomaly, the abnormal acquisition node, and anomaly handling information. The computing device can send the acquisition feedback information to a wearable device, which can display the corresponding content through a heads-up display and output corresponding tactile feedback through a vibration component based on the type of interactive operation.

[0122] Optionally, after the calculation and verification are completed, the computing device can generate two outputs based on the data corresponding to the same sampling time: one is wearable device echo data, which is used to display the acquisition status in the head-up display component; the other is external standardized data, including full-body skeletal motion capture data, panoramic video data, depth point cloud data and validity metadata, which is output through the external interface, and the time base of the two data is consistent.

[0123] As an optional embodiment, the data acquisition and operation interaction includes at least one of the following: display brightness adjustment, preview channel switching, acquisition parameter adjustment, and keyframe marking. Furthermore, in step 104 above, executing the acquisition control command corresponding to the data acquisition and operation interaction includes: adjusting the display brightness of the wearable device according to the acquisition control command corresponding to the display brightness adjustment operation; or switching the data preview channel displayed by the wearable device according to the acquisition control command corresponding to the preview channel switching operation; or adjusting the acquisition parameters of at least one acquisition node according to the acquisition control command corresponding to the acquisition parameter adjustment operation; or adding keyframe markers to the multi-source data or motion capture acquisition results corresponding to the current sampling time according to the acquisition control command corresponding to the keyframe marking operation.

[0124] Specifically, after data acquisition is initiated, the computing device can continuously receive acquisition operation interactions sent by the wearable device, identify the operation type of the acquisition operation interaction, and determine the acquisition control command corresponding to the acquisition operation based on the preset correspondence between operation types and acquisition control commands. The acquisition control command can be used to control the display status of the wearable device, the acquisition parameters of multiple acquisition nodes, or the marking status of multi-source data.

[0125] In one specific embodiment, the wearable device may be equipped with a damped knob. The wearer can adjust the brightness of the head-up display, switch preview channels, or adjust acquisition parameters by rotating the damped knob, and can also input keyframe markers by short-pressing the damped knob. The computing device can identify the corresponding acquisition operation based on the rotation direction, setting change, or pressing type of the damped knob, and execute the acquisition control command corresponding to the identified acquisition operation.

[0126] When the interactive operation is a display brightness adjustment operation, the computing device can determine the target display brightness of the head-up display (HUD) component based on the rotation direction and setting of the damping knob, and send a brightness adjustment command to the wearable device. In response to the brightness adjustment command, the wearable device adjusts the current display brightness of the HUD component to the target display brightness. The computing device can also obtain the brightness adjustment result returned by the wearable device and generate feedback content to display the current brightness setting based on the brightness adjustment result.

[0127] When the interactive operation for data acquisition is a preview channel switching operation, the computing device can determine the target preview channel from multiple candidate preview channels and send the data corresponding to the target preview channel to the wearable device. Candidate preview channels may include, for example, a panoramic visual preview channel, an environmental depth information preview channel, and a data acquisition node status preview channel. The panoramic visual preview channel displays the environmental image acquired by the main acquisition node, the environmental depth information preview channel displays environmental depth information or the corresponding depth point cloud, and the data acquisition node status preview channel displays the connection status, synchronization status, sampling rate, or abnormal status of each acquisition node. The wearer can cycle through these candidate preview channels by repeatedly inputting the preview channel switching operation.

[0128] When the data acquisition interaction operation is a data acquisition parameter adjustment operation, the computing device can determine the target acquisition node and target acquisition parameters based on the data acquisition parameter adjustment operation, and send a parameter adjustment command to the target acquisition node. The target acquisition parameters may include at least one of sampling rate, acquisition channel, or acquisition accuracy. After receiving the parameter adjustment command, the target acquisition node can continue to acquire corresponding action information or environmental information according to the adjusted acquisition parameters. The computing device can determine whether the acquisition parameters have been successfully adjusted based on the data frame rate, sampling status, and data channel status subsequently returned by the target acquisition node, and use the parameter adjustment result as the execution result of the acquisition control command.

[0129] For example, when it's necessary to improve the ability to capture fine hand movements during data acquisition, the wearer can adjust the acquisition parameters by inputting them into the wearable device. Based on this input, the computing device identifies the palm and finger acquisition nodes as target acquisition nodes and increases their sampling rate. After adjustment, the palm and finger acquisition nodes acquire corresponding inertial motion information at the increased sampling rate, while other acquisition nodes continue to acquire data using their original parameters. Thus, the acquisition parameters of the corresponding acquisition nodes can be adjusted according to the needs of motion acquisition without interrupting the current acquisition process.

[0130] When the data acquisition interaction is a keyframe marking operation, the computing device can determine the sampling time corresponding to the received keyframe marking operation and add keyframe markers to the multi-source data and / or motion capture acquisition results corresponding to that sampling time. The keyframe markers can be associated and saved with the corresponding unified reference timestamp, node identifier, and motion capture acquisition results, and are used to locate the operation time of interest to the wearer during subsequent data retrieval, motion analysis, or model training. The computing device can also count the number of keyframes marked during the current acquisition process and send a marking completion notification to the wearable device after keyframe marking is completed.

[0131] For example, when a wearer encounters a critical work point during outdoor exploration or robotic data collection, they can briefly press a damped knob. The computing device recognizes this brief press as a keyframe marking operation, adds a keyframe marker to the multi-source data and motion capture results corresponding to the current sampling time, and sends a marking completion message to the wearable device. The wearable device can display the keyframe marking result through a heads-up display and output vibration feedback corresponding to the brief press operation, allowing the wearer to complete the keyframe marking without stopping their current action or operating a host computer.

[0132] In another embodiment, the data acquisition and interactive operations can also be acquired through a touch-sensitive area or a vibration sensor. For example, the wearer can input a swipe operation in the touch-sensitive area to adjust the display brightness or acquisition parameters, and can input a click operation to complete keyframe marking; or, the wearer can tap the wearable device once to complete keyframe marking, and switch between preview channels such as panoramic vision, environmental depth information, and acquisition node status by tapping the wearable device twice in succession.

[0133] By establishing a correspondence between interactive operations and control commands during data acquisition, the wearer can adjust display brightness, switch preview channels, adjust acquisition parameters, or mark key data moments without stopping the current motion capture. This reduces the wearer's reliance on external display devices or host computers, ensures seamless integration of acquisition control operations with the current motion capture data acquisition process, and facilitates rapid location of required data based on keyframe markers.

[0134] As an optional embodiment, in step 104 above, generating acquisition feedback information based on the execution result of the acquisition control command, the operating status of the plurality of acquisition nodes, and the data validity determination result includes: generating an acquisition dimension overview based on the operating status of the plurality of acquisition nodes to display the access status of the acquisition nodes and the operating status of the data channel; generating data validity content based on the data validity determination result to display the data validity status, failure cause, and at least one of the corresponding acquisition nodes; generating acquisition interaction guidance content based on the acquisition control command and the corresponding execution result to display operation guidance and acquisition operation results; generating display feedback information based on the acquisition dimension overview content, the data validity content, and the acquisition interaction guidance content, and controlling the head-up display component of the wearable device to display the display feedback information; generating haptic feedback control information based on the operation type of the input interactive operation, and controlling the vibration component of the wearable device to output the corresponding vibration feedback.

[0135] Specifically, the computing device can continuously acquire the operating status of multiple acquisition nodes, the execution results of acquisition control commands, and the data validity determination results during the acquisition process, and generate acquisition dimension overview content, data validity content, and acquisition interaction guidance content based on the above information. In one embodiment, the computing device can render the above content as lightweight display feedback information and send it to the head-up display component of the wearable device via a wired transmission link.

[0136] The data acquisition overview provides feedback to the wearer regarding the currently participating acquisition nodes and the operational status of each data channel. The computing device can generate node topology display information corresponding to the human body contour based on the node identifiers, locations, and operational status of multiple acquisition nodes, and generate a data channel list based on the data types acquired by each node. The node topology display information can identify acquisition nodes at locations such as the head, torso, limbs, and hands, and can be set to display corresponding normal, abnormal, or failed states. The data channel list can display the operational status of data channels such as panoramic vision, environmental depth information, torso motion capture, upper limb motion capture, hand motion capture, and lower limb motion capture, and can also display the real-time frame rate or sampling rate of the corresponding data channel.

[0137] Data validity information is used to provide feedback to the wearer on the data quality of the current data frame or acquisition segment. The computing device can generate the valid or invalid status of the current data frame, the percentage of valid frames in a preset number of consecutive data frames, the length of consecutive invalid frames, and the pass / fail status of verification items such as time synchronization, data integrity, motion rationality, and visual validity, based on the data validity determination results. When any verification item fails, the data validity information may also include the specific reason for the failure, the abnormal value, and the corresponding acquisition node or body part.

[0138] For example, when the right-hand acquisition node fails to return data consecutively, the computing device can determine that the right-hand acquisition node has missing data based on the data integrity verification result, and generate data validity content to display "Right-hand acquisition node data abnormality". The wearable device's head-up display (HUD) can display an abnormality marker at the right-hand position in the node topology, and simultaneously display abnormality handling information to prompt the wearer to check the right-hand acquisition node and its connecting lines. As another example, when the number of visual feature points in the environmental image is insufficient, the HUD can display that the visual positioning effectiveness is substandard, and prompt the wearer to check environmental occlusion, lighting conditions, or head orientation.

[0139] The data acquisition interaction guide is used to provide feedback to the wearer regarding the operation methods, execution progress, and execution results corresponding to data acquisition control commands. When no data acquisition control operation is performed, the guide displays the current target acquisition mode, as well as the trigger methods for operations such as brightness adjustment, preview channel switching, data acquisition parameter adjustment, and keyframe marking. During the execution of acquisition control commands, the guide displays the corresponding execution progress; upon completion, it displays the result of success or failure. If a data acquisition control command fails, corresponding handling instructions can be generated based on the reason for the failure.

[0140] For example, after the wearer inputs a preview channel switching operation, the computing device can write the target preview channel and the switching result into the acquisition interaction guidance content. If the target preview channel is the environmental depth information preview channel, the head-up display can show the environmental depth information or depth point cloud preview, and simultaneously display operation results such as "Switched to depth preview channel". After the wearer inputs a keyframe marking operation, the acquisition interaction guidance content can display a keyframe marking completion prompt and the current cumulative number of marks.

[0141] In one specific embodiment, the interface displayed by the head-up display (HUD) component may include a data acquisition dimension overview area, a data validity determination area, and a data acquisition interaction guidance area. The data acquisition dimension overview area displays the node topology and data channel operation status; the data validity determination area displays the validity status of the current data frame, the validity status of data in consecutive acquisition segments, and the corresponding verification results; the data acquisition interaction guidance area displays the current working mode, operation method, operation progress, execution results of acquisition control commands, and anomaly handling information. It should be noted that the above content can also be displayed in the same display area at different times, or selectively displayed according to the current acquisition status. This application embodiment does not limit the specific interface layout.

[0142] In addition to displaying feedback information, the computing device can also generate haptic feedback control information based on the type of interactive operation and control the vibration components in the wearable device to output corresponding vibration feedback. Different interactive operations can correspond to different vibration rhythms, vibration intensities, or vibration durations. For example, a damped knob can output a short, stepped vibration for each rotation. When a short press operation is detected and the corresponding acquisition control command is successfully executed, a strong confirmation vibration can be output. When a long press operation is detected, a continuously increasing vibration can be output during the long press, and a confirmation vibration can be output after a preset duration.

[0143] Haptic feedback can be used to indicate to the computing device that an interaction has been received, or to indicate the result of an operation after the acquisition control command has been executed. Display feedback information can further explain the specific operation performed, the current operating status of the acquisition node, and the validity of the data. The computing device can combine haptic and display feedback information into acquisition feedback information, allowing the wearable device to output corresponding feedback through vibration components and a heads-up display component.

[0144] For example, when the wearer rotates the damped knob to switch preview channels, the wearable device outputs a short, stepped vibration for each rotation to indicate that a change in channel has been detected. After the computing device completes the preview channel switch, the head-up display shows the new preview channel and the switch result. If the new acquisition node status preview channel shows that all acquisition nodes are operating normally, but the data validity check of the right-hand acquisition node fails, the head-up display can simultaneously show the operating status of the right-hand acquisition node in the acquisition dimension overview area and display the reason for the data anomaly in the data validity judgment area.

[0145] By generating acquisition feedback information by combining the execution results of acquisition control commands, the operating status of multiple acquisition nodes, and the data validity determination results, the wearer can simultaneously know whether the operation was executed, whether the acquisition node is functioning normally, and whether the currently acquired data is valid. Through the combination of a heads-up display and vibration feedback, the wearer can complete operation confirmation, status checks, and anomaly troubleshooting during acquisition without needing to look directly at the interactive components or frequently check external display devices or the host computer. This improves interaction efficiency in the wearable state and allows acquisition anomalies to be handled promptly.

[0146] Further optionally, the wearable device is provided with a damped knob; the acquisition method of the wake-up interaction operation, the acquisition start interaction operation, or the acquisition run interaction operation can also respond to the rotation operation of the damped knob, determine the target acquisition option from multiple candidate acquisition options according to the change of the damped knob's position; respond to the pressing operation of the damped knob, identify the pressing type of the pressing operation; determine the corresponding interaction operation according to at least one of the target acquisition option and the pressing type, wherein the pressing type includes at least one of short press, long press, and double click.

[0147] Specifically, the damping knob can be located on the lower side of the head-mounted wearable device, near the wearer's dominant hand, and may include a rotary encoder and a pressure detection structure. The rotary encoder is used to detect the rotation direction and gear changes of the damping knob, while the pressure detection structure is used to detect the timing, duration, and time interval between adjacent pressure operations. The computing device or wearable device can distinguish between rotation, short press, long press, and double-click operations based on the detection results, and determine the corresponding interactive operation in conjunction with the current data acquisition stage. Optionally, a double-click operation can also trigger system calibration. The computing device can send calibration guidance information to the wearable device, and the head-up display sequentially displays guidance steps such as posture maintenance, head rotation, and limb movement, along with the calibration progress; after the wearer completes the guidance, the computing device completes sensor zero-bias calibration and spatial coordinate alignment.

[0148] When a rotation of the damping knob is detected, the system can switch between multiple candidate acquisition options based on the direction and level of rotation. For example, rotating the knob in a first direction allows the candidate acquisition options to be selected sequentially in a preset order; rotating it in a second direction (opposite to the first direction) allows the selection in reverse order. Candidate acquisition options may include acquisition modes such as normal mode, high-precision mode, and low-power mode, as well as brightness levels, preview channels, and acquisition parameter levels on the head-up display (HUD). The computing device can determine the currently selected candidate acquisition option as the target acquisition option and display it on the HUD.

[0149] When a press operation on the damped knob is detected, the press type can be determined based on the duration of the press operation and the time interval between adjacent press operations. For example, if the duration of the press operation is less than a preset press duration, the press operation can be identified as a short press; if the duration of the press operation reaches the preset press duration, the press operation can be identified as a long press; if two press operations are detected consecutively within a preset time interval, the two press operations can be identified as a double-click operation. The preset press duration and preset time interval can be preset according to the detection sensitivity and interaction requirements of the damped knob, and this embodiment does not limit them.

[0150] In one implementation, the corresponding interactive operation can be determined based on the current operating state of the computing device, the target acquisition option, and the press type. When the computing device is in sleep mode, a long press can be determined as a wake-up interactive operation; when the computing device is in a ready-to-acquire state and the target acquisition mode has been selected by rotation, a short press can be determined as a mode confirmation operation; when the target acquisition mode has been determined and multiple acquisition nodes meet the preset acquisition conditions, a short press can be determined as an acquisition start interactive operation; during acquisition operation, a short press can be determined as a pause acquisition operation, a resume acquisition operation, or a keyframe marking operation, a long press can be determined as an acquisition end operation, and a double-click can be determined as a calibration trigger operation or a data preview view switching operation.

[0151] For the same press type, different interactive operations can be determined based on the current target acquisition option. For example, when the target acquisition option is a candidate acquisition mode, a short press is used to confirm the currently selected candidate acquisition mode; when the target acquisition option is a preview channel, a short press is used to confirm switching to the currently selected preview channel; and when the target acquisition option is an acquisition parameter level, a short press is used to confirm using the currently selected acquisition parameter. Therefore, rotation can be used to select options, and pressing can be used to confirm options or trigger corresponding functions.

[0152] Furthermore, the wearable device can output corresponding tactile feedback based on the operation type of the damped knob, and output corresponding display feedback through the head-up display component. For example, when the damped knob is turned one position, the wearable device can output a step-by-step short vibration and update the currently selected candidate acquisition option in the head-up display component; when a short press operation is detected, a single confirmation strong vibration can be output, and the execution result of the confirmed target acquisition option or acquisition control command can be displayed; when a long press operation is detected, a gradually increasing vibration can be output as the pressing duration increases, and a confirmation vibration can be output when the pressing duration reaches the preset pressing duration to indicate that the corresponding wake-up, sleep, or end acquisition operation is about to be executed; when a double-click operation is detected, a vibration rhythm corresponding to the double-click operation can be output, and the system calibration or preview view switching result can be displayed.

[0153] For example, when the system is in sleep mode, the wearer can press and hold the damped knob to initiate a wake-up interaction. The wearable device outputs corresponding vibration feedback and displays the system wake-up result and the operating status of multiple acquisition nodes on the head-up display. Once multiple acquisition nodes are in the ready-to-acquire state, the wearer can rotate the damped knob to cycle through the candidate acquisition modes in the order of normal mode, high-precision mode, and low-power mode. Each time a mode is switched, the wearable device outputs a short, stepped vibration and simultaneously displays the current candidate acquisition mode. After switching to high-precision mode, the wearer can short-press the damped knob to confirm the high-precision mode; the computing device then determines the high-precision mode as the target acquisition mode and sends mode confirmation information to the wearable device.

[0154] Once the target acquisition mode is determined, the wearer can briefly press the damped knob again to input the acquisition start interactive operation. In response to this operation, the computing device sends synchronization references, acquisition trigger signals, and acquisition parameters corresponding to the high-precision mode to multiple acquisition nodes. Upon successful acquisition start, the wearable device outputs a single-confirmation strong vibration and displays the acquisition start result via the head-up display. During acquisition operation, the wearer can rotate the damped knob to adjust display brightness, switch preview channels, or adjust acquisition parameter levels. They can also briefly press the damped knob to mark keyframes and double-click it to trigger system calibration or switch data preview views.

[0155] When the data acquisition task is complete, the wearer can press and hold the damped knob to indicate the end of the acquisition. During the press and hold, the wearable device outputs continuously increasing vibrations; when the press duration reaches the preset duration, a confirmation vibration is output. After the computing device confirms the end of the acquisition, it controls multiple acquisition nodes to stop data acquisition, saves the corresponding multi-source data, motion capture acquisition results, and data validity determination results, and displays the summary information of the acquisition completion via the head-up display.

[0156] By combining rotation, short press, long press, and double-click operations with a single damped knob, operations such as system wake-up, mode selection, data acquisition initiation, operation adjustment, keyframe marking, and data acquisition termination can be completed with fewer external interactive components on the wearable device. The damping levels, differentiated vibration feedback, and head-up display feedback work together to allow the wearer to confirm operation inputs and results without looking directly at the damped knob. This functionality is also possible when wearing gloves or working outdoors, reducing the probability of accidental triggering and improving data acquisition control efficiency while wearing the device.

[0157] Optionally, the wearable device is provided with a touch-sensing area. The method for acquiring the target acquisition mode, the acquisition initiation interaction operation, or the acquisition operation includes: acquiring a swipe operation, a click operation, or a long press operation input on the touch-sensing area; determining an acquisition mode switching operation, a display brightness adjustment operation, or an acquisition parameter adjustment operation based on the swipe operation; determining an option confirmation operation or an acquisition start / stop operation based on the click operation; and determining an acquisition end operation based on the long press operation.

[0158] Specifically, the touch-sensing area can be located on the side of the head-mounted wearable device and may include a touch sensor for detecting touch position and changes in touch position. The wearable device can acquire touch signals input by the wearer in the touch-sensing area, and based on the duration of the touch signal and changes in touch position, identify swipe operations, tap operations, or long-press operations, and send the identified operations and corresponding operation information to a computing device. The operation information may include at least one of the following: operation type, swipe direction, swipe distance, number of taps, and touch duration.

[0159] The computing device can determine the corresponding interactive operation by combining the current interactive interface, the current acquisition stage, and the operation type. When the current interactive interface is an acquisition mode selection interface, the computing device can identify a swipe operation as an acquisition mode switching operation, switch between multiple candidate acquisition modes according to the swipe direction, and display the currently selected candidate acquisition mode on the wearable device. For example, the wearer can switch between daily mode, high-precision mode, and low-power mode sequentially by swiping up or down. After detecting a click operation, the computing device can determine the currently selected candidate acquisition mode as the target acquisition mode.

[0160] When the current interactive interface is the display settings interface, the computing device can identify the swipe operation as a display brightness adjustment operation. The computing device can determine the target brightness level based on the swipe direction and swipe distance, and send a brightness adjustment command to the wearable device to control the head-up display component to display according to the target brightness level. For example, swiping upwards can increase the display brightness, swiping downwards can decrease the display brightness, and the swipe distance or the number of preset touch areas traversed can be used to determine the adjustment amount of the brightness level.

[0161] When the current interactive interface is the data acquisition parameter setting interface, the computing device can identify the sliding operation as a data acquisition parameter adjustment operation. The computing device can determine the adjusted data acquisition parameters based on the currently selected data acquisition parameter item, the sliding direction, and the sliding distance, and then send the adjusted data acquisition parameters to the corresponding acquisition node. The data acquisition parameters may include, for example, the sampling rate or data acquisition parameter level. The computing device can also determine whether the data acquisition parameters have been successfully adjusted based on the subsequent data frame rate or sampling status returned by the acquisition node.

[0162] Click operations can be used to confirm the current option or control the start, pause, and resumption of data acquisition. For example, when the target acquisition mode has been determined and multiple acquisition nodes meet the preset acquisition conditions, the computing device can determine the click operation as an acquisition start interaction operation and send acquisition trigger signals to multiple acquisition nodes; during acquisition operation, the computing device can determine the click operation as a pause acquisition operation or a resumption acquisition operation according to the current acquisition status.

[0163] A long press can be used to end the current data acquisition. The wearable device can obtain the duration for which the touch position remains within the touch-sensing area. When the duration reaches a preset duration, the operation is recognized as a long press, and a data acquisition end interaction operation is sent to the computing device. In response to the data acquisition end interaction operation, the computing device controls multiple acquisition nodes to end data acquisition and performs data saving and acquisition completion feedback processing. To reduce the possibility of premature data acquisition termination due to ordinary clicks or short-term accidental touches, the wearable device can also output a confirmation vibration or display an end confirmation prompt when the long press duration reaches the preset duration.

[0164] For example, after multiple data acquisition nodes enter the data acquisition state, the wearer can slide up and down on the touch-sensitive area on the side of the head-mounted device to switch between daily mode, high-precision mode, and low-power mode. After switching to high-precision mode, the wearer can tap the touch-sensitive area to confirm the high-precision mode and tap again to input the data acquisition start interaction. During data acquisition, the wearer can adjust the brightness of the head-up display or adjust acquisition parameters by sliding, and pause or resume data acquisition by tapping. After the data acquisition task is completed, the wearer can long-press the touch-sensitive area; the computing device will terminate data acquisition from multiple data acquisition nodes after recognizing the long-press operation.

[0165] Furthermore, after the touch-sensing area detects a swipe, tap, or long press operation, the wearable device can output vibration feedback corresponding to the operation type and display the current option or operation result via a heads-up display. For example, when a swipe operation is detected, a stepped short vibration can be output and the currently displayed mode or parameter level can be updated; when a tap operation is detected and the corresponding command is successfully executed, a single confirmation vibration can be output; when a long press operation is detected, a continuous or gradually increasing vibration can be output, and a data acquisition completion prompt can be displayed when the termination condition is met.

[0166] By employing touch-sensitive areas to capture swipe, tap, and long-press operations, interactive functions such as mode selection, parameter adjustment, start / stop data collection, and end data collection can be achieved without the need for multiple physical buttons. The combination of touch operation with vibration feedback and head-up display feedback allows wearers to promptly confirm current options and operation results, and enables the wearable device to adapt to usage scenarios where touch operation is preferred.

[0167] Figure 3 This is a schematic diagram of a wearable motion capture data acquisition process based on tapping interaction, provided as an embodiment of this application. (Refer to...) Figure 3 When the wearable device is in system sleep mode, the wearer taps the side of the helmet once, triggering vibration feedback and displaying "system awakened" on the HUD. A full node self-check is then performed, and the node connection status is displayed on the HUD. If the node connection is normal, the wearer double-clicks the helmet repeatedly to cycle through acquisition modes such as normal, high-precision, and power-saving modes. After switching to the target acquisition mode, a single tap confirms and starts acquisition. All nodes acquire data synchronously and enter the acquisition loop. During acquisition, the wearer can tap the helmet once to mark keyframes or double-click repeatedly to switch between preview channels such as panorama, depth, and node status. To end acquisition, the wearer presses and holds the side of the helmet; the system outputs progressively stronger vibrations during the press, triggering a strong confirmation vibration upon reaching a threshold. After the wearer senses and releases the strong confirmation vibration, the system ends acquisition, saves data, and displays a summary notification on the HUD, indicating the end of the acquisition process.

[0168] Further optionally, the wearable device is equipped with a vibration sensor; the acquisition method of the wake-up interaction operation, the target acquisition mode, the acquisition start interaction operation, or the acquisition run interaction operation includes: detecting a tapping operation input to the wearable device through the vibration sensor; when the wearable device is in a sleep state, determining a single tapping operation as the wake-up interaction operation; when the running state meets the preset acquisition conditions, determining two consecutive tapping operations as an acquisition mode selection interaction operation, and determining the target acquisition mode according to the acquisition mode selection interaction operation; after determining the target acquisition mode, determining a single tapping operation as the acquisition start interaction operation; during acquisition run, determining a single tapping operation as a keyframe marking operation, and determining two consecutive tapping operations as a preview channel switching operation.

[0169] The vibration sensor can be installed inside the head-mounted wearable device to detect vibration signals generated when the wearer taps the device. The wearable device can detect the vibration signals output by the sensor and, based on the amplitude, duration, and time interval between adjacent vibration signals, identify a single tap, two consecutive taps, or a long-press tap. The long-press tap can be an operation where the wearer maintains contact with the device for a preset duration after tapping or touching it.

[0170] To reduce the impact of vibrations caused by wearer movement, data acquisition node operation, or external collisions on operation recognition, the wearable device can compare the detected vibration signal with preset tapping signal conditions. When the vibration signal meets the corresponding amplitude and duration conditions, it is determined as a valid tapping signal; when two valid tapping signals are detected within a preset time interval, they are determined as two consecutive tapping operations. The specific amplitude, duration, and time interval conditions can be set according to the structure of the wearable device and the detection sensitivity of the vibration sensor; this embodiment does not limit these settings.

[0171] The computing device can assign different interactive meanings to the same tapping operation based on the current stage of the wearable device or data acquisition process. When the wearable device is in sleep mode, a single tapping operation can be identified as a wake-up interaction operation. In response to the wake-up interaction operation, the computing device wakes up the wearable device and multiple acquisition nodes, and initiates the detection of the operating status of the multiple acquisition nodes. The wearable device can synchronously output a confirmation vibration and display the wake-up result through a heads-up display component.

[0172] When multiple acquisition nodes meet the preset acquisition conditions, two consecutive taps can be identified as an interactive acquisition mode selection operation. Each time the computing device detects two consecutive taps, it can switch between candidate acquisition modes in a preset order, cycling between daily mode, high-precision mode, and low-power mode. After each mode switch, the wearable device can display the current candidate acquisition mode and output vibration feedback corresponding to the mode switch operation.

[0173] Once the target acquisition mode is determined, a single tap operation can be identified as the acquisition initiation interaction operation. In response to this acquisition initiation interaction operation, the computing device sends synchronization references, acquisition trigger signals, and acquisition parameters to multiple acquisition nodes according to the target acquisition mode, causing the multiple acquisition nodes to initiate data acquisition. The computing device can also send the acquisition initiation result to a wearable device, which displays a "Acquisition Started" message and outputs a confirmation vibration to indicate successful initiation.

[0174] During the data acquisition process, a single tap operation can be identified as a keyframe marking operation. The computing device can determine the sampling time corresponding to the received single tap operation as the target sampling time and add a keyframe marker to the multi-source data or motion capture acquisition results corresponding to the target sampling time. Two consecutive tap operations can be identified as a preview channel switching operation, used to switch between the panoramic visual preview channel, the environmental depth information preview channel, and the acquisition node status preview channel.

[0175] For example, in outdoor exploration or industrial operations, when wearing work gloves, a wearer can wake up the head-mounted device by tapping the side once. After the head-up display shows that multiple data acquisition nodes are functioning normally, the wearer can tap twice to cycle through the candidate acquisition modes: normal mode, high-precision mode, and low-power mode. When the head-up display shows high-precision mode, the wearer can tap once to initiate data acquisition.

[0176] During data acquisition, when the wearer reaches a critical work location, they can tap the wearable device once to add a keyframe marker to the data at the corresponding sampling time. To view different acquired information, they can tap twice consecutively to switch between preview channels such as environmental image, environmental depth information, and acquisition node status on the head-up display. Throughout the entire operation, the wearer does not need to operate an external display device or host computer.

[0177] In another embodiment, after the data collection task is completed, a long press and tap operation can be designated as the end-of-collection interaction. When the wearable device detects a long press and tap operation, it can output a gradually increasing vibration while maintaining contact, and output a confirmation vibration when the contact duration reaches a preset duration. After the wearer senses the confirmation vibration, they release the contact, and the computing device terminates data collection at multiple collection nodes and saves the data.

[0178] By recognizing single taps, double taps, and long presses using vibration sensors, operations such as wake-up, mode selection, data acquisition initiation, keyframe marking, preview channel switching, and data acquisition termination can be achieved without adding physical interaction components protruding from the wearable device's outer surface. This interaction method reduces the risk of snagging on the wearable device's external structure in outdoor or industrial settings and allows wearers wearing gloves to still complete the data acquisition process.

[0179] Further optionally, in response to the input "Collection End Interaction" operation, after the data collection of the multiple collection nodes ends, the collection completion status corresponding to each collection node is determined according to the running status of the multiple collection nodes recorded during the collection period and the data validity judgment result. The collection completion status includes a normal status, an abnormal status, or a failure status. If the collection completion status of any collection node is the abnormal status or the failure status, a second abnormal collection node and its corresponding abnormal reason are determined according to the node identifier, node location, running status, and corresponding data validity judgment result of the collection node. The abnormal reason includes at least one of the following: collection node not started, collection node failure, missing collection data, or abnormal collection data. The collection completion feedback information is generated according to the data collection end status, the collection completion status corresponding to each collection node, and the second abnormal collection node and its corresponding abnormal reason. The collection completion feedback information is sent to the wearable device so that the wearable device can display the collection completion summary information to the wearer, and, if there is an abnormal collection node, display the abnormal collection node and its corresponding abnormal reason.

[0180] Specifically, the computing device can continuously record the operating status of multiple acquisition nodes during the acquisition process and associate the data validity determination results corresponding to each acquisition node with the node identifier and sampling time information. The operating status can include at least one of the following: connection status, synchronization status, pending acquisition status, acquisition start status, and failure status of the acquisition node. The data validity determination results can include a valid or invalid flag for the data frame, the cause of failure, the corresponding acquisition node, and the data validity status of the acquisition segment.

[0181] In response to the wearer's input indicating the end of data collection, the computing device can send a stop data collection command to multiple data collection nodes, ceasing the reception of new multi-source data, and summarizing the operational status and data validity determination results recorded during the collection period. The computing device can statistically analyze, based on node identifiers, whether each data collection node started normally, maintained connection and synchronization during collection, continuously returned data, and the status of valid, invalid, and abnormal frames within the data frames corresponding to that node, thereby determining the collection completion status for each data collection node.

[0182] When a data acquisition node starts normally, maintains normal operation during data acquisition, and its corresponding acquired data meets preset validity requirements, the computing device can determine the acquisition completion status of that node as normal. When a data acquisition node has started but experiences a short-term synchronization anomaly, data frame loss, data jumps, or invalid partial data frames during acquisition, its acquisition completion status can be determined as abnormal. When a data acquisition node does not start, is disconnected for an extended period, loses its synchronization signal for a long time, is marked as in a failed state, or continuously fails to return acquired data, its acquisition completion status can be determined as in a failed state.

[0183] It should be noted that the specific criteria for determining normal, abnormal, and failure states can be set according to the node type of the acquisition node and the current acquisition task. For example, the master acquisition node can determine the acquisition completion status based on the working status of the environmental image acquisition channel, the environmental depth information acquisition channel, and the head inertial information acquisition channel; the slave acquisition nodes can determine the acquisition completion status based on the startup status, synchronization status, data return status, and data validity of the corresponding inertial measurement unit.

[0184] When the acquisition completion status of any acquisition node is in an abnormal or failed state, the computing device can identify the acquisition node as the second abnormal acquisition node, and determine the corresponding abnormal cause based on the node identifier, node location, operating status and corresponding data validity judgment result of the second abnormal acquisition node.

[0185] For example, if the second abnormal acquisition node does not return to the start state during the acquisition start phase or does not return any data frames during the acquisition, the cause of the abnormality can be determined as the acquisition node not starting; if the second abnormal acquisition node is marked as in a failed state due to the long-term loss of the synchronization signal, the cause of the abnormality can be determined as the acquisition node failing; if the data frames that should be returned by the second abnormal acquisition node do not appear in the multi-source data of the corresponding time slice, the cause of the abnormality can be determined as missing acquisition data; if the data returned by the second abnormal acquisition node has continuous jumps, continuous frame drops, or the corresponding data frames fail the data validity check multiple times, the cause of the abnormality can be determined as abnormal acquisition data.

[0186] The computing device can generate acquisition completion feedback information based on the data acquisition termination status, acquisition duration, acquired data volume, number of keyframe markers, acquisition completion status of each acquisition node, second abnormal acquisition node and its corresponding abnormality reason. The acquisition completion feedback information may include at least one of the following: whether the acquisition ended normally, number of normal acquisition nodes, number of abnormal acquisition nodes, number of failed acquisition nodes, body part corresponding to the second abnormal acquisition node, abnormality reason, and corresponding data abnormality time interval.

[0187] The computing device can send data acquisition completion feedback information to the wearable device. The wearable device displays the data acquisition completion summary information via a heads-up display. When no second abnormal acquisition node exists, the data acquisition completion summary information may include information such as acquisition completion, acquisition duration, amount of data acquired, and number of keyframe markers. When a second abnormal acquisition node exists, the wearable device can also display the node location and cause of the abnormality, and provide corresponding error handling prompts.

[0188] For example, if the slave acquisition node located on the wearer's right hand fails to return data continuously during the acquisition process, the computing device can determine the right-hand position of the acquisition node based on the node topology and determine the cause of the anomaly as missing acquisition data based on the data integrity verification result. After the acquisition is completed, the head-up display can show "Acquisition completed, right-hand acquisition node data missing" and prompt the wearer to check the right-hand acquisition node and its connection lines. As another example, if the environmental image acquisition channel in the main acquisition node continuously fails to meet the visual positioning requirements, the acquisition completion feedback information can indicate that the main acquisition node has an acquisition data anomaly and prompt the wearer to check environmental occlusion or lighting conditions.

[0189] Furthermore, the computing device can save the acquisition completion status of each acquisition node, the second abnormal acquisition node and its corresponding abnormal reason as an acquisition status log, along with the original multi-source data, motion capture acquisition results and data validity judgment results. It can also associate the abnormal reason with the corresponding sampling time information to locate the time interval and body part that needs to be re-acquired.

[0190] By determining the completion status of each data collection node based on the recorded operational status and data validity assessment results after data collection, it is possible to distinguish between situations such as a data collection node not starting, a data collection node failing, missing data, and abnormal data. Providing the location and cause of the second abnormal data collection node to the wearer allows them to focus on specific body parts and abnormalities, reducing manual review of the entire data collection process and providing a basis for subsequent supplementary collection or filtering of valid data.

[0191] Optionally, the raw multi-source data and motion capture results can be stored separately and indexed separately. The writing process employs an asynchronous caching mechanism, ensuring no impact on real-time computation. After collection, a collection dataset containing node configurations, calibration parameters, raw data, motion capture results, validity metadata, and collection status logs can be generated and uploaded to the cloud for subsequent processing. When remote monitoring is required, low-bitrate preview data can also be uploaded synchronously during the collection process.

[0192] In this embodiment, the operating status of multiple acquisition nodes is detected before acquisition begins, and the target acquisition mode is determined when the operating status of each acquisition node meets the preset acquisition conditions. This reduces the number of nodes that are not connected, not synchronized, or unable to acquire data. Controlling multiple acquisition nodes to synchronously acquire multi-source data according to the target acquisition mode helps maintain the sampling time relationship between data from different sources. During acquisition, the execution results of acquisition control commands, the operating status of multiple acquisition nodes, and the data validity determination results are fed back to the wearable device. This allows the wearer to promptly know whether the acquisition operation was successful, whether each acquisition node is functioning normally, and whether the acquired data is valid. This enables timely adjustments to actions, environmental checks, or inspection of acquisition nodes, reducing invalid and duplicate acquisitions and improving the interactive efficiency and data availability of wearable motion capture data acquisition.

[0193] The above describes a wearable motion capture data acquisition method in the embodiments of this application. The wearable motion capture data acquisition device that performs the above-described wearable motion capture data acquisition method will be described below.

[0194] See Figure 4 ,like Figure 4 The diagram shows a wearable motion capture data acquisition device, which can be applied to computing devices in image recognition scenarios requiring defense against adversarial attacks. The wearable motion capture data acquisition device in this embodiment can achieve the aforementioned... Figure 2 The steps of the wearable motion capture data acquisition method executed in the corresponding embodiments are described above. The functions implemented by the wearable motion capture data acquisition device can be achieved through hardware or through hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The wearable motion capture data acquisition device may include an input / output module 401 and a processing module 402. The functional implementation of the processing module 402 and the input / output module 401 can be found in [reference missing]. Figure 2 The operations performed in the corresponding embodiments will not be described in detail here. For example, the processing module 402 can be used to control the sending, receiving, and acquiring operations of the input / output module 401.

[0195] The input / output module 401 is configured to receive wake-up interaction operations, acquisition start interaction operations and / or acquisition run interaction operations; send synchronization references and acquisition trigger signals to multiple acquisition nodes, and receive multi-source data returned by the multiple acquisition nodes; send the acquisition feedback information to the wearable device, so that the wearable device can provide feedback on the motion capture acquisition status to the wearer. The processing module 402 is configured to, in response to an input wake-up interaction, detect the operating status of the multiple acquisition nodes and determine a target acquisition mode if the operating status of the multiple acquisition nodes meets preset acquisition conditions; in response to an input acquisition start interaction, send a synchronization reference and acquisition trigger signal to the multiple acquisition nodes through the input / output module according to the target acquisition mode, so that the multiple acquisition nodes acquire the multi-source data at a unified sampling time; perform fusion processing on the multi-source data to obtain motion capture acquisition results, and verify the motion capture acquisition results to obtain corresponding data validity judgment results; during acquisition operation, in response to an input acquisition operation interaction, execute the acquisition control command corresponding to the acquisition operation interaction, and generate acquisition feedback information based on the execution result of the acquisition control command, the operating status of the multiple acquisition nodes, and the data validity judgment result.

[0196] In some implementations, the processing module 402, in response to an input wake-up interaction operation, detects the operating status of multiple acquisition nodes and determines the target acquisition mode when the operating status of the multiple acquisition nodes all meet preset acquisition conditions. Specifically, it is configured to: in response to the wake-up interaction operation, send wake-up information and node query information to the multiple acquisition nodes; receive node identifiers and initial status information returned by each acquisition node, register the multiple acquisition nodes and establish a node topology; determine the operating status of the multiple acquisition nodes based on their connection and synchronization status; if the multiple acquisition nodes are all in a connected and synchronized pending acquisition state, then determine that the operating status of the multiple acquisition nodes all meet the preset acquisition conditions; in response to an input acquisition mode selection interaction operation, determine the target acquisition mode from multiple candidate acquisition modes and send mode feedback information to the wearable device to provide feedback on the target acquisition mode.

[0197] In some embodiments, the processing module 402 is further configured to: if any acquisition node is not connected or has not completed synchronization, determine that the operating status of the plurality of acquisition nodes does not meet the preset acquisition conditions; generate abnormal feedback information based on the first abnormal acquisition node that is not connected or has not completed synchronization, and send the abnormal feedback information to the wearable device; after the operating status of the first abnormal acquisition node is updated, re-detect the operating status of the plurality of acquisition nodes until the operating status of the plurality of acquisition nodes all meet the preset acquisition conditions.

[0198] In some implementations, the processing module 402 sends synchronization references and acquisition trigger signals to multiple acquisition nodes according to the target acquisition mode, so that when the multiple acquisition nodes acquire multi-source data at a unified sampling time, it is specifically configured as follows: A reference timestamp is sent to the multiple acquisition nodes, enabling them to adjust their local clocks accordingly. Multiple sets of synchronization pulses are sent to the multiple acquisition nodes, and the corresponding clock drift and link transmission delay are determined based on the clock feedback values ​​returned by each acquisition node. Clock calibration parameters are generated based on the clock drift and link transmission delay, and these parameters are sent to the corresponding acquisition nodes. Acquisition parameters for each acquisition node are determined according to the target acquisition mode, and the acquisition trigger signal and corresponding acquisition parameters are sent to the multiple acquisition nodes so that they can initiate data acquisition at the unified sampling time according to their calibrated local clocks.

[0199] In some implementations, when the processing module 402 determines the acquisition parameters corresponding to each acquisition node according to the target acquisition mode, it is specifically configured to: increase the sampling rate of the head acquisition node and hand acquisition node among the multiple acquisition nodes when the target acquisition mode is a high-precision mode; decrease the sampling rate of non-critical acquisition nodes among the multiple acquisition nodes when the target acquisition mode is a low-power mode; and control the multiple acquisition nodes to acquire the multi-source data according to the corresponding preset sampling rate when the target acquisition mode is a normal mode.

[0200] In some embodiments, the master acquisition node is located on a head-mounted wearable device, and the multi-source data acquired by the master acquisition node includes environmental images, environmental depth information, and head inertial information; multiple slave acquisition nodes are respectively located on multiple body parts of the wearer's torso, limbs, hands, and fingers, and the multi-source data acquired by each of the multiple slave acquisition nodes includes inertial motion information of the corresponding body part; the multiple acquisition nodes add node identifiers and local timestamps to the acquired multi-source data to form multi-source data with the unified sampling time relationship.

[0201] In some implementations, when the processing module 402 performs fusion processing on the multi-source data to obtain the motion capture acquisition result, it is specifically configured to: receive multi-source data sent by the main acquisition node and multi-source data sent by the multiple slave acquisition nodes through physically isolated wired transmission links; parse and verify the integrity of each data frame in the multi-source data, discard data frames that fail verification and record frame loss information; organize the data frames of each channel according to the local timestamp of each data frame, and remap the local timestamp of each acquisition node to a unified reference time axis; filter out noise from the multi-source data after frame order organization and timestamp remapping, and reconstruct it into data to be solved according to time slices; perform human kinematics calculation and multi-source fusion on the data to be solved to obtain the motion capture acquisition result.

[0202] In some embodiments, when the processing module 402 performs human kinematics calculation and multi-source fusion on the data to be calculated to obtain the motion capture acquisition result, it is specifically configured as follows: according to pre-stored calibration parameters, the inertial motion information in the data to be calculated is transformed from the sensor coordinate system to the local coordinate system of the corresponding skeleton; the environmental image and the environmental depth information are transformed from the acquisition coordinate system to the coordinate system of the head-mounted wearable device; and the transformed inertial motion information, environmental image, and environmental depth information are mapped to the global coordinate system; kinematic calculation is performed on the inertial motion information based on the human skeletal kinematic chain to obtain the relative pose of each skeletal node; visual positioning is performed based on the environmental image to obtain the global pose of the main acquisition node; the cumulative drift corresponding to the relative pose of each skeletal node is corrected using the global pose of the main acquisition node as a constraint to obtain whole-body skeletal motion data; and multi-source fusion is performed on the whole-body skeletal motion data, the environmental image, and the environmental depth information based on human motion constraints and spatial observation constraints to obtain whole-body skeletal motion capture data and environmental three-dimensional perception data carrying global spatial pose, which are used as the motion capture acquisition result.

[0203] In some implementations, when the processing module 402 verifies the motion capture acquisition results to obtain the corresponding data validity determination results, it is specifically configured to: perform time synchronization verification, data integrity verification, and visual validity verification on the multi-source data corresponding to the same sampling time, and perform motion rationality verification for the corresponding motion capture acquisition results; mark the data frames that pass all verifications in the multi-source data as valid frames; mark the data frames that fail any verification in the multi-source data as invalid frames, and record the failure reason and the corresponding acquisition node; and obtain the data validity determination results based on the data frame markings, the failure reason, and the corresponding acquisition node.

[0204] In some embodiments, the processing module 402 is further configured to: after verifying the motion capture acquisition results to obtain the corresponding data validity determination results, to statistically analyze the proportion of valid frames, the length of consecutive invalid frames, and the abnormal frequency of each acquisition node in the multi-source data; to determine the data validity status of the corresponding acquisition segment based on the proportion of valid frames, the length of consecutive invalid frames, and the abnormal frequency; and if the length of consecutive invalid frames reaches a preset length, to mark the corresponding acquisition segment as a failed acquisition segment and to generate abnormal handling information to prompt the wearer to adjust their movements, check the environment, or inspect the acquisition node.

[0205] In some implementations, the data acquisition and operation interaction includes at least one of the following: display brightness adjustment, preview channel switching, acquisition parameter adjustment, and keyframe marking. When the processing module 402 executes the acquisition control command corresponding to the data acquisition and operation interaction, it is specifically configured to: adjust the display brightness of the wearable device according to the acquisition control command corresponding to the display brightness adjustment operation; or, switch the data preview channel displayed by the wearable device according to the acquisition control command corresponding to the preview channel switching operation; or, adjust the acquisition parameters of at least one acquisition node according to the acquisition control command corresponding to the acquisition parameter adjustment operation; or, add keyframe markers to the multi-source data or motion capture acquisition results corresponding to the current sampling time according to the acquisition control command corresponding to the keyframe marking operation.

[0206] In some implementations, when the processing module 402 generates acquisition feedback information based on the execution result of the acquisition control command, the operating status of the plurality of acquisition nodes, and the data validity determination result, it is specifically configured as follows: Based on the operating status of the multiple acquisition nodes, an overview of the acquisition dimensions is generated to display the access status of the acquisition nodes and the operating status of the data channel; based on the data validity determination result, data validity content is generated to display the data validity status, failure reason, and at least one of the corresponding acquisition nodes; based on the acquisition control command and the corresponding execution result, acquisition interaction guidance content is generated to display operation instructions and acquisition operation results; based on the acquisition dimension overview content, the data validity content, and the acquisition interaction guidance content, display feedback information is generated, and the head-up display component of the wearable device is controlled to display the display feedback information; based on the operation type of the input interactive operation, haptic feedback control information is generated, and the vibration component of the wearable device is controlled to output the corresponding vibration feedback.

[0207] In some embodiments, the wearable device is provided with a damped knob; when the processing module 402 acquires the wake-up interaction operation, the data acquisition start interaction operation, or the data acquisition run interaction operation, it is further configured to: in response to the rotation operation of the damped knob, determine a target data acquisition option from multiple candidate data acquisition options based on the change in the damped knob's position; in response to the pressing operation of the damped knob, identify the pressing type of the pressing operation; and determine a corresponding interaction operation based on at least one of the target data acquisition option and the pressing type, wherein the pressing type includes at least one of short press, long press, and double-click.

[0208] In some embodiments, the wearable device is provided with a touch-sensing area; when the processing module 402 acquires the target acquisition mode, the acquisition start interaction operation, or the acquisition run interaction operation, it is specifically configured to: acquire a swipe operation, a click operation, or a long press operation input in the touch-sensing area; determine an acquisition mode switching operation, a display brightness adjustment operation, or an acquisition parameter adjustment operation based on the swipe operation; determine an option confirmation operation or an acquisition start / stop operation based on the click operation; and determine an acquisition end operation based on the long press operation.

[0209] In some embodiments, the wearable device is equipped with a vibration sensor; when the processing module 402 acquires the wake-up interaction operation, the target acquisition mode, the acquisition start interaction operation, or the acquisition run interaction operation, it is specifically configured to: detect a tapping operation input to the wearable device through the vibration sensor; when the wearable device is in a sleep state, determine a single tapping operation as the wake-up interaction operation; when the running state meets the preset acquisition conditions, determine two consecutive tapping operations as acquisition mode selection interaction operations, and determine the target acquisition mode according to the acquisition mode selection interaction operation; after determining the target acquisition mode, determine a single tapping operation as the acquisition start interaction operation; during acquisition run, determine a single tapping operation as a keyframe marking operation, and determine two consecutive tapping operations as a preview channel switching operation.

[0210] In some embodiments, the processing module 402 is further configured to: respond to an input acquisition end interaction operation, after ending data acquisition at the plurality of acquisition nodes, determine the acquisition completion status corresponding to each acquisition node based on the running status of the plurality of acquisition nodes recorded during acquisition and the data validity determination result, wherein the acquisition completion status includes a normal status, an abnormal status, or a failure status; if the acquisition completion status of any acquisition node is the abnormal status or the failure status, determine a second abnormal acquisition node and its corresponding abnormal cause based on the node identifier, node location, running status, and corresponding data validity determination result of the acquisition node, wherein the abnormal cause includes at least one of acquisition node not started, acquisition node failure, missing acquisition data, or abnormal acquisition data; generate acquisition completion feedback information based on the data acquisition end status, the acquisition completion status corresponding to each acquisition node, and the second abnormal acquisition node and its corresponding abnormal cause; send the acquisition completion feedback information to the wearable device so that the wearable device displays the acquisition completion summary information to the wearer, and displays the abnormal acquisition node and its corresponding abnormal cause if the abnormal acquisition node exists.

[0211] Optionally, the device is communicatively connected to a wearable device and multiple acquisition nodes, including a main acquisition node and multiple slave acquisition nodes distributed on multiple body parts of the wearer. The main acquisition node is used to acquire the wearer's head movement information and environmental information, and the slave acquisition nodes are used to acquire movement information of corresponding body parts.

[0212] In this embodiment of the application, the above-mentioned device enables the wearer to keep abreast of the operation execution status and motion capture data acquisition status, ensuring the security and reliability of data acquisition and improving the efficiency of data acquisition interaction.

[0213] The wearable motion capture data acquisition device 40 in this application embodiment has been described above from the perspective of modular functional entities. The wearable motion capture data acquisition device in this application embodiment will be described below from the perspective of hardware processing.

[0214] It should be noted that, Figure 4 The physical device corresponding to the input / output module 401 shown can be a transceiver, radio frequency circuit, communication module, and input / output (I / O) interface, etc., and the physical device corresponding to the processing module 402 can be a processor. Figure 4 The devices shown can all have the following characteristics: Figure 5 The structure shown, when Figure 4 The wearable motion capture data acquisition device 40 shown has, for example, Figure 5 When the structure shown is used, Figure 5The processor and transceiver in the device can perform the same or similar functions as the processing module 402 and input / output module 401 provided in the aforementioned device embodiments. Figure 5 The memory in the processor stores the computer programs that need to be called when executing the wearable motion capture data acquisition method described above.

[0215] This application also provides a computing device; please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of a computing device structure provided in an embodiment of this application. The computing device 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and a memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 can be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the computing device. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the computing device 1100. The computing device 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0216] The steps performed by the computing device in the above embodiments can be based on this Figure 6 The structure of the computing device 1100 shown is illustrated. For example, in the above embodiment, it is composed of... Figure 4 The steps performed by the wearable motion capture data acquisition device 40 shown can be based on this Figure 6 The computing device structure is shown. For example, the central processing unit 1122 performs the following operations by calling instructions from memory 1132: The input / output interface 1158 receives wake-up interaction operations, acquisition start interaction operations, and / or acquisition run interaction operations; sends synchronization references and acquisition trigger signals to multiple acquisition nodes, and receives multi-source data returned by the multiple acquisition nodes; and sends the acquisition feedback information to the wearable device, so that the wearable device can provide feedback on the motion capture acquisition status to the wearer. It can also respond to input wake-up interaction operations, detect the running status of the multiple acquisition nodes, and determine the target acquisition mode when the running status of the multiple acquisition nodes meets the preset acquisition conditions; respond to input acquisition start interaction operations, send synchronization reference and acquisition trigger signals to the multiple acquisition nodes through the input / output module according to the target acquisition mode, so that the multiple acquisition nodes can acquire the multi-source data at a unified sampling time; perform fusion processing on the multi-source data to obtain motion capture acquisition results, and verify the motion capture acquisition results to obtain the corresponding data validity judgment results; during acquisition operation, respond to input acquisition operation interaction operations, execute the acquisition control command corresponding to the acquisition operation interaction operation, and generate acquisition feedback information according to the execution result of the acquisition control command, the running status of the multiple acquisition nodes, and the data validity judgment results.

[0217] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0218] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0219] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0220] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0221] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0222] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0223] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0224] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A wearable motion capture data acquisition method, characterized in that, The method includes: In response to the input wake-up interaction, the running status of multiple acquisition nodes is detected, and the target acquisition mode is determined when the running status of the multiple acquisition nodes meets the preset acquisition conditions. In response to the input acquisition initiation interactive operation, a synchronization reference and acquisition trigger signal are sent to the multiple acquisition nodes according to the target acquisition mode, so that the multiple acquisition nodes can acquire multi-source data at a unified sampling time, and receive the multi-source data returned by the multiple acquisition nodes; The multi-source data is fused to obtain motion capture acquisition results, and the motion capture acquisition results are verified to obtain corresponding data validity judgment results; During the data acquisition process, in response to the input data acquisition operation interaction, the system executes the data acquisition control command corresponding to the data acquisition operation interaction, and generates data acquisition feedback information based on the execution result of the data acquisition control command, the operating status of the multiple data acquisition nodes and the data validity determination result. The system then sends the data acquisition feedback information to the wearable device, so that the wearable device can provide feedback on the motion capture data acquisition status to the wearer.

2. The wearable motion capture data acquisition method according to claim 1, characterized in that, The response to the input wake-up interaction operation, detecting the operating status of multiple acquisition nodes, and determining the target acquisition mode when the operating status of all multiple acquisition nodes meets the preset acquisition conditions, includes: In response to the wake-up interaction operation, wake-up information and node query information are sent to the multiple acquisition nodes; Receive node identifiers and initial status information returned by each acquisition node, register the multiple acquisition nodes and establish a node topology; The operating status of the multiple acquisition nodes is determined based on their connection and synchronization status. If all the multiple acquisition nodes are in a connected and synchronized state ready for acquisition, then it is determined that the operating state of the multiple acquisition nodes meets the preset acquisition conditions. In response to the input acquisition mode selection interaction, the target acquisition mode is determined from multiple candidate acquisition modes, and mode feedback information for providing feedback on the target acquisition mode is sent to the wearable device.

3. The wearable motion capture data acquisition method according to claim 2, characterized in that, The method further includes: If any acquisition node is not connected or has not completed synchronization, it is determined that the operating status of the multiple acquisition nodes does not meet the preset acquisition conditions. Anomaly feedback information is generated based on the first abnormal acquisition node that is not connected or has not completed synchronization, and the anomaly feedback information is sent to the wearable device. After the running status of the first abnormal acquisition node is updated, the running status of the multiple acquisition nodes is re-detected until the running status of the multiple acquisition nodes all meet the preset acquisition conditions.

4. The wearable motion capture data acquisition method according to claim 1, characterized in that, The step of sending synchronization references and acquisition trigger signals to multiple acquisition nodes according to the target acquisition mode, so that the multiple acquisition nodes can acquire multi-source data at a unified sampling time, includes: A reference timestamp is sent to the multiple acquisition nodes, so that the multiple acquisition nodes adjust their local clocks according to the reference timestamp; Multiple sets of synchronization pulses are sent to the multiple acquisition nodes, and the corresponding clock drift and link transmission delay are determined based on the clock feedback values ​​returned by each acquisition node. Clock calibration parameters are generated based on the clock drift and the link transmission delay, and the clock calibration parameters are sent to the corresponding acquisition nodes; The acquisition parameters corresponding to each acquisition node are determined according to the target acquisition mode, and the acquisition trigger signal and the corresponding acquisition parameters are sent to the multiple acquisition nodes so that the multiple acquisition nodes can start data acquisition at the unified sampling time according to the calibrated local clock.

5. The wearable motion capture data acquisition method according to claim 4, characterized in that, The step of determining the acquisition parameters corresponding to each acquisition node according to the target acquisition mode includes: When the target acquisition mode is high-precision mode, the sampling rate of the head acquisition node and hand acquisition node among the multiple acquisition nodes is increased; When the target acquisition mode is a low-power mode, the sampling rate of non-critical acquisition nodes among the multiple acquisition nodes is reduced; When the target acquisition mode is the daily mode, the multiple acquisition nodes are controlled to acquire the multi-source data according to the corresponding preset sampling rate.

6. The wearable motion capture data acquisition method according to claim 1, characterized in that, The main acquisition node is set on the head-mounted wearable device, and the multi-source data acquired by the main acquisition node includes environmental images, environmental depth information, and head inertial information; The multiple acquisition nodes are respectively set in multiple body parts of the wearer's torso, limbs, palms and fingers, and the multi-source data collected by each of the multiple acquisition nodes includes the inertial motion information of the corresponding body part. The multiple acquisition nodes add node identifiers and local timestamps to the acquired multi-source data to form multi-source data with the unified sampling time relationship.

7. The wearable motion capture data acquisition method according to claim 6, characterized in that, The process of fusing the multi-source data to obtain the motion capture acquisition results includes: Through physically isolated wired transmission links, the system receives multi-source data sent by the main acquisition node and multi-source data sent by the multiple slave acquisition nodes, respectively. Each data frame in the multi-source data is parsed and its integrity is verified. Data frames that fail the verification are discarded and the lost frame information is recorded. The data frames of each channel are ordered according to their local timestamps, and the local timestamps of each acquisition node are remapped to a unified reference time axis. Noise is filtered out from the multi-source data after frame ordering and timestamp remapping, and the data is reconstructed into the data to be solved according to time slices; The data to be solved is subjected to human kinematics calculation and multi-source fusion to obtain the motion capture acquisition results.

8. The wearable motion capture data acquisition method according to claim 7, characterized in that, The process of performing human kinematics calculations and multi-source fusion on the data to be solved to obtain the motion capture acquisition results includes: Based on the pre-stored calibration parameters, the inertial motion information in the data to be solved is transformed from the sensor coordinate system to the local coordinate system of the corresponding skeleton, the environmental image and the environmental depth information are transformed from the acquisition coordinate system to the coordinate system of the head-mounted wearable device, and the transformed inertial motion information, environmental image and environmental depth information are mapped to the global coordinate system. Based on the human skeletal kinematic chain, kinematic calculations are performed on the inertial motion information to obtain the relative pose of each skeletal node; Visual localization is performed based on the environmental image to obtain the global pose of the main acquisition node; Using the global pose of the main acquisition node as a constraint, the cumulative drift corresponding to the relative pose of each skeletal node is corrected to obtain whole-body skeletal motion data. Based on human motion constraints and spatial observation constraints, the whole-body skeletal motion data, the environmental images, and the environmental depth information are fused from multiple sources to obtain whole-body skeletal motion capture data and environmental 3D perception data carrying global spatial pose, which are used as the motion capture acquisition results.

9. The wearable motion capture data acquisition method according to claim 1, characterized in that, The verification of the motion capture acquisition results to obtain the corresponding data validity judgment results includes: Perform time synchronization verification, data integrity verification, and visual validity verification on multi-source data corresponding to the same sampling time, and perform motion rationality verification on the corresponding motion capture acquisition results; Mark the data frames that pass all verifications in the multi-source data as valid frames; Mark any data frame that fails verification in the multi-source data as an invalid frame, and record the reason for the failure and the corresponding acquisition node; The data validity determination result is obtained based on the data frame marker, the cause of failure, and the corresponding acquisition node.

10. The wearable motion capture data acquisition method according to claim 9, characterized in that, After obtaining the corresponding data validity determination result by verifying the motion capture acquisition result, the method further includes: The percentage of valid frames, the length of consecutive invalid frames, and the frequency of anomalies at each acquisition node in the multi-source data are statistically analyzed. The validity status of the data in the corresponding acquisition segment is determined based on the percentage of valid frames, the length of consecutive invalid frames, and the frequency of anomalies. If the length of the consecutive invalid frames reaches a preset length, the corresponding acquisition segment will be marked as a failed acquisition segment, and abnormal handling information will be generated to prompt the wearer to adjust their actions, check the environment, or inspect the acquisition node.

11. The wearable motion capture data acquisition method according to claim 1, characterized in that, The data acquisition and operation interactive operations include at least one of the following: display brightness adjustment operation, preview channel switching operation, data acquisition parameter adjustment operation, and keyframe marking operation; The execution of the acquisition control command corresponding to the acquisition operation includes: Adjust the display brightness of the wearable device according to the acquisition control command corresponding to the display brightness adjustment operation; or... According to the acquisition control command corresponding to the preview channel switching operation, switch the data preview channel displayed by the wearable device; or... According to the acquisition control command corresponding to the acquisition parameter adjustment operation, adjust the acquisition parameters of at least one acquisition node; or, Based on the acquisition control command corresponding to the keyframe marking operation, add keyframe markers to the multi-source data or motion capture acquisition results corresponding to the current sampling time.

12. The wearable motion capture data acquisition method according to claim 1, characterized in that, The step of generating acquisition feedback information based on the execution result of the acquisition control command, the operating status of the multiple acquisition nodes, and the data validity determination result includes: Based on the operating status of the multiple acquisition nodes, an overview of the acquisition dimensions is generated to display the access status of the acquisition nodes and the operating status of the data channel; Based on the data validity determination result, generate data validity content to display the data validity status, failure reason, and at least one of the corresponding acquisition nodes; Generate interactive guidance content for displaying operation instructions and acquisition operation results based on the acquisition control commands and corresponding execution results; Based on the overview content of the collection dimensions, the data validity content, and the collection interaction guidance content, display feedback information is generated, and the head-up display component of the wearable device is controlled to display the display feedback information; Based on the type of operation of the input interaction, haptic feedback control information is generated, and the vibration component of the wearable device is controlled to output corresponding vibration feedback.

13. The wearable motion capture data acquisition method according to claim 1, characterized in that, The wearable device is equipped with a damped knob; The method for obtaining the wake-up interaction operation, the data acquisition start interaction operation, or the data acquisition run interaction operation further includes: In response to the rotation of the damping knob, the target acquisition option is determined from multiple candidate acquisition options based on the change in the damping knob's position. In response to a press operation on the damped knob, the press type of the press operation is identified; The corresponding interactive operation is determined based on at least one of the target acquisition option and the press type, wherein the press type includes at least one of short press, long press and double click.

14. The wearable motion capture data acquisition method according to claim 1, characterized in that, The wearable device is equipped with a touch-sensing area; The methods for obtaining the target acquisition mode, the acquisition initiation interactive operation, or the acquisition operation include: The system can detect swipe, tap, or long press inputs within the touch-sensing area. The sliding operation determines the acquisition mode switching operation, the display brightness adjustment operation, or the acquisition parameter adjustment operation. Based on the click operation, confirm the option to perform an operation or start / stop the data collection process; The long press operation determines the end of the data collection process.

15. The wearable motion capture data acquisition method according to claim 1, characterized in that, The wearable device is equipped with a vibration sensor; The methods for obtaining the wake-up interaction operation, the target acquisition mode, the acquisition start interaction operation, or the acquisition run interaction operation include: The vibration sensor detects tapping inputs to the wearable device. When the wearable device is in sleep mode, a single tap operation is determined as the wake-up interaction operation; When the operating state meets the preset acquisition conditions, two consecutive tapping operations are determined as acquisition mode selection interaction operations, and the target acquisition mode is determined according to the acquisition mode selection interaction operations. After determining the target acquisition mode, a single tap operation is defined as the acquisition initiation interaction operation; During the acquisition process, a single tap operation is defined as a keyframe marking operation, and two consecutive tap operations are defined as preview channel switching operations.

16. The wearable motion capture data acquisition method according to claim 1, characterized in that, After the data acquisition at the multiple acquisition nodes ends in response to the input acquisition termination interaction, the method further includes: Based on the operating status of the multiple acquisition nodes recorded during the acquisition period and the data validity determination results, the acquisition completion status corresponding to each acquisition node is determined respectively. The acquisition completion status includes normal status, abnormal status or failure status. If the acquisition completion status of any acquisition node is in the abnormal state or the failure state, the second abnormal acquisition node and the corresponding abnormal reason are determined according to the node identifier, node location, running status and corresponding data validity judgment result of the acquisition node. The abnormal reason includes at least one of the following: acquisition node not started, acquisition node failure, missing acquisition data or abnormal acquisition data. The data collection completion feedback information is generated based on the data collection end status, the data collection completion status of each collection node, the second abnormal collection node and its corresponding abnormal reason; The data collection completion feedback information is sent to the wearable device so that the wearable device can display the data collection completion summary information to the wearer, and if there is an abnormal data collection node, display the abnormal data collection node and the corresponding abnormal reason.

17. A wearable motion capture data acquisition device, characterized in that, The device is used to implement the wearable motion capture data acquisition method as described in any one of claims 1-16, the device comprising: The input / output module is configured to receive wake-up interaction operations, acquisition start interaction operations, and / or acquisition run interaction operations; send synchronization references and acquisition trigger signals to multiple acquisition nodes, and receive multi-source data returned by the multiple acquisition nodes; and send the acquisition feedback information to the wearable device, so that the wearable device can provide feedback on the motion capture acquisition status to the wearer. The processing module is configured to, in response to an input wake-up interaction, detect the operating status of the multiple acquisition nodes and determine the target acquisition mode if the operating status of the multiple acquisition nodes meets the preset acquisition conditions; in response to an input acquisition start interaction, send a synchronization reference and acquisition trigger signal to the multiple acquisition nodes through the input / output module according to the target acquisition mode, so that the multiple acquisition nodes acquire the multi-source data at a unified sampling time; perform fusion processing on the multi-source data to obtain motion capture acquisition results, and verify the motion capture acquisition results to obtain corresponding data validity judgment results; during acquisition operation, in response to an input acquisition operation interaction, execute the acquisition control command corresponding to the acquisition operation interaction, and generate acquisition feedback information based on the execution result of the acquisition control command, the operating status of the multiple acquisition nodes, and the data validity judgment result.

18. A wearable motion capture data acquisition device, characterized in that, The wearable motion capture data acquisition device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the wearable motion capture data acquisition method according to any one of claims 1-16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the wearable motion capture data acquisition method as described in any one of claims 1-16.

Images