Manipulation ability detection system and method based on deep sea closed environment simulation

Abstract

This invention discloses a maneuverability detection system and method based on deep-sea enclosed environment simulation, belonging to the field of data processing technology. The method includes the following steps: acquiring and generating synchronous data frames; performing time alignment, filtering, and denoising on the synchronous data frames; calculating visual-tactile time delay based on visual and tactile measurements and outputting a perceptual consistency coefficient, thereby generating a visual trust coefficient; calculating hand micro-motion variance based on hand temperature and equipment temperature and generating a thermal performance factor; establishing a personnel capability evaluation model based on the visual trust coefficient, thermal performance factor, visual performance coefficient, and tactile performance coefficient to calculate a comprehensive operational score; and calculating and applying force feedback gain based on the perceptual consistency coefficient and the thermal performance factor. This invention addresses the shortcomings of multimodal data fusion and rule-based scoring methods in handling complex coupling risks such as visual-tactile misalignment, thermal degradation, and time delay coupling.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a control capability detection system and method based on deep-sea enclosed environment simulation. Background Technology

[0002] The deep-sea operating environment presents extremely high challenges to human-machine interaction and remote control. Unlike terrestrial environments, the deep-sea environment is often accompanied by extremely low light levels, turbid water, and complex light scattering. These conditions severely degrade camera image quality and increase the uncertainty of visual information. At the same time, due to communication latency, sensor bandwidth limitations, and environmental disturbances, the visual channel often exhibits delays or distortions. In this context, tactile and force perception become key sensory sources for ensuring precise control and safe operation. If the detection, evaluation, and training of deep-sea control capabilities rely solely on vision or artificially enhance vision in simulations (e.g., through screens or augmented reality) without simultaneously and realistically reproducing force perception, perceptual conflicts (inconsistencies between vision and touch) are highly likely to occur. This can lead to significant deviations or over-reliance on vision by operators who perform well in simulation environments in real-world operating scenarios, thereby reducing the external validity and safety of test results.

[0003] On the other hand, deep-sea platforms often operate in confined spaces, where environmental temperature control and equipment heat dissipation capabilities may be limited. Heat accumulation in localized components or operating surfaces can lead to increased hand temperature, muscle stiffness, and micro-movements (tremors), thereby impairing the delicacy and stability of maneuvering. Hand temperature and high-frequency micro-movements revealed by the inertial measurement unit can serve as objective indicators for assessing the operator's physiological state and maneuvering stability; their coupling can also further degrade the system's control performance by affecting the perception-execution time lag. Therefore, conducting maneuverability testing in the confined environment of the deep sea requires not only considering the consistency of the sensing channels but also incorporating thermal and neuromechanical stability into a comprehensive evaluation and adaptive compensation system.

[0004] Therefore, the present invention provides a control capability detection system and method based on deep-sea enclosed environment simulation. Summary of the Invention

[0005] The purpose of this invention is to provide a control capability detection system and method based on deep-sea enclosed environment simulation, so as to solve the existing problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a maneuverability detection system based on deep-sea confined environment simulation, comprising:

[0007] The environmental and control data acquisition module is used to collect visual energy, tactile energy, hand inertial data, hand temperature and device temperature and generate synchronous data frames;

[0008] A multimodal synchronization and preprocessing module is used to perform time alignment, filtering, and noise reduction on the synchronized data frames;

[0009] The perceptual consistency assessment module is used to calculate the visual-tactile time lag based on visual and tactile measurements and output the perceptual consistency coefficient.

[0010] A visual trust coefficient generation module is used to generate a visual trust coefficient based on the perceptual consistency coefficient.

[0011] The thermal estimation and compensation module is used to calculate the variance of hand micro-motion based on hand temperature and equipment temperature and generate thermal performance factors;

[0012] The operational ability evaluation and training module is used to establish a personnel ability evaluation model based on the visual trust coefficient, thermal performance factor, visual energy level and tactile energy level to calculate the comprehensive operational score;

[0013] The feedback gain adaptive module is used to calculate and apply the force feedback gain based on the perception consistency coefficient and the thermal performance factor.

[0014] A further improvement of this invention is that the perceptual consistency evaluation module calculates the visual measurement at each sampling point based on visual and tactile data within a time window. With tactile measurement The root mean square error is used to obtain the visual-tactile time delay. And through preset scale parameters Mapping-perceptual consistency coefficient The visual trust coefficient is obtained by using interval mapping to determine the perceived consistency. .

[0015] A further improvement of the present invention is that the thermal estimation and compensation module includes hand inertial data. The acceleration channel is high-pass filtered to separate non-autonomous jitter components, retaining high-frequency micro-jitter components to obtain a micro-motion sequence; the micro-motion sequence is then calculated within a window. The variance within the range is denoted as the hand microvariance variance. This leads to the hot-state performance factor, expressed as:

[0016] ;

[0017] Represents the hot performance factor. Indicates hand temperature. Indicates the equipment temperature. , , , Indicates the preset calibration parameters; Output and To the feedback gain adaptive module and the operational capability evaluation and training module; when When the value is below a preset threshold, a cooling event is triggered.

[0018] A further improvement of this invention is that the personnel competence evaluation model is based on an LSTM model, using hot performance factors. Visual energy level tactile energy level Based on the above data, a temporal feature sequence is constructed using the corresponding basic operation scores. In the forward computation, a bias update strategy is implemented, introducing a visual trust coefficient at each time step of the bias term of the input gate. Combined with the baseline bias, a visual trust bias term is obtained. Subsequently, an LSTM single-step forward pass is run to obtain the hidden representation, and the operation comprehensive score is output by the FC layer. .

[0019] A further improvement of this invention is that the bias update strategy includes presetting a baseline bias before initializing the personnel competency evaluation model. and learning components And according to the affine relation and the smoothed visual confidence coefficient obtained by short-time exponential moving average, Candidate biases are obtained by combining the visual bias calculation sub-strategy with the visual bias calculation sub-strategy. Then, the rate of change of the candidate bias relative to the previous bias is limited and passed through a safe interval. Crop the image to obtain the visual correction bias. , used to replace the constant bias of the LSTM input gate, thereby updating the input gate of the personnel competence evaluation model, wherein This represents the standard visual trust coefficient.

[0020] A further improvement of this invention is that the visual bias calculation sub-strategy is based on the operational warning coefficient. The specific steps to achieve this include:

[0021] Set instability threshold and warning threshold ,when At that time, the visual trust coefficient is taken as its original value;

[0022] when At that time, the visual trust coefficient is updated according to a decreasing rule, represented as ,in, This represents the preset decreasing coefficient;

[0023] when When this occurs, forcibly stop personnel operations and update the task tolerance according to the task tolerance increase rule. ,in, This indicates the preset increment coefficient.

[0024] A further improvement of this invention is that the operational early warning coefficient... The scaled thermal-time delay coupling index and the neuro-mechanical stability index, along with their product, are fused using the hyperbolic tangent function, and then weighted by a weight vector. Allocate contributions from each part and adjust the operational early warning coefficient. Mapped to The calculation formula is expressed as:

[0025] ;

[0026] in, Indicates the neuromechanical stability index. Indicates the thermal-time delay coupling index. , , This indicates the preset operation warning weight.

[0027] A further improvement of this invention is that the thermal-time delay coupling index is obtained through the cross-modal time delay thermistor coefficient. Time-delay thermosensitive critical value Hot performance factor Hand temperature Hand temperature baseline The calculation formula is as follows: ;in, Indicates the thermal-time delay coupling index. For a safe temperature difference scale, the transmodal time-delay thermistor is expressed as: .

[0028] A further improvement of this invention is that the neuro-mechanical stability index is obtained through the variance of hand micro-movements. Baseline variance of hand micro-movements Hot performance factor The coupling is obtained, and the calculation formula is expressed as: ,in, This indicates the set sensitivity coefficient.

[0029] A further improvement of the present invention is that the basic operational score is determined by the hot performance factor. and visual energy level With tactile energy The product of the weighted sums is obtained.

[0030] A further improvement of the present invention is that the force feedback gain By setting the base gain Simultaneously extract the perceived consistency coefficient. and the aforementioned hot performance factor Calculate feedback gain ;in, , This indicates the set gain weight;

[0031] As a real-time control command, it is sent to the force feedback device via the real-time control bus, and the force feedback device adjusts the tactile intensity accordingly.

[0032] On the other hand, the present invention provides a method for detecting maneuverability based on deep-sea confined environment simulation, comprising the following steps:

[0033] S1. Collect visual energy, tactile energy, hand inertial data, hand temperature and device temperature and generate a synchronous data frame;

[0034] S2. Perform time alignment, filtering, and noise reduction on the synchronous data frame;

[0035] S3. Calculate the visual-tactile time delay based on visual and tactile measurements and output the perceptual consistency coefficient;

[0036] S4. Generate a visual trust coefficient based on the perceptual consistency coefficient;

[0037] S5. Calculate the hand micro-motion variance based on hand temperature and equipment temperature and generate thermal performance factor;

[0038] S6. Based on the aforementioned visual trust coefficient, thermal performance factor, visual performance intensity and tactile performance intensity, establish a personnel competency evaluation model to calculate the comprehensive operational score;

[0039] S7. Calculate and apply the force feedback gain based on the perception consistency coefficient and the thermal performance factor.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. This invention first generates synchronized data frames by multimodal temporal synchronization and real-time consistency quantization. Through visual trust coefficients, it realizes real-time alignment detection and coefficient adjustment of vision, touch, and force. When vision is enhanced but force is not synchronized in the simulation, the system can automatically reduce the visual weight and adjust the scoring and control according to the visual trust coefficient. This avoids false superior performance caused by vision priority in the simulation, ensures that the evaluation results are closer to the real ability on site, and reduces operational errors and risks caused by perceptual conflicts.

[0042] 2. By combining temperature and micro-motion monitoring and thermal compensation strategies, early identification and intervention can be achieved for the impact of heat accumulation on fine control capabilities. When local high temperature or thermal stress in the cabin causes hand stiffness, the mission precision can be relaxed in a timely manner, the force feedback can be adjusted, or the mission can be paused to protect operator safety and maintain the reliability of scoring and control. Attached Figure Description

[0043] Figure 1 This is a framework diagram of the maneuverability detection system based on deep-sea enclosed environment simulation of the present invention;

[0044] Figure 2 This is a flowchart of the control capability testing method based on deep-sea enclosed environment simulation of the present invention. Detailed Implementation

[0045] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0046] The term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone.

[0047] Example 1

[0048] Figure 1 This embodiment illustrates the framework diagram of the maneuverability detection system based on deep-sea confined environment simulation, including:

[0049] The environmental and control data acquisition module is used to collect visual measurements. tactile measurement Hand inertial data Hand temperature With equipment temperature And generate synchronization data frames;

[0050] A multimodal synchronization and preprocessing module is used to perform time alignment, filtering, and noise reduction on the synchronized data frames;

[0051] Perceptual consistency assessment module, used for visual measurement With tactile measurement Calculate the visual-tactile time delay and output the perceptual consistency coefficient. ;

[0052] The visual trust coefficient generation module is used to generate a visual trust coefficient based on the perceptual consistency coefficient. ;

[0053] The thermal estimation and compensation module is used to estimate hand temperature. and equipment temperature Calculate the variance of hand micro-movements And generate hot performance factor ;

[0054] The operational capability evaluation and training module is used to evaluate and train based on the visual trust coefficient. Hot performance factor Visual energy level With tactile energy Establish a personnel competency evaluation model to calculate a comprehensive operational score. ;

[0055] The feedback gain adaptive module is used to adjust the perceived consistency coefficient. The aforementioned thermal performance factor

[0056] Calculate and apply force feedback gain .

[0057] The perceptual consistency assessment module calculates visual measurements at each sampling point based on visual and tactile data within a time window. With tactile measurement The root mean square error is used to obtain the visual-tactile time delay. ;

[0058] The smaller, The closer it is to 1, the higher the consistency between vision and touch; The larger, The closer the value is to 0, the more significant the misalignment. This embodiment provides explicit quantitative indicators. It can detect visual-tactile asynchrony or inconsistency in real time, thereby triggering weight adjustment or synchronization calibration;

[0059] Increasing the window N suppresses short-term noise and detects slow-changing misalignments, which is beneficial for migration detection, but slows down the response.

[0060] Reducing the window size N results in a faster response, but it is more susceptible to false alarms due to transient errors and may be triggered by halo / frame loss.

[0061] Directly quantify hand-eye misalignment; high visual-tactile lag indicates visual unreliability; visual-tactile lag temporarily increases when switching interfaces, and a suitable window can distinguish between transient migration fluctuations and continuous misalignment; at the same time, heat-induced micro-shaking increases inter-point deviation, indirectly increasing visual-tactile lag, indicating thermal correlation.

[0062] By preset scale parameters Mapping-perceptual consistency coefficient The visual trust coefficient is obtained by using interval mapping to determine the perceived consistency. , , This represents the threshold for the perceived consistency coefficient;

[0063] Exponential mapping naturally produces a smooth, monotonically decreasing confidence curve; it has a soft threshold characteristic than linear truncation, avoiding threshold abrupt changes; λ controls the error to the confidence decay rate.

[0064] Micro-motion caused by heat may lead to visual measurement With tactile measurement The difference increases, visual measurement With tactile measurement and This will reflect the problem, thereby indirectly capturing the impact of thermal effects on perceived consistency.

[0065] The degree of hand-eye alignment is directly quantified as an important driving factor for visual credibility and provides real-time alignment information for interface migration determination.

[0066] Even a slight increase in visual-tactile lag makes Rapid shrinking and thermal vibrations increase the visual-tactile time lag. This leads to a decrease, thereby driving downstream compensation;

[0067] Using windowing and exponential mapping to quantitatively assess hand-eye consistency transforms the existence of perceptual misalignment into a continuous, thresholdable indicator, facilitating real-time decision-making, reducing visual weight, triggering calibration, or interface transitions.

[0068] The thermal estimation and compensation module includes hand inertial data. The acceleration channel is high-pass filtered to separate non-autonomous jitter components, retaining high-frequency micro-jitter components to obtain a micro-motion sequence; the micro-motion sequence is then calculated within a window. The variance within the range is denoted as the hand microvariance variance. This leads to the hot-state performance factor, expressed as:

[0069] ;

[0070] Represents the hot performance factor. , , , Indicates the preset calibration parameters; Output and To the feedback gain adaptive module and the operational capability evaluation and training module; when When the value is below a preset threshold, a cooling event is triggered.

[0071] Converting temperature rise and micro-motion into quantifiable performance degradation factors, supporting real-time compensation, and integrating with... Together they serve as input for a comprehensive judgment.

[0072] A value close to 1 indicates minimal thermal impact, while a value close to 0 indicates severe thermal impairment of precise control.

[0073] Converting temperature rise and micro-variance variance into a single thermal performance factor and exporting it is beneficial because it quantifies the impact of temperature-micro-fluctuations, providing input for real-time degradation (heat dissipation, rest) and force feedback / scoring correction.

[0074] By reducing the visual trust coefficient, the system can weaken the influence of vision on the final score control; adjusting k by decreasing or increasing it during the transfer phase... This allows for slower weight changes, preventing erroneous actions; hot state makes This reduces the visual-tactile time lag, causing the system to rely on tactile and thermal information.

[0075] The personnel competency evaluation model is based on an LSTM model and uses hot performance factors. Visual energy level tactile energy level Based on the above data, a temporal feature sequence is constructed using the corresponding basic operation scores. In the forward computation, a bias update strategy is implemented, introducing a visual trust coefficient at each time step of the bias term of the input gate. Combined with the baseline bias, a visual trust bias term is obtained. Subsequently, an LSTM single-step forward pass is run to obtain the hidden representation, and the operation comprehensive score is output by the FC layer. The basic operational score is determined by the hot performance factor. and visual energy level With tactile energy The product of the weighted sums is obtained.

[0076] The initial input to the personnel competency evaluation model is the calculated basic operational score, and subsequent inputs are the historical comprehensive operational scores obtained based on the model. .

[0077] The bias update strategy includes presetting a baseline bias and a learning component before initializing the personnel competency evaluation model. It can be set as a scalar and, according to an affine relation, be smoothed by a visual confidence coefficient obtained through a short-time exponential moving average. Candidate biases are obtained by combining the visual bias calculation sub-strategy with the visual bias calculation sub-strategy. Then, the rate of change of the candidate bias relative to the previous bias is limited and passed through a safe interval. Crop the image to obtain the visual correction bias. , used to replace the constant bias of the LSTM input gate, thereby updating the input gate of the personnel competence evaluation model, wherein This represents the standard visual trust coefficient.

[0078] The advantage lies in achieving a virtuous cycle of long-term learning (the model can learn long-term visual reliability) and short-term adaptation (visual-tactile time lag reflects immediate reliability), while avoiding drastic fluctuations in real-time parameters.

[0079] A larger bias makes the change in visual-tactile time lag have a stronger impact on the bias (amplifying / suppressing visual information more quickly), but may lead to model instability;

[0080] A smaller bias makes the bias change smoother but the response is slower; a larger bias results in a faster response but the model is prone to jitter.

[0081] When the visual-tactile lag decreases, This will change accordingly to reduce forward computation, thereby reducing the amount of suspicious visual data written into memory and lowering subsequent misjudgments; a smaller configurable size can be achieved during the transfer phase. With larger Local strategies to avoid premature forgetting or incorrect writing; heat-induced During descent, bias adjustment inhibits the writing of visual information, and training triggers can protect the system.

[0082] Setting the gate bias to the baseline bias plus the learned scaling factor multiplied by (visual-tactile lag − 0.5) creates a centrally symmetric, small adjustment by increasing the bias when the visual-tactile lag is greater than the midpoint and decreasing it when it is less than the midpoint. The midpoint 0.5 can be replaced with the median confidence estimated during training.

[0083] Directly replacing the bias can lead to large, uncontrollable changes, resulting in unstable training / inference. Employing a "baseline + small visual-haptic lag driver + learnable scaling" approach balances long-term learning with immediate adaptation, while also facilitating regularization and amplitude limiting.

[0084] Visual reliability is an immediate assessment of the reliability of a perceptual modality. By determining the adjustable parameter (i.e., the strength of the visual channel) by the visual-tactile time lag, the model's dependence on visual information can be changed immediately when the modality's reliability changes. This is extremely important in the face of visual degradation, fogging, echo, or tactile failure that are common in the confined environment of the deep sea.

[0085] Compared to using visual-tactile lag only as an input feature, directly using visual-tactile lag to scale the visual pathway can more strongly and structurally affect the memory and output of LSTM, thus more effectively avoiding the learning and amplification of bad visual information in control / evaluation.

[0086] The system can learn the typical value of vision in this task / operator / device over a long period of time, while the vision-tactile time lag provides time-varying adjustments. The combination of the two achieves long-term and short-term adaptive capabilities.

[0087] The visual bias calculation sub-strategy is based on the operational warning coefficient. The specific steps to achieve this include:

[0088] Set instability threshold and warning threshold ,when At that time, the visual trust coefficient is taken as its original value;

[0089] when When this occurs, it indicates that personnel operation requires an alert, and the visual trust coefficient is updated according to a decreasing rule, represented as... ,in, Indicates a preset decreasing coefficient; reduces the impact of visual feedback;

[0090] when When this occurs, it indicates that the operator's actions have become unstable. The operator's actions will be forcibly stopped, and the task tolerance will be updated according to the task tolerance rules. ,in, This represents a preset increment coefficient. It increases the tolerance for task precision while reducing the requirements for control precision.

[0091] In this embodiment, Forced decrease when rising It can quickly suppress visual impact; in short-term early warning ( In the lower range, it can first decrease. Instead of immediately adjusting the interface, we should reduce misjudgments. High (triggered by thermal-time delay coupling exponent) will lead to Decline and Increased size, thereby protecting the operator and reducing task crashes.

[0092] The advantage lies in automatically relaxing evaluation criteria and reducing visual impact during the early warning / instability phase, protecting personnel / equipment safety and avoiding misjudgments.

[0093] The operational early warning coefficient The scaled thermal-time delay coupling index and the neuro-mechanical stability index, along with their product, are fused using the hyperbolic tangent function, and then weighted by a weight vector. Allocate contributions from each part and adjust the operational early warning coefficient. Mapped to The calculation formula is expressed as:

[0094] ;

[0095] in, Indicates the neuromechanical stability index. This represents the thermal-time delay coupling index.

[0096] tanh provides bounded output, which is beneficial for threshold determination; interaction terms This can amplify the multiplicative risk of simultaneously increasing both (i.e., the coupling risk is more dangerous).

[0097] The high-order risk indicators of time delay, temperature rise, micro-motion, and time delay coupling are integrated into a single core feature Z(t) for early warning of complex interactive instability. This directly serves the identification of thermal-time delay coupling and the detection of sensing misalignment caused by time delay and migration failure caused by unstable windings.

[0098] The thermal-time delay coupling index is obtained through the cross-modal time delay thermistor coefficient. Time-delay thermosensitive critical value Hot performance factor Hand temperature Hand temperature baseline The calculation formula is as follows: ;in, Indicates the thermal-time delay coupling index. To establish a safe temperature difference scale, the rate of change of time delay is directly quantified with the temperature difference; normalizing it and comparing it with a critical value indicates when the temperature rise is sufficient to affect the sensing / control time delay; the cross-modal time-delay thermistor coefficient is expressed as... .

[0099] The neuro-mechanical stability index is measured by the variance of hand micro-movements. Baseline variance of hand micro-movements Hot performance factor The coupling is obtained, and the calculation formula is expressed as: ,in, This represents the set sensitivity coefficient. The variance ratio reflects the jitter increase, which is multiplied by the exponential term of the delay to amplify the effect of the delay on stability; α controls the delay sensitivity.

[0100] The force feedback gain By setting the base gain Simultaneously extract the perceived consistency coefficient. and the aforementioned hot performance factor Calculate feedback gain ;in, , Indicates the set gain weight

[0101] As a real-time control command, it is sent to the force feedback device via a real-time control bus, such as EtherCAT / RTPS, and the force feedback device adjusts the tactile intensity accordingly.

[0102] Visual unreliability or thermal discrepancies are interpreted as requiring adjustments to the force feedback. The multiplicative structure ensures that the adjustment magnitude is amplified when both defects coexist. If express Increase assist / guidance intensity For example, when vision is impaired, tactile cues can be enhanced to compensate for visual impairment.

[0103] If the goal is to replace vision with touch, increase It can enhance tactile cues when visual impairment is present, helping operators make corrections; however, if the tactile channel itself is noisy, it can increase... This can actually amplify noise and cause control jitter; temporarily adding tactile guidance can help with the transition, or reducing feedback can prevent the operator from becoming overly reliant on the mechanical differences of the new interface; thermal differences can also cause... Low, if increased according to the formula The instability may be amplified when there is significant hand tremor; therefore, this embodiment is designed for use in thermal scenarios. reduce Or limit the gain.

[0104] Automatically adjusts force feedback intensity when visual conditions are unreliable or thermal conditions are poor, preventing noise or micro-jitter from being amplified into dangerous control inputs.

[0105] Adjusting the force feedback intensity via a real-time bus is beneficial for adjusting tactile or force feedback when visual feedback is unreliable or thermally unreliable, avoiding amplifying noise or micro-vibrations into dangerous feedback, and enhancing tactile guidance to compensate for perceptual deficiencies when visual feedback is used.

[0106] The threshold and weight settings can be set by default according to the present invention, or they can be set by those skilled in the art.

[0107] Example 2

[0108] Figure 2 The flowchart of the maneuverability testing method based on deep-sea confined environment simulation of the present invention is shown. Based on the same inventive concept as Embodiment 1, the present invention provides a maneuverability testing method based on deep-sea confined environment simulation, the specific steps of which include:

[0109] S1. Collect visual energy, tactile energy, hand inertial data, hand temperature and device temperature and generate a synchronous data frame;

[0110] S2. Perform time alignment, filtering, and noise reduction on the synchronous data frame;

[0111] S3. Calculate the visual-tactile time delay based on visual and tactile measurements and output the perceptual consistency coefficient;

[0112] S4. Generate a visual trust coefficient based on the perceptual consistency coefficient;

[0113] S5. Calculate the hand micro-motion variance based on hand temperature and equipment temperature and generate thermal performance factor;

[0114] S6. Based on the aforementioned visual trust coefficient, thermal performance factor, visual performance intensity and tactile performance intensity, establish a personnel competency evaluation model to calculate the comprehensive operational score;

[0115] S7. Calculate and apply the force feedback gain based on the perception consistency coefficient and the thermal performance factor.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A system for detecting the ability to operate based on a deep-sea closed environment simulation, characterized by: Comprise: An environment and manipulation data acquisition module for acquiring visual energy metrics, tactile energy metrics, hand inertia data, hand temperature and device temperature and generating a synchronized data frame; A multi-modal synchronization and preprocessing module for time alignment, filtering and denoising the synchronized data frame; A perception consistency evaluation module for calculating visual-tactile time lag based on visual and tactile measurements and outputting a perception consistency coefficient; A visual trust coefficient generation module for generating a visual trust coefficient according to the perception consistency coefficient; A thermal state estimation and compensation module for calculating hand micro-motion variance based on hand temperature and device temperature and generating a thermal state performance factor; An operation capability evaluation and training module for establishing a personnel capability evaluation model based on the visual trust coefficient, thermal state performance factor, visual performance metrics and tactile performance metrics to calculate an operation comprehensive score; A feedback gain adaptive module for calculating and issuing force feedback gain based on the perception consistency coefficient and the thermal state performance factor; The perceptual consistency assessment module calculates visual measurements at each sampling point based on visual and tactile data within a time window. With tactile measurement The root mean square error is used to obtain the visual-tactile time delay. And through preset scale parameters Mapping-perceptual consistency coefficient The visual trust coefficient is obtained by using interval mapping to determine the perceived consistency. ; The thermal estimation and compensation module includes hand inertial data. The acceleration channel is high-pass filtered to separate non-autonomous jitter components, retaining high-frequency micro-jitter components to obtain a micro-motion sequence; the micro-motion sequence is then calculated within a window. The variance within the range is denoted as the hand microvariance variance. This leads to the hot-state performance factor, expressed as: ； represents a hot state performance factor, represents a hand temperature, represents a device temperature, , , , represents a preset calibration parameter; output and to a feedback gain adaptive module and an operating ability evaluation and training module; when is less than a preset threshold, a cooling event is triggered.

2. The deep-sea closed environment simulation based manipulation ability detection system according to claim 1, characterized in that: The personnel capability evaluation model is based on an LSTM model, constructs a time sequence feature sequence through a hot state performance factor , a visual performance measure , a tactile performance measure , and a corresponding basic operation score obtained based on the above data. In the forward calculation, the bias update strategy is carried, the visual trust coefficient is introduced in each time step of the input gate bias item, the baseline bias is combined to obtain the visual trust bias item, and then the LSTM single step sequence forward is run to obtain the hidden representation and the comprehensive score is output by the FC layer operation .

3. The deep sea hyperbaric environment simulation based manipulation ability detection system according to claim 2, wherein: The bias updating strategy comprises presetting a baseline bias before initialization of the personnel capability evaluation model and a learning component , and combining the affine relationship with a smoothed visual trust coefficient obtained by a short-term exponential moving average , and combining the affine relationship with a smoothed visual trust coefficient obtained by a short-term exponential moving average , and combining the affine relationship with a smoothed visual trust coefficient obtained by a short-term exponential moving average , and combining the affine relationship with a smoothed visual trust coefficient obtained by a short-term exponential moving average , and combining the affine relationship with a smoothed visual trust coefficient obtained by a short-term exponential moving average , and combining the affine relationship with a smoothed visual trust coefficient obtained by a short-term exponential moving average 4. The deep sea hyperbaric environment simulation based manipulation ability detection system according to claim 3, wherein: The visual bias calculation sub-strategy is based on an operating alert coefficient The implementation, the specific steps include: Setting instability threshold and early warning threshold When the visual trust coefficient takes the original value; When the visual trust coefficient is updated according to a decreasing rule, expressed as wherein, represents a preset decreasing coefficient; When the personnel operation is forcibly stopped, and the task tolerance is updated according to the task tolerance increasing rule wherein, represents a preset incremental coefficient.

5. The deep-sea closed environment simulation-based manipulation ability detection system according to claim 4, characterized in that: The operation early warning coefficient The hyperbolic tangent function is fused with the scaled thermal-time delay coupling index and the neural-mechanical stability index and the product term of the two, and a weight vector The operation early warning coefficient is mapped to , and the calculation formula is represented as: ； wherein, denotes the neuro-mechanical stability index, denotes the thermo-time-lag coupling index, denotes a preset operating warning weight.​​ 6. The deep-sea closed environment simulation-based manipulation ability detection system according to claim 5, characterized in that: the heat-time lag coupling index is obtained by a cross-modal time-lag heat-sensitive coefficient , a time-lag heat-sensitive critical value , a thermal state performance factor , a hand temperature , a hand temperature baseline , a calculation formula is represented as: ; wherein, the heat-time lag coupling index, a safety temperature difference scale, the cross-modal time-lag heat-sensitive coefficient is represented as .

7. The deep sea hyperbaric environment simulation based manipulation ability detection system according to claim 6, wherein: The neuro-mechanical stability index is derived by hand micro-motion variance , hand micro-motion baseline variance , thermal performance factor is coupled, and the calculation formula is expressed as: , wherein represents the set sensitivity coefficient.

8. The deep sea hyperbaric environment simulation based manipulation ability detection system according to claim 7, characterized in that: The base operation score is obtained by multiplying the product of the hot state performance factor and the visual performance metric and the haptic performance metric weighted sums.

9. The deep sea hyperbaric environment simulation based manipulation ability detection system according to claim 8, wherein: The force feedback gain By setting a base gain While extracting the perceptual consistency coefficient And the thermal performance factor Calculate the feedback gain ; wherein, , Indicates the set gain weight; As real-time control commands, the force feedback device is instructed to adjust the haptic intensity by the real-time control bus.

10. A method for detecting the operating ability based on the simulation of the deep-sea closed environment, used for executing the system for detecting the operating ability based on the simulation of the deep-sea closed environment as claimed in any one of claims 1-9, characterized in that: Specific steps include: S1, acquiring visual energy metrics, tactile energy metrics, hand inertia data, hand temperature and device temperature and generating a synchronized data frame; S2, time alignment, filtering and denoising the synchronized data frame; S3, calculating visual-tactile time lag based on visual and tactile measurements and outputting a perception consistency coefficient; S4, generating a visual trust coefficient according to the perception consistency coefficient; S5, calculating hand micro-motion variance based on hand temperature and device temperature and generating a thermal state performance factor; S6, establishing a personnel capability evaluation model based on the visual trust coefficient, thermal state performance factor, visual performance metrics and tactile performance metrics to calculate an operation comprehensive score; S7, calculating and issuing force feedback gain based on the perception consistency coefficient and the thermal state performance factor.

Images