Underwater glasses based on multi-modal interaction and photographing control method thereof

Abstract

This invention proposes a diving goggles based on multimodal interaction and its image capture control method. It belongs to the field of intelligent wearable devices and underwater interaction technology. The method includes: simultaneously acquiring the user's eye movement trajectory, facial electromyography signals, head posture data, and underwater environmental visual images through a multimodal sensor group built into the diving goggles, generating four sets of biological and environmental signal data; performing spatiotemporal alignment preprocessing on the four sets of signal data to construct a joint feature vector of user intent and environmental state; through the coordinated interaction of the four sets of signals (eye movement, electromyography, posture, and vision), this method can accurately capture the user's shooting intent in complex underwater environments, effectively reducing the false trigger rate caused by water flow interference or misoperation, and improving the accuracy of shooting commands.

Description

Technical Field

[0001] This invention proposes a diving goggles based on multimodal interaction and its photo control method, belonging to the field of intelligent wearable devices and underwater interaction technology. Background Technology

[0002] The current camera function of diving goggles has significant limitations. The interaction method relies heavily on single blink detection, which is easily affected by environmental factors such as strong underwater light and water flow, resulting in a high false trigger rate.

[0003] Meanwhile, existing technologies lack a deep understanding of the user's true intentions, often resulting in blank or blurry images due to the inability to determine the focus of attention, making it difficult to guarantee image quality. More importantly, dynamic factors such as head shaking and target movement are not effectively compensated for, with photo blur rates consistently exceeding 40%, severely limiting the underwater photography experience.

[0004] From an ethical and privacy perspective, existing solutions lack protection mechanisms for companions' faces and vulnerable ecological areas, posing a dual risk of privacy breaches and ecological disturbance. From a technical perspective, most publicly available solutions only focus on isolated functions such as "blinking to take a picture" and "voice control," failing to overcome the limitations of "more sensitive shutter speeds."

[0005] While the "twice-blinking for a photo" concept in the technical disclosure document is inspiring, the industry has not yet extended it to a trusted architecture that integrates multimodal biosignal fusion, underwater context awareness, and ethical adaptation. Therefore, there is an urgent need for a multimodal interaction method that deeply integrates eye movement, electromyography, posture, and vision to achieve a paradigm shift in intelligent photography that "understands intent, makes judgments, and adheres to ethics," providing a new generation of core technologies for trusted interaction in underwater photography. Summary of the Invention

[0006] This invention provides a diving goggles based on multimodal interaction and a method for controlling its photographing, in order to solve the problems mentioned in the background art above: The present invention proposes a method for controlling the taking of photos with diving goggles based on multimodal interaction, the method comprising: S1. Through the multimodal sensor group built into the diving goggles, the user's eye movement trajectory, facial electromyography signals, head posture data and underwater environmental visual images are collected simultaneously to generate four kinds of biological and environmental signal data; based on the four kinds of signal data, spatiotemporal alignment preprocessing is performed to construct a joint feature vector of user intent and environmental state; S2. Multimodal fusion decision based on joint feature vectors. The user's shooting intention is cross-validated by three indicators: eye movement dwell time, electromyographic tension, and head stability. When all three exceed the preset threshold and there is a clear target in the visual image, an effective shooting trigger signal is generated. S3. The high-precision optical image stabilization module is activated based on the valid shooting trigger signal. The imaging angle is corrected in real time by combining head posture data. At the same time, the shutter speed and ISO parameters are dynamically adjusted by the target motion vector prediction algorithm in the visual image to generate anti-shake optimized shooting instructions. Real-time recognition is performed on fragile ecological areas or companion faces. When such targets are detected, the system automatically switches to ethical protection mode and pauses shooting. S4. The optical image stabilization module executes anti-shake optimization shooting instructions to complete underwater image acquisition and generate raw image data; multimodal information embedding processing is performed on the raw image data, and the eye movement focus coordinates, electromyography confirmation signal intensity and ethical protection mark at the shooting time are written as metadata into the image file. At the same time, the pre-set AI species recognition model is called to label the marine life in the image and generate enhanced image data with species annotation. S5. Based on the enhanced image data, perform quality assessment and feedback adjustment. When the image blur exceeds the preset standard, automatically trigger the secondary reshoot process and adjust the shooting parameters. Store the finally confirmed enhanced image data in the local database of the diving goggles, and upload it to the cloud server through an encrypted channel to generate a reliable intelligent shooting full-process log.

[0007] The present invention proposes a diving goggles based on multimodal interaction, the diving goggles comprising: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0008] The beneficial effects of this invention are as follows: Through the coordinated interaction of four signals—eye movement, electromyography, posture, and vision—this method can accurately capture the user's shooting intentions in complex underwater environments, effectively reducing the false trigger rate caused by water flow interference or misoperation, and improving the accuracy of shooting commands. Simultaneously, the multimodal fusion decision-making mechanism, combined with dynamic image stabilization and target tracking technology, significantly reduces image blurring caused by head shaking or target movement, resulting in a substantial improvement in image clarity. From an ethical perspective, the system automatically identifies and protects companions' faces and vulnerable ecological areas, avoiding privacy leaks and ecological disturbance risks, and enhancing the compliance of underwater photography. Furthermore, this method achieves high-quality imaging through high-precision optical image stabilization and dynamic parameter adjustment, and automatically embeds tag information based on AI species recognition, enhancing the knowledge and practicality of the captured content. Overall, this technology overcomes the limitations of traditional diving goggles, which have limited shooting functions and poor reliability, providing users with a more intelligent, reliable, and ethically compliant underwater photography experience. Attached Figure Description

[0009] Figure 1This is a diagram illustrating the steps of the method described in this invention. Detailed Implementation

[0010] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0011] One embodiment of the present invention, such as Figure 1 As shown, a method for controlling the taking of photos using diving goggles based on multimodal interaction is described, the method comprising: S1. Through the multimodal sensor group built into the diving goggles, the user's eye movement trajectory, facial electromyography signals, head posture data and underwater environmental visual images are collected simultaneously to generate four kinds of biological and environmental signal data; based on the four kinds of signal data, spatiotemporal alignment preprocessing is performed to construct a joint feature vector of user intent and environmental state; S2. Multimodal fusion decision-making is performed based on joint feature vectors. The user's shooting intention is cross-validated by three indicators: eye movement dwell time, electromyographic tension, and head stability. When all three exceed the preset threshold and there is a clear target in the visual image, an effective shooting trigger signal is generated. Invalid signals are suppressed and interference data caused by water flow impact or misoperation is filtered out. S3. The high-precision optical image stabilization module is activated based on the valid shooting trigger signal. The imaging angle is corrected in real time by combining head posture data. At the same time, the shutter speed and ISO parameters are dynamically adjusted by the target motion vector prediction algorithm in the visual image to generate anti-shake optimized shooting instructions. Real-time recognition is performed on fragile ecological areas or companion faces. When such targets are detected, the system automatically switches to ethical protection mode and pauses shooting. S4. The optical image stabilization module executes anti-shake optimization shooting instructions to complete underwater image acquisition and generate raw image data; multimodal information embedding processing is performed on the raw image data, and the eye movement focus coordinates, electromyography confirmation signal intensity and ethical protection mark at the shooting time are written as metadata into the image file. At the same time, the pre-set AI species recognition model is called to label the marine life in the image and generate enhanced image data with species annotation. S5. Based on the enhanced image data, perform quality assessment and feedback adjustment. When the image blur exceeds the preset standard, automatically trigger the secondary reshoot process and adjust the shooting parameters. Store the finally confirmed enhanced image data in the local database of the diving goggles, and upload it to the cloud server through an encrypted channel to generate a reliable intelligent shooting full-process log.

[0012] The working principle and effects of the above technical solution are as follows: Multimodal signal cross-verification of shooting intent improves the accuracy of underwater shooting triggers, reduces invalid shots caused by misoperation, and avoids resource waste caused by accidental triggering due to water flow or unintentional movements. The optical image stabilization module, combined with dynamic parameter adjustment, enhances the clarity of underwater images and reduces the probability of image blurring caused by head shaking or target movement. AI species tagging enhances the practical value of shooting, making diving records both entertaining and educational. Dual protection through local storage and encrypted upload enhances data security and reduces the risk of data loss in unexpected situations. A quality assessment and reshoot mechanism prevents the generation of unusable footage due to improper shooting parameters or environmental interference. An ethical protection mode avoids damaging fragile ecological areas or infringing on the privacy of fellow divers, satisfying the shooting needs of divers while also considering environmental protection and privacy, further enhancing the safety and convenience of the diving experience.

[0013] In one embodiment of the present invention, S1 includes: S11. The multimodal sensor group built into the diving goggles works in a different way: the eye movement sensor captures the user's eye movement trajectory, the electromyography sensor collects facial electromyography signals, the posture sensor records head posture data, and the vision sensor captures visual images of the underwater environment to generate initial four-fold biological and environmental signal data. S12. Perform synchronous calibration on the initial four-fold signal data to unify the data acquisition frequency of each sensor and eliminate time delay differences between different devices. S13. Perform spatiotemporal alignment preprocessing on the calibrated quadruple signal data to unify the spatial coordinate system of the data and regularize the temporal arrangement of the data. S14. Extract key features of user behavior and core information of the underwater environment from the spatiotemporally aligned signal data, and filter effective data dimensions; integrate the filtered behavioral features and environmental information to construct a joint feature vector of user intent and environmental state.

[0014] The working principle and effects of the above technical solution are as follows: By using multiple sensors to collect four signals, the comprehensiveness of underwater scene data acquisition is improved, allowing user behavior and environmental conditions to be fully captured. Synchronous calibration of the unified acquisition frequency reduces the impact of time delays between different devices, avoiding judgment biases caused by data asynchrony. Spatiotemporal alignment normalizes coordinates and time sequences, enhancing data consistency and reducing information conflicts in subsequent processing. Valid features are selected and integrated to construct joint vectors, improving data relevance and avoiding redundant information that slows down processing efficiency. This provides accurate data support for core functions such as subsequent shooting triggers and navigation guidance, and makes interactive responses during diving more aligned with actual user needs, further reducing the risk of operational errors caused by inaccurate data.

[0015] In one embodiment of the present invention, step S14 includes: After receiving spatiotemporally aligned signal data, user behavior-related data and underwater environment-related data are separated, and the core representation information of each type of data is extracted to generate a preliminary feature set. The initial feature set is screened for dimensions to remove redundant dimensions that have excessive data fluctuations or are irrelevant to the shooting intention, while retaining key and effective features to form an effective feature subset. The user behavior features and underwater environment features in the effective feature subset are quantified separately, and the data scale and measurement standard of the two types of features are unified to generate quantified feature data. Based on the temporal correlation of data, the quantified user behavior features are organically integrated with underwater environment features to eliminate conflicts and interference between features and generate a fused feature set. Based on the fused feature set, the inherent relationships between features are sorted out, and a joint feature vector covering user intent and the real-time state of the underwater environment is constructed.

[0016] The working principle and effects of the above technical solution are as follows: Data is split and core information is extracted, improving the targeting of feature capture and reducing interference from irrelevant information in subsequent processing. Dimensional screening eliminates redundant dimensions, reducing the computational burden of data processing and preventing redundant data from slowing down response efficiency. Quantitative processing unifies data standards, enhancing the consistency of feature data and reducing conflicts during fusion. Organic integration of quantified features improves the accuracy of joint vectors and avoids judgment bias caused by feature contradictions. This solution accurately meets users' actual needs for shooting and navigation, and can present the underwater environment status in real time, providing solid data support for functions such as safety warnings and AI species recognition, making diving interactions smoother and reducing the troubles caused by operational errors.

[0017] In one embodiment of the present invention, S2 includes: S21. Taking the joint feature vector of user intent and environmental state as the core, carry out multimodal fusion decision-making and decompose three core evaluation indicators: eye movement dwell time, facial electromyographic tension and head posture stability. S22. Set qualification standards for three core evaluation indicators and clarify the quantitative threshold values ​​for each indicator; S23. Cross-validate the three indicators and analyze the clarity and completeness of the target outline in the underwater environment visual image to comprehensively judge the user's shooting intention. S24. When all three indicators exceed the preset threshold and there is a clear target in the visual image, a valid shooting trigger signal is generated. S25. Suppress signals that do not meet the triggering conditions, filter out sensor fluctuation data caused by water flow impact and invalid signals caused by user misoperation, and purify the shooting triggering environment.

[0018] The working principle and effects of the above technical solution are as follows: It decomposes three core indicators and cross-validates them to improve the accuracy of underwater shooting intention judgment and reduce false triggers caused by water flow impact or misoperation. Setting quantitative thresholds provides clear standards for judgment, reducing errors caused by subjective bias. Simultaneous analysis of target outline clarity and integrity avoids invalid triggers when there is no clear subject, reducing the generation of unusable shots. It suppresses signals that do not meet the conditions, purifies the shooting trigger environment, and avoids unnecessary resource consumption. This solution accurately meets the user's need for blinking to capture images, makes underwater operation more convenient and smooth, protects device battery power, and makes the shooting response during diving more realistic, allowing users to capture the desired image without repeated adjustments, further enhancing the convenience and experience of underwater photography.

[0019] In one embodiment of the present invention, S3 includes: S31: The effective shooting trigger signal is transmitted to the high-precision optical image stabilization module, which starts the module's running program and enters the shooting preparation state. S32. Retrieve head posture data, analyze the amplitude and direction of user head movement in real time, and dynamically correct the shooting angle. S33. Analyze the target motion trajectory in the underwater environment visual image, deduce the target movement trend through the algorithm, and obtain the target motion vector; S34. Dynamically adjust shutter speed and ISO parameters based on the target motion vector, and combine the angle correction results to form an anti-shake optimized shooting command; S35. Real-time recognition of scenes and people in visual images, distinguishing features of vulnerable ecological areas and the facial contours of companions. When such targets are detected, the system automatically switches to ethical protection mode and pauses the shooting action.

[0020] The working principle and effects of the above technical solution are as follows: It activates a high-precision optical image stabilization module to correct the imaging angle, improving the stability of underwater images and reducing blur caused by head movement. It analyzes the target's movement trajectory and dynamically adjusts shooting parameters, enhancing the ability to capture moving marine life and reducing focus loss due to target movement. It identifies sensitive targets in real time and switches to an ethical protection mode to avoid damaging fragile ecosystems or infringing on companions' privacy, reducing adverse effects caused by improper shooting. This allows divers to easily capture clear underwater images while also considering environmental protection and privacy, making the shooting process more worry-free and compliant, further enhancing the practicality and experience of underwater photography, and avoiding regrets due to complex environments or improper operation.

[0021] In one embodiment of the present invention, S32 includes: Retrieve raw head posture data recorded by the diving goggles attitude sensor to obtain real-time updated information on head position and angle changes; Fluctuation filtering is applied to the raw head posture data to remove sudden abnormal data caused by water flow impact and retain the effective posture data with stable changes. Analyze the swaying features in the effective attitude data, extract the peak amplitude and direction change period of the head swaying, and generate attitude fluctuation feature data; Based on attitude fluctuation feature data, the deviation value from the ideal shooting angle is calculated, and real-time angle correction parameters are generated. The angle correction parameters are transmitted to the imaging adjustment unit to adjust the lens pointing angle in real time, thus counteracting the effect of head shaking on imaging.

[0022] The working principle and effects of the above technical solution are as follows: It retrieves real-time head posture data, ensuring angle correction closely matches actual movement and improving the clarity of underwater footage. It filters out sudden abnormal data caused by water currents, reducing the impact of interference factors on the correction effect and lowering the probability of blurry images due to data distortion. It accurately extracts the amplitude and direction characteristics of head movement, enhancing the targeted nature of angle correction and avoiding shooting deviations caused by blind adjustments. It calculates the ideal angle deviation in real time and adjusts the lens direction, effectively counteracting the impact of head movement and reducing out-of-focus shots. This allows divers to capture clear images underwater without deliberately remaining still, adapts to shooting needs in complex water current environments, makes the blink-triggered photo function more practical, further improving the convenience and quality of underwater photography and avoiding regrets due to unstable posture.

[0023] In one embodiment of the present invention, step S4 includes: S41, the high-precision optical image stabilization module receives anti-shake optimization shooting instructions, performs underwater image acquisition operations, captures target detail information, and generates raw image data; S42. Extract the associated information of the shooting time, including eye-tracking focal coordinates, electromyography confirmation signal intensity, ethical protection markers, etc.; write the extracted associated information as metadata into the file corresponding to the original image data to improve the image information dimensions. S43. Run the preset AI species recognition model, scan the marine life features in the original image, and match them with information from the species database; S44. Add species tags to marine life in the image, and merge the original image data, metadata and species tags to generate enhanced image data with species annotations.

[0024] The working principle and effects of the above technical solution are as follows: The optical image stabilization module accurately captures images, improving the detail reproduction of underwater scenes and reducing image quality loss caused by water currents or shaking. Embedded metadata related to the shooting moment enriches the image information dimensions, avoiding the loss of key background information during subsequent tracing. An AI species recognition model scans and matches marine life characteristics, enhancing the knowledge and interest of the images and reducing blind spots in understanding unfamiliar creatures during diving. The fusion of multiple types of information to generate enhanced images allows divers to easily preserve underwater beauty and quickly learn about relevant biological information after shooting, making underwater photography both a record and a learning experience, further enhancing the underwater exploration experience and avoiding the limitations of simply taking photos without added value.

[0025] In one embodiment of the present invention, S43 includes: S431. Start the preset AI species recognition model, load the feature extraction algorithm and computing resources required for model operation, and generate a model ready signal; S432. Perform biological region screening on the original image data, locate regions whose pixel gray values ​​and texture features match the characteristics of marine organisms, and generate biological candidate region data. S433. Extract key features from biological candidate region data, including key features such as contour morphology, color distribution, and texture details, to generate standardized marine biological feature vectors. S434. Retrieve the local pre-set species database, obtain massive marine organism feature template data, and establish a feature comparison dataset; S435. Compare the standardized marine biological feature vectors with the feature comparison dataset dimension by dimension, calculate the feature overlap, and generate species matching similarity data.

[0026] The working principle and effects of the above technical solution are as follows: It initiates the AI ​​species recognition model and loads the necessary resources, improving the stability of model operation and reducing startup failures or lag. It accurately screens biological areas, locating ranges that match marine biological characteristics, reducing interference from irrelevant backgrounds, and preventing misidentification. It extracts core features to generate standardized vectors, enhancing the consistency of feature matching and reducing misjudgments caused by inconsistent feature formats. It accesses a local species database to establish a comparison set, improving recognition response speed and reducing reliance on the network. It calculates feature overlap dimensionally, improving the accuracy of marine life identification and avoiding the embarrassment of misidentifying species. This allows divers to understand the biological information they encounter in real time, making underwater exploration both fun and educational, further enriching the diving experience and reducing confusion about unfamiliar marine life.

[0027] In one embodiment of the present invention, S345 includes: Receive standardized marine biological feature vectors and feature comparison datasets, define core comparison dimensions, including contour, color and texture, and generate a dimension index list; Following the order of the dimension index list, extract the feature values ​​of the corresponding dimensions of the two types of data one by one to form a set of dimension feature pairs; perform numerical difference calculation on each set of dimension feature pairs to obtain the feature deviation data of each dimension. By combining the importance weights of marine biological identification features, the deviation data of each dimension are weighted and fused to generate a comprehensive deviation coefficient; Based on the comprehensive deviation coefficient, the feature overlap ratio is converted to quantify the matching degree of different marine species and generate species matching similarity data.

[0028] The working principle and effects of the above technical solution are as follows: Core comparison dimensions such as contour, color, and texture are defined to improve the targeting of feature comparison and reduce interference from irrelevant dimensions on the recognition results. Corresponding dimension feature values ​​are extracted sequentially to form a pairing set, enhancing the systematic nature of the comparison process and avoiding errors caused by computational chaos. The degree of deviation in each dimension is obtained through numerical difference calculations, making feature differences more intuitive and reducing the probability of fuzzy judgments. Weighted fusion is performed by combining feature importance weights to improve the rationality of the comprehensive evaluation and avoid misjudgments caused by a single dimension dominating. The overlap ratio is calculated based on the comprehensive deviation coefficient to enhance the accuracy of species matching and reduce the occurrence of misidentifying marine life. This allows divers to quickly obtain accurate information about the organisms they see underwater, while also making the diving process both exploratory and learning-oriented, avoiding the confusion of unclear understanding of unfamiliar marine life, making underwater AI recognition functions more practical and reliable, and further enriching the knowledge and fun of the diving experience.

[0029] In one embodiment of the present invention, step S5 includes: S51. Start the image quality assessment program to detect key indicators of the enhanced image data, including sharpness, contrast and color reproduction. S52. Compare the detection results with the preset quality standards. When the image blur exceeds the preset standards, automatically start the secondary reshoot process and simultaneously adjust the shutter speed, ISO and image stabilization parameters. S53. After the enhanced image data from the second reshoot passes quality assessment, it is stored in the local database of the diving goggles, thus completing local data retention. S54. Establish an encrypted transmission channel to upload the final confirmed enhanced image data to the cloud server to achieve off-site data backup; S55. Integrate key information from the entire shooting process, including signal acquisition, intent judgment, shooting execution, data processing, storage and uploading, and generate a reliable intelligent shooting process log.

[0030] The working principle and effects of the above technical solution are as follows: It detects key indicators such as sharpness and contrast, improving the quality of underwater images and reducing blurring or color distortion caused by water currents and shaking. When blur exceeds the acceptable level, it automatically reshoots and adjusts parameters to avoid missing optimal shooting scenes and reduce the hassle of manual reshoots. Local storage combined with encrypted cloud uploads enhances image data security and reduces data loss due to equipment malfunction or loss. It integrates key information throughout the entire process to generate logs, making each stage of shooting traceable and preventing unverifiable issues later. This allows divers to easily obtain high-quality, species-annotated images while confidently storing backup data, balancing shooting quality and data security, making the underwater photography experience more worry-free and secure, and preventing quality issues or data risks from affecting the enjoyment of exploration.

[0031] In one embodiment of the present invention, S54 includes: Extract the final confirmed enhanced image data, integrate the image ontology, metadata and species tag information, and generate a unified format upload data package; Activate the built-in encrypted transmission module of the diving goggles to generate a unique dynamic encryption key and establish a point-to-point secure transmission link; It detects the strength and bandwidth of underwater wireless communication signals, dynamically adjusts the fragment size of uploaded data packets, and adapts to complex underwater communication environments. The encrypted upload data packets are sent to the cloud server sequentially through a secure transmission link in the order of fragments, and the transmission status of each fragment is synchronized in real time. After receiving all the fragmented data packets, the cloud server performs combination splicing and integrity verification. Once the verification is successful, the data is stored and an off-site backup confirmation signal is generated.

[0032] The working principle and effects of the above technical solution are as follows: Integrating image ontology, metadata, and species tags to generate a unified data packet improves the regularity of data uploads and avoids transmission chaos caused by fragmented information. Dynamic encryption keys combined with point-to-point links enhance data transmission security and reduce the risk of privacy or biological information leakage. Detecting communication signals and bandwidth adjusts fragment size, improving transmission adaptability to complex underwater environments and reducing the probability of transmission interruptions or stuttering. Fragmented transmission and synchronized status make the transmission process clear and traceable, preventing irreparable fragment loss. Cloud-based stitching and verification ensure data integrity, preventing data corruption and unusable data after backup. It can securely preserve underwater images with species annotations and adapt to special underwater communication environments, making data backup reliable and worry-free, eliminating concerns about transmission problems or information security, and further enhancing peace of mind regarding diving data preservation.

[0033] One embodiment of the present invention provides a diving goggle based on multimodal interaction, the diving goggle comprising: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0034] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for controlling the taking of photos using diving goggles based on multimodal interaction, characterized in that, The method includes: S1. Through the multimodal sensor group built into the diving goggles, the user's eye movement trajectory, facial electromyography signals, head posture data and underwater environmental visual images are collected simultaneously to generate four kinds of biological and environmental signal data; based on the four kinds of signal data, spatiotemporal alignment preprocessing is performed to construct a joint feature vector of user intent and environmental state; S2. Multimodal fusion decision based on joint feature vectors. The user's shooting intention is cross-validated by three indicators: eye movement dwell time, electromyographic tension, and head stability. When all three exceed the preset threshold and there is a clear target in the visual image, an effective shooting trigger signal is generated. S3. The high-precision optical image stabilization module is activated based on the valid shooting trigger signal. The imaging angle is corrected in real time by combining head posture data. At the same time, the shutter speed and ISO parameters are dynamically adjusted by the target motion vector prediction algorithm in the visual image to generate anti-shake optimized shooting instructions. Real-time recognition is performed on fragile ecological areas or companion faces. When such targets are detected, the system automatically switches to ethical protection mode and pauses shooting. S4. The optical image stabilization module executes anti-shake optimization shooting instructions to complete underwater image acquisition and generate raw image data; multimodal information embedding processing is performed on the raw image data, and the eye movement focus coordinates, electromyography confirmation signal intensity and ethical protection mark at the shooting time are written as metadata into the image file. At the same time, the pre-set AI species recognition model is called to label the marine life in the image and generate enhanced image data with species annotation. S5. Based on the enhanced image data, perform quality assessment and feedback adjustment. When the image blur exceeds the preset standard, automatically trigger the secondary reshoot process and adjust the shooting parameters. Store the finally confirmed enhanced image data in the local database of the diving goggles, and upload it to the cloud server through an encrypted channel to generate a reliable intelligent shooting full-process log.

2. The method for controlling the taking of photos using diving goggles based on multimodal interaction according to claim 1, characterized in that, S1 includes: S11. The multimodal sensor group built into the diving goggles works in a different way: the eye movement sensor captures the user's eye movement trajectory, the electromyography sensor collects facial electromyography signals, the posture sensor records head posture data, and the vision sensor captures visual images of the underwater environment to generate initial four-fold biological and environmental signal data. S12. Perform synchronous calibration on the initial four-fold signal data to unify the data acquisition frequency of each sensor and eliminate time delay differences between different devices. S13. Perform spatiotemporal alignment preprocessing on the calibrated quadruple signal data to unify the spatial coordinate system of the data and regularize the temporal arrangement of the data. S14. Extract key features of user behavior and core information of the underwater environment from the spatiotemporally aligned signal data, and filter effective data dimensions; integrate the filtered behavioral features and environmental information to construct a joint feature vector of user intent and environmental state.

3. The method for controlling the taking of photos using diving goggles based on multimodal interaction according to claim 1, characterized in that, The S2 includes: S21. Taking the joint feature vector of user intent and environmental state as the core, carry out multimodal fusion decision-making and decompose three core evaluation indicators: eye movement dwell time, facial electromyographic tension and head posture stability. S22. Set qualification standards for three core evaluation indicators and clarify the quantitative threshold values ​​for each indicator; S23. Cross-validate the three indicators and analyze the clarity and completeness of the target outline in the underwater environment visual image to comprehensively judge the user's shooting intention. S24. When all three indicators exceed the preset threshold and there is a clear target in the visual image, a valid shooting trigger signal is generated. S25. Suppress signals that do not meet the triggering conditions, filter out sensor fluctuation data caused by water flow impact and invalid signals caused by user misoperation, and purify the shooting triggering environment.

4. The method for controlling the taking pictures of diving goggles based on multimodal interaction according to claim 1, characterized in that, The S3 includes: S31: The effective shooting trigger signal is transmitted to the high-precision optical image stabilization module, which starts the module's running program and enters the shooting preparation state. S32. Retrieve head posture data, analyze the amplitude and direction of user head movement in real time, and dynamically correct the shooting angle. S33. Analyze the target motion trajectory in the underwater environment visual image, deduce the target movement trend through the algorithm, and obtain the target motion vector; S34. Dynamically adjust shutter speed and ISO parameters based on the target motion vector, and combine the angle correction results to form an anti-shake optimized shooting command; S35. Real-time recognition of scenes and people in visual images, distinguishing features of vulnerable ecological areas and the facial contours of companions. When such targets are detected, the system automatically switches to ethical protection mode and pauses the shooting action.

5. The method for controlling the taking of photos using diving goggles based on multimodal interaction according to claim 4, characterized in that, S32 includes: Retrieve raw head posture data recorded by the diving goggles attitude sensor to obtain real-time updated information on head position and angle changes; Fluctuation filtering is applied to the raw head posture data to remove sudden abnormal data caused by water flow impact and retain the effective posture data with stable changes. Analyze the swaying features in the effective attitude data, extract the peak amplitude and direction change period of the head swaying, and generate attitude fluctuation feature data; Based on attitude fluctuation feature data, the deviation value from the ideal shooting angle is calculated, and real-time angle correction parameters are generated. The angle correction parameters are transmitted to the imaging adjustment unit to adjust the lens pointing angle in real time, thus counteracting the effect of head shaking on imaging.

6. The method for controlling the taking of photos using diving goggles based on multimodal interaction according to claim 1, characterized in that, The S4 includes: S41, the high-precision optical image stabilization module receives anti-shake optimization shooting instructions, performs underwater image acquisition operations, captures target detail information, and generates raw image data; S42. Extract the relevant information of the shooting time, and write the extracted relevant information as metadata into the file corresponding to the original image data to improve the image information dimensions. S43. Run the preset AI species recognition model, scan the marine life features in the original image, and match them with information from the species database; S44. Add species tags to marine life in the image, and merge the original image data, metadata and species tags to generate enhanced image data with species annotations.

7. The method for controlling the taking of photos using diving goggles based on multimodal interaction according to claim 6, characterized in that, S43 includes: S431. Start the preset AI species recognition model, load the feature extraction algorithm and computing resources required for model operation, and generate a model ready signal; S432. Perform biological region screening on the original image data, locate regions whose pixel gray values ​​and texture features match the characteristics of marine organisms, and generate biological candidate region data. S433. Extract key features from biological candidate region data and generate standardized marine biological feature vectors; S434. Retrieve the local pre-set species database, obtain massive marine organism feature template data, and establish a feature comparison dataset; S435. Compare the standardized marine biological feature vectors with the feature comparison dataset dimension by dimension, calculate the feature overlap, and generate species matching similarity data.

8. The method for controlling the taking of photos using diving goggles based on multimodal interaction according to claim 7, characterized in that, The S345 includes: Receive standardized marine biological feature vectors and feature comparison datasets, define core comparison dimensions, including contour, color and texture, and generate a dimension index list; Following the order of the dimension index list, extract the feature values ​​of the corresponding dimensions of the two types of data one by one to form a set of dimension feature pairs; perform numerical difference calculation on each set of dimension feature pairs to obtain the feature deviation data of each dimension. By combining the importance weights of marine biological identification features, the deviation data of each dimension are weighted and fused to generate a comprehensive deviation coefficient; Based on the comprehensive deviation coefficient, the feature overlap ratio is converted to quantify the matching degree of different marine species and generate species matching similarity data.

9. The method for controlling the taking of photos using diving goggles based on multimodal interaction according to claim 1, characterized in that, The S5 includes: S51. Start the image quality assessment program to detect key indicators of the enhanced image data; S52. Compare the detection results with the preset quality standards. When the image blur exceeds the preset standards, automatically start the secondary reshoot process and simultaneously adjust the shutter speed, ISO and image stabilization parameters. S53. After the enhanced image data from the second reshoot passes quality assessment, it is stored in the local database of the diving goggles, thus completing local data retention. S54. Establish an encrypted transmission channel to upload the final confirmed enhanced image data to the cloud server to achieve off-site data backup; S55 integrates key information from the entire shooting process to generate a reliable intelligent shooting process log.

10. Diving goggles based on multimodal interaction, characterized in that, The diving goggles include: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

Images