Picture adjustment method and device of head-mounted display equipment, equipment and storage medium

Abstract

The embodiment of the invention provides a picture adjustment method and device for head-mounted display equipment, the equipment and a storage medium, and belongs to the field of head-mounted display equipment. The method comprises the following steps: acquiring a periocular electromyographic signal of a user wearing the head-mounted display equipment; performing recognition processing on the watching intention of the user based on the characteristic quantity of the periocular electromyographic signal to obtain a target watching intention of the user; and according to the target watching intention, adjusting a display position of a virtual picture in the head-mounted display device. According to the embodiment of the invention, the accuracy, convenience and efficiency of adjusting the display position of the virtual picture in the head-mounted display equipment can be improved, the operation burden of a user is relieved, and meanwhile, the intelligent level of the head-mounted display equipment in the aspect of human-computer interaction is improved.

Description

Technical Field

[0001] This invention relates to the field of head-mounted display technology, and in particular to a method, apparatus, device, and storage medium for adjusting the image of a head-mounted display device. Background Technology

[0002] With the rapid development of Augmented Reality (AR) technology, AR devices, as a type of head-mounted display, have been widely used in various fields such as education and training, industrial assistance, medical care, and entertainment. By overlaying virtual images onto the user's field of vision, these head-mounted display devices achieve an immersive interactive experience that blends the virtual and real worlds. To ensure accurate adaptation between the virtual images and the user's gaze position, the head-mounted display device needs to adjust the display position of the virtual images.

[0003] Currently, adjusting the position of virtual images still relies on manual operation by the user, such as control via controllers or buttons. However, this manual adjustment method has significant limitations. In scenarios where the user's hands are occupied or their attention is highly concentrated, such as maintenance work or medical surgery, manual adjustment is not only inconvenient but may also increase the user's operational burden, affecting the user's immersion and the level of intelligence in human-computer interaction of head-mounted displays. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for adjusting the image of a head-mounted display device, aiming to solve the technical problems of low convenience, increased user workload, and low level of intelligence in manually adjusting the position of the virtual image in a head-mounted display device.

[0005] Acquire the periocular electromyography (EMG) signals of the user wearing the head-mounted display device; The periocular electromyography (EMG) signal is subjected to feature extraction processing to obtain the feature quantities of the periocular EMG signal; Based on the characteristic quantities of the periocular electromyography signal, the user's viewing intention is identified and processed to obtain the user's target viewing intention; The display position of the virtual image in the head-mounted display device is adjusted according to the target viewing intention.

[0006] Secondly, embodiments of the present invention also provide a screen adjustment device for a head-mounted display device, comprising: The signal acquisition module is used to acquire the periocular electromyographic signals of the user wearing the head-mounted display device; The feature extraction module is used to perform feature extraction processing on the periorbital electromyography signal to obtain the feature quantity of the periorbital electromyography signal; The intent recognition module is used to identify and process the user's viewing intent based on the feature quantities of the periocular electromyography signal to obtain the user's target viewing intent; The position adjustment module is used to adjust the display position of the virtual image in the head-mounted display device according to the target's viewing intention.

[0007] Thirdly, embodiments of the present invention also provide a head-mounted display device, the head-mounted display device including a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the screen adjustment method of the head-mounted display device as described in the first aspect.

[0008] Fourthly, embodiments of the present invention also provide a storage medium for computer-readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the screen adjustment method for a head-mounted display device as described in the first aspect.

[0009] This invention provides a method, apparatus, device, and storage medium for adjusting the display image of a head-mounted display device. By accurately capturing periocular electromyographic signals reflecting the user's eye muscle activity, and based on key features extracted from these signals that effectively reflect the user's eye muscle activity state, the invention accurately identifies the user's target viewing intention and viewing needs. This allows for timely and adaptive adjustment of the virtual image's display position within the head-mounted display device to meet the user's viewing requirements, eliminating the need for manual operation and significantly reducing the user's workload. This makes virtual image adjustment more intelligent, convenient, and efficient, thereby significantly enhancing the user's immersion. Simultaneously, it improves the intelligence level of human-computer interaction, enhancing the smoothness and naturalness of the interaction. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a method for adjusting the screen of a head-mounted display device according to an embodiment of the present invention; Figure 2 yes Figure 1 A schematic diagram of a specific implementation method for step S102; Figure 3 yes Figure 1A schematic diagram of a specific implementation method for step S103; Figure 4 This is a schematic block diagram of the structure of a screen adjustment device for a head-mounted display device provided in an embodiment of the present invention; Figure 5 This is a schematic block diagram of the structure of a head-mounted display device provided in an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0014] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0015] This invention provides a method for adjusting the display screen of a head-mounted display device. By accurately capturing periocular electromyographic signals reflecting the user's eye muscle activity, and based on key features extracted from these signals that effectively reflect the user's eye muscle activity state, the method accurately identifies the user's target viewing intention, clearly defines the user's viewing needs, and adaptively adjusts the display position of the virtual image in the head-mounted display device to meet those needs. This eliminates the need for manual operation by the user, greatly reducing their workload and making virtual image adjustment more intelligent, convenient, and efficient, thereby significantly enhancing the user's immersion. Simultaneously, it improves the intelligence level of human-computer interaction, enhancing the smoothness and naturalness of the interaction.

[0016] The image adjustment method for a head-mounted display device provided in this invention can be applied to a head-mounted display device or can be used as image adjustment software running on a head-mounted display device. The head-mounted display device may include AR devices (such as AR glasses and AR helmets), mixed reality (MR) devices (such as MR glasses and MR helmets), etc.

[0017] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for adjusting the screen of a head-mounted display device according to an embodiment of the present invention.

[0018] like Figure 1 As shown, the screen adjustment method of the head-mounted display device includes steps S101 to S103.

[0019] S101: Acquire the periocular electromyographic signals of a user wearing a head-mounted display device.

[0020] The image adjustment method provided in this invention adopts an intelligent dynamic adjustment mechanism based on electromyography, which enables the display position of the virtual image in the head-mounted display device to adaptively adjust according to the user's natural eye behavior, thereby achieving automatic and precise adjustment of the display effect of the virtual image without the need for manual operation by the user, reducing the user's operational burden, ensuring the user's visual continuity and immersion, effectively improving the convenience and efficiency of virtual image display position adjustment, and improving the intelligence level of head-mounted display devices in human-computer interaction, especially suitable for scenarios where users have difficulty with hand operation or require high concentration.

[0021] To facilitate understanding, the terms used in the embodiments of this invention will first be explained: Electromyographic sensing (EMS) refers to the process of detecting and collecting electrical activity in human muscles in response to nerve impulses, using sensing technology. When muscles receive signals transmitted by nerves, they contract, generating weak bioelectrical signals within the muscle fibers. These bioelectrical signals can be sensed and collected by EMS sensors.

[0022] Electromyographic signal (EMG) refers to the bioelectrical signal generated by nerve impulses during muscle contraction. This bioelectrical signal reflects the intensity, frequency, and pattern of muscle activity, and has a voltage amplitude in the microvolt to millivolt range.

[0023] Based on this, the image adjustment method provided by the present invention will be described in detail below.

[0024] For step S101, electromyography (EMG) sensors (such as surface electrodes or needle electrodes) can be pre-configured or installed at specific locations around the eyes and related facial muscles (such as eyelid muscles, orbicularis oculi muscles, etc.) of the head-mounted display device. This allows for the stable and interference-free sensing and acquisition of EMG signals related to eye muscle activity or changes in viewing focus, serving as periocular EMG signals. For example, in AR glasses, EMG sensors can be integrated into the frame, nose pads, temples, or forehead contact points of the AR glasses.

[0025] Users wearing head-mounted displays can trigger the automatic adjustment function of the head-mounted display device through voice commands or gesture commands. This allows the head-mounted display device to collect the user's periocular electromyography (EMG) signals in real time or periodically through its configured or installed EMG sensors. These periocular EMG signals are closely related to the user's eye movements, focus shifts, and concentration, providing important and valuable data support for adaptively adjusting the display position of the virtual image.

[0026] In some embodiments, after acquiring the user's periocular electromyography (EMG) signal, feature extraction processing can be performed on the EMG signal to obtain the feature quantities of the periocular EMG signal. The feature quantities of the periocular EMG signal represent the intensity and stability of the periocular EMG signal and can reflect the activity state of the user's eyes and related facial muscles.

[0027] In some embodiments, feature extraction processing of the periocular electromyography (EMG) signal may involve: amplifying the periocular EMG signal to obtain an amplified periocular EMG signal; filtering the amplified periocular EMG signal to obtain a filtered periocular EMG signal; rectifying the filtered periocular EMG signal to obtain a rectified periocular EMG signal; performing integral calculation on the rectified periocular EMG signal to obtain an EMG integral value; or performing root mean square (RMS) calculation on the rectified periocular EMG signal to obtain an EMG RMS value; and using the EMG integral value or EMG RMS value as a feature quantity.

[0028] Specifically, considering that the electromyographic signals around the eyes are usually weak, they can be amplified to increase their amplitude. For example, an amplification circuit (such as an operational amplifier) ​​can be used to amplify the electromyographic signals around the eyes to a reasonable level suitable for processing and analysis, resulting in amplified electromyographic signals around the eyes. This amplified signals can more clearly and delicately reflect the activity of the user's eyes and related facial muscles.

[0029] Considering that the periocular electromyography (EMG) signal may be affected by low-frequency noise (such as power frequency interference) or high-frequency noise (such as electromagnetic noise), in order to remove these unwanted noises, the amplified periocular EMG signal can be filtered. For example, a bandpass filter can be used to filter the amplified periocular EMG signal, effectively filtering out low-frequency and high-frequency noise, obtaining the filtered periocular EMG signal, and improving the signal-to-noise ratio of the filtered periocular EMG signal.

[0030] Since periocular electromyography (EMG) signals typically exhibit bidirectional waveforms with positive and negative periodic variations, the filtered EMG signals can be rectified to eliminate their negative half-waves, resulting in rectified EMG signals that are more stable and easier to analyze.

[0031] For example, rectification methods include absolute rectification (also known as unipolar rectification), full-wave rectification, or half-wave rectification. For instance, absolute rectification can be performed on the filtered periocular electromyography (EMG) signal to convert the negative values ​​into positive values, ensuring that the rectified EMG signal contains only positive values. Alternatively, full-wave rectification can be performed on the filtered EMG signal to convert the entire negative half-wave portion into positive values, resulting in a completely positive waveform and ensuring full utilization of the EMG signal's energy. Another example is half-wave rectification, which converts the negative values ​​into zero while retaining the positive values.

[0032] After obtaining the rectified periocular electromyography (EMG) signal, integration can be performed on it, that is, the rectified periocular EMG signal is accumulated over time to obtain a stable integrated electromyography (iEMG) value. The formula for integration calculation is as follows:

[0033] Wherein, EMG(t) represents the change of the rectified periorbital electromyographic signal with time t, and ∫ represents the integration with respect to time t.

[0034] Electromyography integral values ​​reflect the total energy or charge of the user's eye and related facial muscle activities, and are used to quantify the intensity of periocular electromyographic signals.

[0035] Alternatively, the rectified periocular electromyography (EMG) signal can be processed to obtain a stable root mean square (RMS) value. The formula for calculating the RMS value is shown below:

[0036] Where T is a preset time window (t∈T), which is a fixed time interval, such as 1 second, 10 seconds, etc., indicating that the above calculation process is performed within this time window.

[0037] The root mean square value of electromyography (EMG) reflects the instantaneous energy of the periocular EMG signal and is used to quantify the intensity of short-term muscle activity in the user's eyes and related facial areas.

[0038] In this way, the electromyography integral value or the root mean square value of electromyography can be used as the characteristic quantity (EMG-real) of the periocular electromyography signal. The characteristic quantity can comprehensively measure the intensity and stability of the periocular electromyography signal, thus reflecting the activity state of the user's eyes and related facial muscles.

[0039] S102: Based on the characteristic quantities of the periorbital electromyography signal, the user's viewing intention is identified and processed to obtain the user's target viewing intention.

[0040] In step S102, after obtaining the characteristic quantities of the periocular electromyography signal, the user's target viewing intention is identified based on the characteristic quantities. That is, based on the characteristic quantities, it is identified whether the user intends to adjust the virtual image in the head-mounted display device, and the user's specific adjustment needs are clarified.

[0041] In some embodiments, please refer to Figure 2 Step S102 may include the following steps: S1021: Obtain the preset threshold values ​​for electromyographic signal features and the preset effective electromyographic delay; S1022: Based on the characteristic quantity, the threshold of the electromyographic signal characteristic quantity, and the effective electromyographic delay, the user's viewing intention is discriminated to obtain the target viewing intention.

[0042] For step S1021, specifically, a preset threshold for electromyographic signal characteristics and a preset effective electromyographic delay can be obtained.

[0043] The electromyography (EMG) feature threshold, also known as the EMG feature baseline, refers to the reference feature quantity of the standard periocular EMG signal measured when the user is in a comfortable viewing state. Specifically, when the user first uses the head-mounted display device or enters calibration mode, the user is guided to view or gaze at a virtual image with the display position set at a standard reference distance (e.g., 2 meters). During this process, the user's periocular EMG signal is collected and recorded as the standard periocular EMG signal. The reference feature quantity is extracted from the standard periocular EMG signal and used as the EMG feature threshold.

[0044] Effective electromyographic delay (T-delay) is an important time parameter for identifying a user's viewing intention. It is used to distinguish between short-term fluctuations or occasional periocular electromyographic signals and the user's actual intentional periocular electromyographic activity. It helps to ensure the stability and accuracy of the adjustment process and avoids frequent adjustments to the virtual screen due to occasional small fluctuations, so as to avoid affecting the user experience.

[0045] Therefore, the user's target viewing intention can be determined based on the characteristic quantity, the threshold of the electromyographic signal characteristic quantity, and the effective electromyographic delay, ensuring the accuracy and stability of the viewing intention determination, thereby ensuring the accuracy and stability of the virtual screen display position adjustment.

[0046] In step S1022 of some embodiments, the feature quantity can be compared with an electromyographic signal feature quantity threshold; if the feature quantity is greater than the electromyographic signal feature quantity threshold, a first duration in which the feature quantity is greater than the electromyographic signal feature quantity threshold is determined, and if the first duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be close-range focusing; if the feature quantity is less than the electromyographic signal feature quantity threshold, a second duration in which the feature quantity is less than the electromyographic signal feature quantity threshold is determined, and if the second duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be distant relaxation.

[0047] In step S1022, the characteristic quantities of the user's periocular electromyography (EMG) signal can be compared with the threshold values ​​of the EMG signal characteristic quantities, and the effective EMG time delay can be referenced to obtain the comparison result. Based on the comparison result, the user's target viewing intention can be determined. The viewing intention includes near-field focusing, far-field relaxation, or no particular viewing intention.

[0048] Specifically, if the characteristic quantity (EMG-real) of the user's periocular electromyography signal is greater than the characteristic quantity threshold (EMG-base), and the first duration of the characteristic quantity (EMG-real) of the user's periocular electromyography signal being greater than the characteristic quantity threshold is greater than the effective electromyography delay (T-delay), it indicates that the user has actually performed effective periocular muscle contraction activity. It is determined that the user's target viewing intention is close-range focusing, which means that the user wants the virtual image to be closer (i.e., the virtual image is closer or larger).

[0049] For example, assuming the effective electromyographic delay (T-delay) is set to 1 second, if the characteristic quantity (EMG-real) of the user's periocular electromyographic signal is greater than the characteristic quantity threshold (EMG-base), wait for 1 second to determine whether the characteristic quantity will continue to be greater than the characteristic quantity threshold. If the characteristic quantity is always greater than the characteristic quantity threshold within this 1 second, it is determined that the user's target viewing intention is close-range focusing, and the user wants the virtual image to be zoomed in. Subsequently, the virtual image will be automatically adjusted accordingly.

[0050] If the characteristic quantity (EMG-real) of the user's periocular electromyography signal is greater than the characteristic quantity threshold (EMG-base), but the first duration is less than or equal to the effective electromyography delay (T-delay), it indicates that the user may have only experienced a very brief or accidental periocular muscle contraction activity. It is determined that the user has no particular viewing intention, which means that the user wants the virtual screen position to remain at the current display position.

[0051] If the characteristic quantity (EMG-real) of the user's periocular electromyography signal is less than the characteristic quantity threshold (EMG-base), and the second duration of the characteristic quantity (EMG-real) of the user's periocular electromyography signal is less than the characteristic quantity threshold, and if the second duration exceeds the effective electromyographic delay (T-delay), it indicates that the user has actually performed effective periocular muscle relaxation activity. It is determined that the user's target viewing intention is distance relaxation, which means that the user wants the virtual image to be moved further away (i.e., the virtual image is farther away or smaller in size).

[0052] If the characteristic quantity (EMG-real) of the user's periocular electromyography signal is less than the threshold (EMG-base), but the second duration is less than or equal to the effective electromyography delay (T-delay), it indicates that the user may have only experienced a very brief or accidental periocular muscle relaxation activity. It is determined that the user has no particular viewing intention, which means that the user wants the virtual screen position to remain at the current display position.

[0053] Furthermore, if the characteristic quantity of the user's periocular electromyography signal (EMG-real) is equal to the characteristic quantity threshold of the electromyography signal (EMG-base), it indicates that the user's periocular muscle state has not changed effectively, and it is determined that the user has not generated any special viewing intention, which means that the user wants the virtual image to remain in the current display position.

[0054] In this way, by comparing the characteristic quantities of the user's periocular electromyography (EMG) signal with the threshold of the EMG signal characteristic quantities, and referring to the effective EMG delay, the user's target viewing intention can be accurately determined, ensuring that the automatic adjustment of the virtual screen display position is triggered only when the user generates a truly effective special viewing intention.

[0055] S103: Adjust the display position of the virtual image in the head-mounted display device according to the target's viewing intention.

[0056] In step S103, after clarifying the user's target viewing intention, the display position of the virtual image in the head-mounted display device can be adjusted to match the user's target viewing intention, reducing the trouble of manual adjustment and providing the user with a more natural and intelligent interactive experience.

[0057] In some embodiments, please refer to Figure 3 Step S103 may include the following steps: Step S1031: Perform parameter matching processing on the target viewing intention to obtain the required adjustment parameters for the viewing intention.

[0058] Step S1032: Adjust the parameters according to requirements to adjust the display position of the virtual screen.

[0059] For step S1031, parameters can be adjusted to match the user's target viewing intentions and needs, so as to improve the accuracy of virtual screen display position adjustment.

[0060] In step S1031 of some embodiments, the demand adjustment parameters can be determined based on the mapping relationship between the target viewing intention and the pre-built viewing intention and adjustment parameters.

[0061] In step S1031, the adjustment parameters include the focal plane distance of the virtual image and the size of the imaging position (also known as the sensing distance).

[0062] Focal plane distance refers to the physical distance between the actual optical focusing plane in a virtual image and the human eye (usually the center of the eyeball).

[0063] Imaging position refers to the location where the virtual image is presented in three-dimensional space.

[0064] Size refers to the angular resolution of the virtual image or the actual display area.

[0065] To facilitate differentiation and matching, a mapping relationship between viewing intent and adjustment parameters can be pre-established through calibration. This mapping relationship is, for example:

[0066] In this way, we can first look for the target viewing intention from the mapping relationship. Figure 1 The intended viewing will be aligned with the target viewing intention. Figure 1 The adjustment parameters corresponding to the desired viewing intention are determined as the required adjustment parameters for the target viewing intention, so as to provide an effective and reliable basis for the automatic adjustment of the virtual screen display position.

[0067] For example, assuming the user's target viewing intention is close-up relaxation, the demand adjustment parameter is "focal plane distance D = D_near, imaging position P = P_near or size S = S_near", that is, at least one of focal plane distance D = D_near, imaging position P = P_near and size S = S_near.

[0068] In step S1032, the parameters are adjusted according to the target's viewing intention, and the display position of the virtual image in the head-mounted display device is automatically and accurately adjusted to achieve more efficient, convenient and flexible personalized virtual display control, so that the head-mounted display device can provide users with a more immersive visual experience.

[0069] In step S1032 of some embodiments, the virtual screen can be adjusted to a display position that meets the required adjustment parameters according to a preset step size.

[0070] To prevent visual abrupt changes and discomfort, and to facilitate fine-tuning and dynamic control, the virtual image can be gradually adjusted according to a preset step size (Delta D) until it is adjusted to the display position required by the adjustment parameters.

[0071] The preset step size refers to the incremental value used to change the focal plane distance, imaging position, or size each time the virtual image is adjusted. It can be flexibly set according to the user's actual needs for precision or response speed, and is not limited here.

[0072] For example, assuming the user's target viewing intention is close-up relaxation, and the required adjustment parameters are focal plane distance D_near = 0.5 meters, and the current focal plane distance D_start = 1.0 meter, the preset step size can be set to Delta D = 0.01 meters. Then, the focal plane distance is gradually decreased as follows: Adjust the focal plane distance of the virtual image to 0.5 meters.

[0073] In this way, by making gradual adjustments to the virtual image, the precision and naturalness of the adjustment of the virtual image's display position can be improved, while avoiding adjustment errors.

[0074] In some embodiments, when the target viewing intention is close-range focusing, the required adjustment parameters include at least one of a first focal plane distance, a first imaging position, and a first size. Adjusting the virtual image to a display position that meets the required adjustment parameters may involve reducing the current focal plane distance of the virtual image until it is reduced to the first focal plane distance. The current imaging position of the virtual image is adjusted to be closer until it reaches the first imaging position; or, the current size of the virtual image is increased until it reaches the first size, so as to adjust the display position of the virtual image to present a zoomed-in effect in the user's field of vision space.

[0075] Specifically, when the user's target viewing intention is close-range focusing, the required adjustment parameters include at least one of the following: first focal plane distance, first imaging position, and first size. For example, the first focal plane distance is D_near, the first imaging position is P_near, and the first size is S_near.

[0076] In order to meet the user's viewing needs for close-range focusing, at least one of the following operations can be performed: the current focal plane distance of the virtual image is reduced until it reaches the first focal plane distance, so that the user can clearly focus on the virtual image at close range; the current imaging position of the virtual image is moved closer until it reaches the first imaging position, ensuring that the virtual image is closer to the user's field of vision and enhancing the sense of immersion; the current size of the virtual image is increased until it reaches the first size requirement, making the virtual image more prominent and thus enhancing the visual effect of the virtual image.

[0077] The above adjustments effectively position the virtual image so that it appears as an ideal close-up within the user's field of vision, ensuring a better close-up viewing experience.

[0078] In some embodiments, when the target viewing intention is for distant relaxation, the required adjustment parameters include at least one of a second focal plane distance, a second imaging position, and a second size. Adjusting the virtual image to a display position that meets the required adjustment parameters includes at least one of the following: increasing the current focal plane distance of the virtual image until it reaches the second focal plane distance; moving the current imaging position of the virtual image further away until it reaches the second imaging position; or decreasing the current size of the virtual image until it reaches the second size, so as to adjust the display position of the virtual image to present a distanced effect in the user's field of vision space.

[0079] Specifically, when the user's target viewing intention is to relax from a distance, in order to meet the user's viewing needs for relaxation from a distance, the demand adjustment parameters include at least one of the following: second focal plane distance, second imaging position, and second size. For example, the first focal plane distance is D_far, the first imaging position is P_far, and the first size is S_far.

[0080] In order to meet the user's need for relaxed viewing from a distance, at least one of the following operations can be performed: increase the current focal plane distance of the virtual image until it reaches the second focal plane distance, so that the user can comfortably view the virtual image from a distance; adjust the current imaging position of the virtual image to move further away from the user until it reaches the second imaging position, so that the virtual image appears wider to the user and provides a relaxed viewing experience; reduce the current size of the virtual image until it is reduced to the second size requirement, which helps to create a sense of openness and allows the user to feel visual comfort and relaxation when viewing from a distance.

[0081] The above adjustments effectively position the virtual image so that it presents an ideal distance effect within the user's field of vision, ensuring that the user can easily and comfortably view the virtual image.

[0082] In some embodiments, during the adjustment process, visual or auditory feedback may be provided to the user to inform them that the virtual image is being adjusted according to the electromyographic intention around the eyes, thereby increasing the user's sense of participation and control over the head-mounted display device.

[0083] For example, visual feedback could display text such as "Prompt: Close-up viewing intent detected, adjusting focus"; auditory feedback could issue a low-frequency voice prompt such as "Adjusting to close-up vision mode" to enhance the immersive experience of human-computer interaction.

[0084] The image adjustment method provided in this embodiment of the invention can bring the following beneficial effects: On the one hand, it enables a more natural and convenient hands-free adjustment method: by using electromyography signals around the eyes to identify the user's target viewing intention, the system automatically and flexibly adjusts the display position of the virtual image in the head-mounted display device accordingly, replacing traditional adjustment methods such as handles and buttons, thus achieving a more natural and intuitive human-computer interaction. Users do not need to manually adjust the display position of the virtual image, which is particularly suitable for scenarios where hands are occupied or mobility is limited (such as medical surgery, industrial operations, virtual meetings, etc.).

[0085] On the other hand, it improves the accuracy and personalization of virtual image adjustment: by extracting key feature quantities from the electromyographic signals around the eyes, the user's target viewing intention can be accurately identified based on the feature quantities, so as to clearly understand the user's current focus needs. This allows for the automatic and accurate adjustment of the virtual image display position to match the user's focus needs, achieving more efficient, convenient, and flexible virtual display control and realizing a personalized view.

[0086] On the other hand, it improves adjustment efficiency and enhances user immersion: by quickly responding to the user's target viewing intention to adjust the virtual screen display position, it can improve adjustment efficiency, give users instant feedback, thereby achieving a low-latency interactive experience and enhancing the user's immersive experience.

[0087] On the other hand, it improves adaptability and compatibility: the image adjustment method provided in this embodiment of the invention does not rely on a complex external optical system, can be applied to a variety of head-mounted display devices, and has good scalability and commercial potential.

[0088] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention. Where there is no conflict, the above embodiments and features can be combined with each other.

[0089] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of a screen adjustment device for a head-mounted display device provided in an embodiment of the present invention.

[0090] like Figure 4 As shown, the image adjustment device 100 of the head-mounted display device includes: Signal acquisition module 110 is used to acquire the periocular electromyographic signals of the user wearing the head-mounted display device; The intent recognition module 120 is used to recognize and process the user's viewing intent based on the feature quantities of the periocular electromyography signal to obtain the user's target viewing intent; The position adjustment module 130 is used to adjust the display position of the virtual image in the head-mounted display device according to the target's viewing intention.

[0091] In some embodiments, the intent recognition module 120 is specifically used for: Obtain preset threshold values ​​for electromyographic signal features and preset effective electromyographic delay; Based on the aforementioned feature quantity, the threshold value of the electromyographic signal feature quantity, and the effective electromyographic delay, the user's viewing intention is determined to obtain the target viewing intention.

[0092] In some embodiments, the intent recognition module 120 is further configured to: The feature quantity is compared with the threshold value of the electromyographic signal feature quantity; If the feature quantity is greater than the threshold value of the electromyographic signal feature quantity, a first duration for which the feature quantity is greater than the threshold value of the electromyographic signal feature quantity is determined, and if the first duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be close-range focusing; If the feature quantity is less than the threshold value of the electromyographic signal feature quantity, a second duration in which the feature quantity is less than the threshold value of the electromyographic signal feature quantity is determined, and if the second duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be distance relaxation.

[0093] In some embodiments, the position adjustment module 130 is specifically used for: The target viewing intention is subjected to parameter matching processing to obtain the required adjustment parameters of the target viewing intention; Adjust the parameters according to the requirements to adjust the display position of the virtual image.

[0094] In some embodiments, the position adjustment module 130 is further configured to: According to the preset step size, the virtual screen is adjusted to the display position that meets the required adjustment parameters.

[0095] In some embodiments, when the target viewing intention is close-range focusing, the required adjustment parameters include at least one of a first focal plane distance, a first imaging position, and a first size. The position adjustment module 130 is further configured to perform at least one of the following steps: Adjusting the virtual image to a display position that meets the required adjustment parameters can be achieved by reducing the current focal plane distance of the virtual image until it is reduced to the first focal plane distance. The current imaging position of the virtual image is adjusted until it is close to the first imaging position; The current size of the virtual image is increased until it reaches the first size, so that the display position of the virtual image is adjusted to present a zoomed-in effect in the user's field of vision.

[0096] In some embodiments, when the target viewing intention is distant relaxation, the demand adjustment parameters include at least one of a second focal plane distance, a second imaging position, and a second size. The position adjustment module 130 is further configured to perform at least one of the following steps: The current focal plane distance of the virtual image is increased until it reaches the second focal plane distance; The current imaging position of the virtual image is adjusted further away until it is adjusted to the second imaging position; The current size of the virtual image is reduced until it is reduced to the second size, so that the display position of the virtual image is adjusted to present a distance effect in the user's field of vision space.

[0097] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the image adjustment device for the head-mounted display device described above can be referred to the corresponding process in the aforementioned embodiment of the image adjustment method for the head-mounted display device, and will not be repeated here.

[0098] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of a head-mounted display device provided in an embodiment of the present invention.

[0099] like Figure 5As shown, the head-mounted display device 200 includes a processor 201 and a memory 202, which are connected via a bus 203, such as an I2C (Inter-integrated Circuit) bus.

[0100] Specifically, processor 301 provides computing and control capabilities to support the operation of the entire head-mounted display device. Processor 201 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0101] Specifically, the memory 202 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0102] Those skilled in the art will understand that Figure 5 The structures shown are merely block diagrams of some structures related to the embodiments of the present invention, and do not constitute a limitation on the head-mounted display devices on which the embodiments of the present invention are applied. Specific head-mounted display devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.

[0103] The processor 201 is used to run a computer program stored in the memory 202, and implements any of the image adjustment methods for head-mounted display devices provided in the embodiments of the present invention when executing the computer program.

[0104] In one embodiment, the processor 201 is configured to run a computer program stored in a memory, and to perform the following steps when executing the computer program: Acquire the periocular electromyography (EMG) signals of the user wearing the head-mounted display device; The periocular electromyography (EMG) signal is subjected to feature extraction processing to obtain the feature quantities of the periocular EMG signal; Based on the characteristic quantities of the periocular electromyography signal, the user's viewing intention is identified and processed to obtain the user's target viewing intention; The display position of the virtual image in the head-mounted display device is adjusted according to the target viewing intention.

[0105] In some embodiments, when the processor 201 performs processing to identify the user's viewing intention based on the feature quantities of the periocular electromyography signal to obtain the user's target viewing intention, it is configured to: Obtain preset threshold values ​​for electromyographic signal features and preset effective electromyographic delay; Based on the aforementioned feature quantity, the threshold value of the electromyographic signal feature quantity, and the effective electromyographic delay, the user's viewing intention is determined to obtain the target viewing intention.

[0106] In some embodiments, when the processor 201 performs the process of determining the user's viewing intention based on the feature quantity, the electromyographic signal feature quantity threshold, and the effective electromyographic delay to obtain the target viewing intention, it is configured to: The feature quantity is compared with the threshold value of the electromyographic signal feature quantity; If the feature quantity is greater than the threshold value of the electromyographic signal feature quantity, a first duration for which the feature quantity is greater than the threshold value of the electromyographic signal feature quantity is determined, and if the first duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be close-range focusing; If the feature quantity is less than the threshold value of the electromyographic signal feature quantity, a second duration in which the feature quantity is less than the threshold value of the electromyographic signal feature quantity is determined, and if the second duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be distance relaxation.

[0107] In some embodiments, when the processor 201 adjusts the display position of the virtual image in the head-mounted display device according to the target viewing intention, it is configured to: The target viewing intention is subjected to parameter matching processing to obtain the required adjustment parameters of the target viewing intention; Adjust the parameters according to the requirements to adjust the display position of the virtual image.

[0108] In some embodiments, when the processor 201 adjusts the display position of the virtual image according to the required adjustment parameters, it is configured to: According to the preset step size, the virtual image is adjusted to the display position that meets the required adjustment parameters.

[0109] In some embodiments, when the target viewing intention is close-range focusing, the demand adjustment parameters include at least one of a first focal plane distance, a first imaging position, and a first size. When the processor 201 adjusts the virtual image to a display position that satisfies the demand adjustment parameters, it performs at least one of the following steps: The current focal plane distance of the virtual image is reduced until it is reduced to the first focal plane distance; The current imaging position of the virtual image is adjusted until it is close to the first imaging position; The current size of the virtual image is increased until it reaches the first size, so that the display position of the virtual image is adjusted to present a zoom-in effect in the user's field of vision.

[0110] In some embodiments, when the target viewing intention is distant relaxation, the demand adjustment parameters include at least one of a second focal plane distance, a second imaging position, and a second size. When adjusting the virtual image to a display position that satisfies the demand adjustment parameters, the processor 201 performs at least one of the following steps: The current focal plane distance of the virtual image is increased until it reaches the second focal plane distance; The current imaging position of the virtual image is adjusted further away until it is adjusted to the second imaging position; The current size of the virtual image is reduced until it is reduced to the second size, so that the display position of the virtual image is adjusted to present a distance effect in the user's field of vision space.

[0111] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the head-mounted display device described above can be referred to the corresponding process in the aforementioned embodiment of the head-mounted display device screen adjustment method, and will not be repeated here.

[0112] This invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement any of the image adjustment methods for head-mounted display devices provided in the specification of this invention.

[0113] The storage medium can be volatile or non-volatile. It can be an internal storage unit of the head-mounted display device described in the foregoing embodiments, such as the hard drive or memory of the head-mounted display device. Alternatively, it can be an external storage device of the head-mounted display device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the head-mounted display device.

[0114] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware embodiments, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0115] It should be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0116] The sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The above descriptions are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for adjusting the image of a head-mounted display device, characterized in that, include: Acquire the periocular electromyography (EMG) signals of the user wearing the head-mounted display device; Based on the characteristic quantities of the periocular electromyography signal, the user's viewing intention is identified and processed to obtain the user's target viewing intention; The display position of the virtual image in the head-mounted display device is adjusted according to the target viewing intention.

2. The image adjustment method according to claim 1, characterized in that, The process of identifying the user's viewing intention based on the feature quantities of the periocular electromyography signal to obtain the user's target viewing intention includes: Obtain preset threshold values ​​for electromyographic signal features and preset effective electromyographic delay; Based on the aforementioned feature quantity, the threshold value of the electromyographic signal feature quantity, and the effective electromyographic delay, the user's viewing intention is determined to obtain the target viewing intention.

3. The image adjustment method according to claim 2, characterized in that, The step of determining the user's viewing intention based on the feature quantity, the electromyographic signal feature quantity threshold, and the effective electromyographic delay to obtain the target viewing intention includes: The feature quantity is compared with the threshold value of the electromyographic signal feature quantity; If the feature quantity is greater than the threshold value of the electromyographic signal feature quantity, a first duration for which the feature quantity is greater than the threshold value of the electromyographic signal feature quantity is determined, and if the first duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be close-range focusing; If the feature quantity is less than the threshold value of the electromyographic signal feature quantity, a second duration in which the feature quantity is less than the threshold value of the electromyographic signal feature quantity is determined, and if the second duration is greater than the effective electromyographic delay, the user's target viewing intention is determined to be distance relaxation.

4. The image adjustment method according to claim 3, characterized in that, Adjusting the display position of the virtual image in the head-mounted display device according to the target viewing intention includes: The target viewing intention is subjected to parameter matching processing to obtain the required adjustment parameters of the target viewing intention; Adjust the parameters according to the requirements to adjust the display position of the virtual image.

5. The image adjustment method according to claim 4, characterized in that, The step of adjusting the display position of the virtual image according to the required adjustment parameters includes: According to the preset step size, the virtual image is adjusted to the display position that meets the required adjustment parameters.

6. The image adjustment method according to claim 5, characterized in that, When the target viewing intention is close-range focusing, the required adjustment parameters include at least one of a first focal plane distance, a first imaging position, and a first size. Adjusting the virtual image to a display position that satisfies the required adjustment parameters includes at least one of the following: The current focal plane distance of the virtual image is reduced until it is reduced to the first focal plane distance; The current imaging position of the virtual image is adjusted until it is close to the first imaging position; The current size of the virtual image is increased until it reaches the first size, so that the display position of the virtual image is adjusted to present a zoom-in effect in the user's field of vision.

7. The image adjustment method according to claim 5, characterized in that, When the target viewing intention is for distant relaxation, the demand adjustment parameters include at least one of a second focal plane distance, a second imaging position, and a second size. Adjusting the virtual image to a display position that satisfies the demand adjustment parameters includes at least one of the following: The current focal plane distance of the virtual image is increased until it reaches the second focal plane distance; The current imaging position of the virtual image is adjusted further away until it is adjusted to the second imaging position; The current size of the virtual image is reduced until it is reduced to the second size, so that the display position of the virtual image is adjusted to present a distance effect in the user's field of vision space.

8. A screen adjustment device for a head-mounted display device, characterized in that, include: The signal acquisition module is used to acquire the periocular electromyographic signals of the user wearing the head-mounted display device; The intent recognition module is used to identify and process the user's viewing intent based on the feature quantities of the periocular electromyography signal to obtain the user's target viewing intent; The position adjustment module is used to adjust the display position of the virtual image in the head-mounted display device according to the target's viewing intention.

9. A head-mounted display device, characterized in that, The head-mounted display device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for establishing communication between the processor and the memory, wherein when the computer program is executed by the processor, it implements the steps of the screen adjustment method of the head-mounted display device as described in any one of claims 1 to 7.

10. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the method for adjusting the screen of the head-mounted display device according to any one of claims 1 to 7.

Images