Control method of head-mounted device, head-mounted device, and storage medium

Abstract

The application discloses a control method of a head-mounted device, the head-mounted device and a storage medium, and relates to the technical field of head-mounted intelligent devices, and comprises the following steps: in response to a blind area display instruction, blind area video data corresponding to a blind area direction is displayed in a blind area picture display window; in the process of displaying the blind area video data, an actual field of view area is determined according to eye movement tracking information and head posture detection information; and if the actual field of view area overlaps a target area corresponding to the blind area direction, the blind area picture display window is hidden. The application adaptively adjusts the display state of the blind area video data by detecting whether the actual field of view area overlaps the target area corresponding to the blind area direction, and solves the problem that the head-mounted device in the prior art cannot dynamically adjust the display of the blind area picture in combination with the actual field of view area of a user, so that the blind area content display window does not block the actual field of view of the user.

Description

Technical Field

[0001] This application relates to the field of head-mounted device technology, and more particularly to control methods, devices and storage media for head-mounted devices. Background Technology

[0002] Head-mounted displays (such as VR and AR devices) are widely used in various scenarios including immersive experiences and spatial interactions. When users wear these devices, their field of vision is limited by the device's structure and the user's head posture, resulting in significant blind spots. To address this blind spot observation problem, existing head-mounted displays typically use multiple cameras to collect panoramic data of the device's surroundings and simultaneously display this data in a fixed window. However, users actively change their field of vision by rotating their heads to view content in the blind spots. When the actual scene content in the observed field of vision matches the content displayed in the window, the display window used to show the panoramic data obstructs the user's actual field of vision, interfering with normal observation.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this application is to provide a control method for a head-mounted device, a head-mounted device, and a storage medium, aiming to solve the technical problem of blind spot content display windows obstructing the user's actual field of vision.

[0005] To achieve the above objectives, this application proposes a control method for a head-mounted device, the steps of which include: In response to the blind spot display command, the video data of the blind spot corresponding to the location of the blind spot is displayed in the blind spot screen display window; During the display of the blind spot video data, the actual field of vision is determined based on eye-tracking information and head posture detection information; If the actual field of view overlaps with the target area corresponding to the blind spot, the blind spot display window is hidden.

[0006] For example, the step of displaying the blind spot video data corresponding to the blind spot location in the blind spot screen display window includes: The current visual field area is determined based on the eye-tracking information and the head posture detection information; The location of the blind spot is determined based on the current field of view; Based on the location of the blind spot, the video data of the blind spot is filtered from the camera data, and the video data of the blind spot is displayed in the blind spot screen display window.

[0007] For example, the step of determining the location of the blind spot based on eye-tracking information and head posture detection information includes: The eye-tracking information is used to determine the gaze deviation angle, and the head rotation angle is used to determine the head posture detection information. The location of the blind spot is determined based on the angle of visual deviation and the angle of head rotation.

[0008] For example, the step of determining the location of the blind spot based on the line-of-sight offset angle and the head rotation angle includes: The target offset angle is determined based on the line of sight offset angle and the head rotation angle. Determine the visible orientation corresponding to the current field of view area, and use the orientation obtained by offsetting the visible orientation by the target offset angle as the blind zone orientation.

[0009] For example, the step of determining the actual visual field area based on the eye-tracking information and the head posture detection information during the display of the blind spot video data includes: The user's actual visual orientation is determined based on the eye-tracking information and the head posture detection information. The actual field of view area is determined based on the actual visible orientation.

[0010] The step of filtering blind spot video data from camera data based on the blind spot location and displaying the blind spot video data in the blind spot display window, as exemplarily described, includes: Acquire video data corresponding to at least one range of acquisition angles; Acquire target video data corresponding to the target acquisition angle range that matches the blind zone orientation, and use the target video data as the blind zone video data; The target blind spot video data is displayed in the blind spot image display window, and the display area corresponding to the blind spot image display window is smaller than the display area of ​​the head-mounted device's display window.

[0011] For example, the step of displaying the blind spot video data in the blind spot screen display window includes: Determine the display parameters of the blind spot display window, the display parameters including at least one of the display position, display area, and display transparency; According to the display parameters, the blind spot image display window that will display the blind spot video data will be superimposed on the display window of the head-mounted device.

[0012] For example, after determining the actual visual field area based on the eye-tracking information and the head posture detection information during the process of displaying the blind spot video data, the method further includes: If the actual field of view area does not overlap with the target area corresponding to the blind spot location, then return to the step of displaying the blind spot video data corresponding to the blind spot location in the blind spot screen display window.

[0013] Furthermore, to achieve the above objectives, this application also proposes a head-mounted device, the head-mounted device comprising: Acquisition Unit: Used to acquire video data of the blind spot location; The display unit includes a blind spot display window for displaying the blind spot video data; The sensor unit is used to detect eye-tracking information and head posture detection information when the user wears the head-mounted device; The processing unit is used to respond to the blind spot display command, acquire blind spot video data, and control the display unit to display the blind spot video data in the blind spot screen display window; The processing unit is also used to determine the actual field of view area based on the detected eye-tracking information and head posture detection information. If the actual field of view area overlaps with the target area corresponding to the blind spot location, the display unit is controlled to hide the blind spot screen display window. The head-mounted device further includes a memory and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the head-mounted device as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the control method for the head-mounted device as described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the control method for the head-mounted device as described above.

[0016] The one or more technical solutions proposed in this application have at least the following technical effects: By responding to blind spot display commands, the timing of the display of blind spot video data in the blind spot display window is precisely controlled, achieving adaptive control of blind spot display. Simultaneously, during the display of blind spot video data, eye-tracking information and head posture detection information are combined to determine the user's actual field of vision. When the actual field of vision overlaps with the target area corresponding to the blind spot location, the blind spot display window is hidden, effectively avoiding interference with normal observation caused by the overlap between the blind spot display window and the user's actual field of vision, thereby improving user convenience and observation smoothness. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the module structure of the control device for the head-mounted device of this application; Figure 2 A flowchart illustrating the second embodiment of the control method for the head-mounted device of this application; Figure 3 This is a detailed flowchart of step S10 in the second embodiment of the control method for the head-mounted device of this application; Figure 4 This is an example diagram of the field of view area involved in the second embodiment of the control method for the head-mounted device of this application; Figure 5 This is a detailed flowchart of step S12 in the second embodiment of the control method for the head-mounted device of this application; Figure 6 This is a detailed flowchart illustrating step S20 of the third embodiment of the control method for the head-mounted device of this application; Figure 7 This is a schematic diagram of the hardware operating environment involved in the control method of the head-mounted device in the embodiments of this application. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0019] In related technologies, head-mounted smart devices (such as VR and AR devices) have been widely used in various scenarios such as immersive experiences and spatial interaction. When users wear these devices, their field of vision is limited by the device's structure and affected by the user's head posture, resulting in significant blind spots. To address this blind spot observation problem, existing head-mounted devices typically use multiple cameras to collect panoramic data of the device's surroundings and simultaneously display the panoramic image in a fixed display area. However, these technologies do not incorporate user eye-tracking information and head posture detection information for targeted adjustment of the blind spot display. Consequently, when displaying blind spot information, the head-mounted device cannot dynamically adjust the blind spot image display state according to the user's actual field of vision, easily leading to overlap between the blind spot image display window and the user's actual field of vision, interfering with normal observation.

[0020] This application provides a solution that, in response to a blind spot display command, acquires blind spot video data corresponding to the blind spot location, displays the blind spot video data in a blind spot display window, and then determines the actual field of view area based on eye-tracking information and head posture detection information; if the actual field of view area overlaps with the target area corresponding to the blind spot location, the blind spot display window is hidden, thereby controlling the display state of the blind spot display window to dynamically display the blind spot video data and prevent the displayed blind spot video data from obstructing the user's field of view.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] This invention provides a control method for a head-mounted device, a control device for a head-mounted device, and a computer-readable storage medium.

[0023] This invention provides a control device for a head-mounted device that can be integrated into the head-mounted device. The head-mounted device includes, but is not limited to, AR head-mounted devices, VR head-mounted devices, and smart helmets with AR or VR functions, such as motorcycle helmets, industrial safety helmets, and firefighter helmets. Optionally, the control device for the head-mounted device can also be an independent electronic device. This electronic device can be a mobile terminal or a fixed terminal, including but not limited to smartphones, smartwatches, tablets, laptops, smart vehicles, and other devices integrating cameras and display modules. Fixed terminals include, but are not limited to, desktop computers and smart TVs. Optionally, the electronic device can also be a server or similar device. This server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but is not limited to these.

[0024] It should be noted that the control method of the head-mounted device in the embodiments of this application can be implemented by the server alone, by the head-mounted device alone, or by the head-mounted device and the server together.

[0025] The following description uses the control method of the head-mounted device in the embodiment of this application as an example to illustrate the method.

[0026] First Embodiment like Figure 1 As shown, the control device for the head-mounted device provided in this embodiment of the invention includes a head-mounted device 10, which may be AR glasses. The head-mounted device 10 includes: Acquisition unit 11 is used to acquire blind spot video data of the blind spot location; Display unit 12, the display unit includes a blind spot display window, the blind spot display window is used to project and display blind spot video data; Sensor unit 13 is used to detect eye-tracking information and head posture detection information when the user wears the head-mounted device 10; The processing unit 14 is used to respond to the blind spot display command, acquire blind spot video data, and control the display unit 12 to display the blind spot video data in the blind spot screen display window; the processing unit is also used to determine the actual field of vision area based on the detected eye tracking information and head posture detection information, and control the display state of the blind spot screen display window based on the actual field of vision area.

[0027] The head-mounted device 10 also includes a wearing unit, comprising temples and an elastic band for wearing the head-mounted device on the user's head.

[0028] It should be noted that the acquisition unit includes at least one blind spot camera 111, which is integrated into the front end of the temples on both the left and right sides of the AR glasses. These cameras are used to acquire real-world video data of the side blind spots of the user's surroundings. It is understood that by integrating the blind spot camera 111 into the front end of the temples, the lens of the blind spot camera 111 can be angled to the user's visible direction. The left blind spot camera primarily covers the left side blind spot, and the right blind spot camera primarily covers the right side blind spot. The dual-camera collaboration can achieve full coverage of the left and right side blind spots without any blind spots. Optionally, the blind spot camera 111 also includes cameras integrated into the rear end of the temples on both sides of the AR glasses. These rear-end cameras primarily cover the blind spot behind the user, acquiring real-world video data of the blind spot behind the user.

[0029] In addition, the blind spot camera 111 can also be integrated into the lens edges on the top and bottom sides of the AR glasses to collect real-world video data of the blind spots above and below the user's location.

[0030] Optionally, the display unit 12 includes a projection component and a lens. The projection component can project and display virtual images on the lens, thereby simulating the augmented reality effect of superimposing virtual images in a real environment for the user. It should be noted that the lens may include a single lens or two lenses. When the head-mounted device includes two lenses, these two lenses correspond to the user's two eyes respectively, thereby presenting the image elements / objects in the virtual space observed by both eyes, thus creating a three-dimensional display effect for the user.

[0031] Optionally, the sensor unit 13 includes an eye-tracking module 131, which is disposed within the lens and is used to collect position information of at least one key point on the user's eye. Based on this position information, eye-tracking information is generated, reflecting data such as the user's eye movement trajectory, coordinates of the gaze point, and gaze duration. This information is used to determine the visual field area corresponding to the user's current gaze focus direction. The key points include, but are not limited to, one or more key points on the eyelids, one or more key points on the iris, and one or more key points on the pupil. For example, if the user's gaze is focused on the center of the display unit, the coordinates of the gaze point collected by the eye-tracking module are (640, 360) (assuming a display resolution of 1280×720), and the gaze duration is 2 seconds. It is understood that when the user's eyes focus on different directions and depths, the positions of the key points on the user's eye will also differ. Conversely, by analyzing the positions of the key points on the human eye, the user's gaze focus direction can be deduced, and thus the user's current visual field area can be determined.

[0032] The sensor unit 13 also includes a head posture sensor 132. The head posture sensor 132 can be installed in the wearing unit or in the lens, which is not limited here. The head posture sensor 132 is used to detect the user's head posture change information and generate head posture detection information based on the posture change information. The head posture detection information reflects the user's head rotation angle, pitch angle, translation trajectory, head spatial pose, and head orientation dwell time and other related data, so as to determine the visual field area corresponding to the user's head orientation.

[0033] It should be noted that the sensor unit 13 is communicatively connected to the processing unit 14, and sends the detected eye-tracking information and head posture detection information to the processing unit 14 in real time. The processing unit 14 then determines the actual visual field area based on the received eye-tracking information and head posture detection information, and determines the display state of the blind spot display window based on the actual visual field area. The display state includes hidden display.

[0034] In this embodiment, after the user wears the head-mounted device 10, the acquisition unit 11 of the head-mounted device 10 is activated to acquire blind spot video data corresponding to the blind spot location in real time. The processing unit 14 controls the display unit 12 to display the blind spot video data in the blind spot screen display window in real time. During the display of the blind spot video data, the sensor unit 13 acquires eye-tracking information and head posture detection information in real time. The processing unit 14 determines the actual field of view area in real time. When the actual field of view area overlaps with the target area corresponding to the blind spot location, the processing unit 14 controls the display unit 12 to hide the blind spot screen display window to prevent the displayed blind spot video data from obstructing the user's observation of the actual screen using the lenses.

[0035] Second Embodiment This application provides a control method for a head-mounted device. The control method is applied to a control device for the head-mounted device as described in the first embodiment. The control device may be built into the head-mounted device, or it may be a mobile terminal that controls the head-mounted device, such as a mobile phone, tablet, or computer. It may also be a cloud server that communicates with the head-mounted device. No limitation is made here. The control device for the head-mounted device is used to realize the display control of content in the blind spot. This application embodiment uses the example of the control device being built into the head-mounted device for illustration.

[0036] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the control method for the head-mounted device of this application.

[0037] In this embodiment, the control method of the head-mounted device includes steps S10 to S30: Step S10: In response to the blind spot display command, display the blind spot video data corresponding to the blind spot location in the blind spot screen display window.

[0038] In one alternative implementation, the head-mounted device may be AR glasses (augmented reality glasses).

[0039] For example, when a user wears AR glasses, the AR glasses include a blind spot camera, a projection component, and a display window. The blind spot camera is used to collect real scene video data in the blind spot location. The projection component is used to generate virtual content based on the collected blind spot video data and overlay the virtual content onto the display window, so that the user can see the real scene data after the virtual content is overlaid on the display window. The display window is the display window that the human eye can see through the lenses after the user wears AR glasses.

[0040] In one optional implementation, the blind spot camera includes at least one camera integrated into the front end of the temples on both sides of the AR glasses. This camera is used to collect real-world video data of the side blind spots in the user's surroundings. It is understood that by integrating the blind spot camera into the front end of the temples, the lens of the blind spot camera can be angled to the user's line of sight. The left blind spot camera primarily covers the left-side blind spot, and the right blind spot camera primarily covers the right-side blind spot. The dual-camera collaboration achieves full coverage of the left and right side blind spots, eliminating blind spots. Optionally, the blind spot camera also includes cameras integrated into the rear end of the temples on both sides of the AR glasses. These rear-end cameras primarily cover the blind spot behind the user, collecting real-world video data of the blind spot behind the user.

[0041] Understandably, when users wear AR glasses normally, their field of vision mainly covers the area directly in front, while the left and right sides (areas outside the visible area) are blind spots. Especially in scenarios such as walking, cycling, and industrial inspection, obstacles, people, and objects in these areas are easily overlooked, posing safety hazards. In such cases, it is necessary to turn the head or rotate the eyes to shift the visual orientation, so that the blind spot content corresponding to the blind spot content is within the user's visible field of vision, in order to see the blind spot scene information.

[0042] Based on this, upon detecting that a user is wearing a head-mounted device, a blind spot display command is triggered to acquire the blind spot content and display it in the display window. This blind spot content is the video data captured by the blind spot camera. This embodiment provides three ways to trigger the blind spot display command.

[0043] In one alternative implementation, the blind spot display command can be triggered by the user, such as by tapping the temple of the glasses, or by receiving a voice command from the user and using the voice command as the blind spot display command; the blind spot display command can also be triggered directly after detecting that the user is wearing and turning on the AR glasses.

[0044] In another optional implementation, the blind spot display command can be triggered automatically based on the current scene. This includes: acquiring the current scene; and triggering the blind spot display command when the current scene is a preset scene. The preset scene can be user-set or system-configured, and includes scenarios such as skiing, driving, and running. For example, if the current scene is a skiing scene, and the user cannot clearly see the surrounding scene information, they will also be unable to see other skiers nearby. Due to the high speed of skiing, collisions are likely to occur if the distance between the user and other skiers is not controlled in time. In this case, triggering the blind spot display command can display images of surrounding skiers in the area corresponding to the blind spot to the user, reducing the occurrence of collisions.

[0045] In another alternative implementation, the blind spot display command can be triggered based on the content of the blind spot video data. When the blind spot video data contains preset content, the blind spot display command is triggered. The preset content can be an obstacle during the user's walking process, or an object that is less than or equal to a preset distance from the user, such as a car driving towards the user. This is not limited here.

[0046] Furthermore, in response to the blind spot display command, the blind spot video data corresponding to the blind spot location is acquired, and the blind spot video data is rendered and displayed in the blind spot display window so that the user can view the real scene information corresponding to the blind spot location. In one embodiment, the video data collected by each blind spot camera can be rendered and displayed in the blind spot display window. In this case, the blind spot display window is divided into multiple sub-display windows, and each sub-display window displays video data collected by different blind spot cameras. For example, the blind spot display window includes a left blind spot sub-display window and a right blind spot sub-display window. The left blind spot sub-display window displays the left blind spot video data collected by the left blind spot camera, and the right blind spot sub-display window displays the right blind spot video data collected by the right blind spot camera. In another embodiment, only the blind spot video data containing preset content can be displayed in the blind spot display window. For example, if the left blind spot video data includes obstacles, then the left blind spot video data is displayed in the blind spot display window.

[0047] In one embodiment, the blind spot location that the user needs to focus on can also be determined based on the user's eye-tracking information and head posture detection information. The blind spot video data of that location is then displayed in the blind spot display window. For example, if the blind spot location that the user needs to focus on is the left side, the blind spot video data captured by the left blind spot camera is displayed in the blind spot display window. Figure 3 Step S10 includes steps S11 to S13: Step S11: Determine the current visual field area based on the eye-tracking information and the head posture detection information; Step S12: Determine the location of the blind spot based on the current field of view area; Step S13: Filter blind spot video data from camera data according to the blind spot location, and display the blind spot video data in the blind spot screen display window.

[0048] In this embodiment, after detecting that a user is wearing a head-mounted device, the eye-tracking module is activated to obtain eye-tracking information, and the head posture sensor is activated to obtain head posture detection information. The visible orientation is determined based on the eye-tracking information and the head posture detection information, and the area corresponding to the visible orientation is taken as the current field of view. Then, the blind spot orientation is determined based on the current field of view. Alternatively, the blind spot orientation can be determined after determining the visible orientation; or the blind spot area can be determined after determining the current field of view, and the orientation corresponding to the blind spot area is taken as the blind spot orientation.

[0049] In one optional implementation, after the user wears the AR glasses, eye-tracking information and head posture detection information are acquired by the eye-tracking module and head posture sensor, respectively. Then, the user's head orientation and gaze focus direction are determined based on the eye-tracking information and head posture detection information. The actual visible area, superimposed with the first visible area corresponding to the head orientation and the second visible area corresponding to the gaze focus direction, is taken as the current field of view, i.e., the area the user can directly observe. Further, after determining the current field of view, the blind spot orientation is determined based on the boundary of the current field of view. The blind spot orientation is the angular range corresponding to the area outside the current field of view that the user cannot directly observe, including the angular range corresponding to the area covered by the blind spot camera located at the front of the AR glasses temples. The blind spot orientation is determined by the boundary of the current field of view.

[0050] It should be noted that the method for determining the first visible area based on head posture detection information is as follows: taking the user's head as the origin, the field of view corresponding to a preset horizontal angle (e.g., a preset horizontal angle of 15°) is defined as the horizontal visible direction, and the field of view corresponding to a preset vertical angle (e.g., a preset vertical angle of 10°) is defined as the vertical visible direction. The combination of these two forms the first visible direction corresponding to the head's orientation, and the area corresponding to the first visible direction is taken as the first visible area. Specifically, the head posture detection information can be defined as follows: the user's head is facing forward, and the spatial coordinates of the head correspond to the origin. Based on this head posture detection information, the aforementioned horizontal and vertical visible areas can be defined, forming the first visible area, which corresponds to a horizontal angle of -15° to +15°, a vertical angle of -10° to +10°, and a distance of 0 meters to 10 meters.

[0051] The method for determining the second visual area based on eye-tracking information is as follows: taking the point where the gaze lands as the origin, expand left and right by a preset horizontal visual angle (e.g., a preset horizontal visual angle of 8°) and up and down by a preset vertical visual angle (e.g., a preset vertical visual angle of 5°). The field of vision covered by this expansion is the visual orientation corresponding to the second visual area. For example, if the eye-tracking information is: the user's gaze focuses on a point 3 meters directly in front, the coordinates of the gaze landing point are (640, 360) (assuming a display resolution of 1280×720), and the gaze stays for ≥1 second, then based on this eye-tracking information, the aforementioned visual orientation can be defined to form the second visual area. This area corresponds to a horizontal angle of -8° to +8°, ​​a vertical angle of -5° to +5°, and a distance of 0 meters to 3.5 meters.

[0052] After determining the first visible area and the second visible area, the first visible area can be used as the current field of view, or the second visible area can be used as the current field of view, or the overlapping area between the first and second visible areas can be used as the current field of view; there is no limitation here. It should be noted that if there is no overlap between the first and second visible areas, the process returns to step S11.

[0053] In one optional implementation, after acquiring eye-tracking information and head posture detection information, a first visible area in the eye's three-dimensional coordinate system is determined based on the eye-tracking information, and a second visible area in the head's three-dimensional coordinate system is determined based on the head posture detection information. Then, the first and second visible areas are mapped to a three-dimensional standard coordinate system to determine the degree of overlap between them. The eye's three-dimensional coordinate system can be used as the three-dimensional standard coordinate system, or the head's three-dimensional coordinate system can be used as the three-dimensional standard coordinate system, or a three-dimensional world coordinate system can be set and used as the three-dimensional standard coordinate system. It is understood that whether it is the eye coordinate system, the head coordinate system, or the standard coordinate system, all are three-dimensional coordinate systems that can accurately represent the user's visible range in both horizontal and vertical angles.

[0054] Furthermore, after determining the current field of view area, the current field of view area is a region generated in a standard coordinate system, with the user's line of sight and / or the user's head position as the origin, after being rotated left and right by a preset angle and shifted up and down by a preset angle. (Refer to...) Figure 4 , Figure 4This diagram illustrates the target area corresponding to the current field of view and the blind spot in a two-dimensional coordinate system. It shows the current field of view and the blind spot in a horizontal angle. The current field of view corresponds to a horizontal angle of -8° to +8°, ​​while the target area corresponding to the blind spot is a horizontal angle of -8° to +352°. It can be understood that the current field of view is the range of the real-world scene that can be directly observed, while the target area corresponding to the blind spot is the area outside the current field of view that the user cannot directly observe; that is, the area corresponding to the blind spot cannot be directly seen by the user's eye.

[0055] It is understandable that the blind spot range is larger than the visible range. If all video data within the blind spot range is displayed to the user, it is easy to distract the user and obstruct the view. Therefore, this application proposes a method to filter out the blind spot location required by the user from each candidate blind spot location within the blind spot range, and only push the content within the blind spot location to the user for viewing. It should be noted that the blind spot location is the angle range corresponding to the blind spot content that the user needs to pay attention to. The blind spot location includes the vertical angle range and the horizontal angle range.

[0056] In one optional implementation, the blind spot location can be determined by dividing the blind spot area into multiple fan-shaped regions, determining the angle difference between each fan-shaped region and the current field of view, and using the angle range corresponding to the fan-shaped region with an angle difference greater than or equal to a preset angle difference as the blind spot location. It is understood that when a user's eyes are looking at something, the smaller the angle difference between the fan-shaped region and the current field of view, the easier it is for the user to see the content of that fan-shaped region by rotating their eyes or slightly turning their head. Conversely, the larger the angle difference between the fan-shaped region and the current field of view, the harder it is for the user to see. In this case, displaying the content of the fan-shaped region that the user cannot easily see can provide the user with a wider field of view and improve the user experience of using AR glasses. For example, if the current field of view corresponds to a horizontal angle of -8° to +8°, ​​a vertical angle of -5° to +5°, and a distance of 0 meters to 3.5 meters, and the blind spot location the user is currently focusing on is in the left blind spot, then a horizontal angle of -20° to -10° and a vertical angle of -5° to +5° are used as the blind spot location.

[0057] In another optional implementation, the blind spot location can be determined by using the angle range corresponding to a fan-shaped area with an angle difference less than or equal to a preset angle difference as the blind spot location, so that the blind spot content closest to the user is displayed to the user. For example, when the user is in a preset scenario, such as a skiing scenario, the information of other skiers closer to the user is what the user needs to pay more attention to. In this case, displaying the blind spot content closest to the user can improve the accuracy of hazard avoidance. It should be noted that the preset angle difference can be a user-defined setting or a system configuration, and there are no restrictions here. For example, if the current field of view area is the area corresponding to a horizontal angle of -8° to +8°, ​​a vertical angle of -5° to +5°, and a distance of 0 meters to 3.5 meters, and the blind spot location that the user is currently focusing on is in the left blind spot, then the location corresponding to a horizontal angle of -18° to -8° and a vertical angle of -5° to +5° is used as the blind spot location.

[0058] In another optional implementation, the blind spot location can be determined by identifying the real-time content in each sector within the blind spot area. The sector containing the preset content is designated as the target sector, and the angle range corresponding to the target sector is used as the blind spot location. The preset content type can be obstacles, high-speed vehicles, etc., and can be customized according to user needs. For example, when a user is driving, the real-time content in the candidate blind spot location can include roadside vegetation, the sky behind the driver, and vehicles traveling to the side or behind. If the preset content is a moving vehicle, then the location of the moving vehicle is used as the blind spot location.

[0059] In another optional implementation, the direction of gaze deviation is determined based on eye-tracking information, and the direction of head rotation is determined based on head posture detection information. The location of the blind spot is determined based on the direction of gaze deviation and the direction of head rotation. When both the direction of gaze deviation and the direction of head rotation are to the left, the left blind spot is determined as the location of the blind spot, and the video data of the blind spot captured by the left blind spot camera is displayed in the blind spot display window. When both the direction of gaze deviation and the direction of head rotation are to the right, the right blind spot is determined as the location of the blind spot, and the video data of the blind spot captured by the right blind spot camera is displayed in the blind spot display window.

[0060] Optionally, to accurately determine the location of the blind spot, refer to Figure 5 Step S12 includes: Step S121: Determine the gaze offset angle based on the eye tracking information, and determine the head rotation angle based on the head posture detection information; Step S122: Determine the location of the blind spot based on the line of sight offset angle and the head rotation angle.

[0061] Optionally, after the user wears AR glasses, and the current field of view is determined based on eye tracking information and head posture detection information, the user continues to capture eye tracking information corresponding to the user's eyes based on the eye tracking module, and captures head posture detection information corresponding to the user's head based on the head posture sensor, and then determines the location of the blind spot based on the eye tracking information and the head posture detection information.

[0062] In one embodiment, the blind spot location can be determined by determining the gaze offset angle based on eye-tracking information. The gaze offset angle represents the angular interval between the gaze focus direction corresponding to the current visual field and the current gaze focus direction after eye movement. The current gaze focus direction is the gaze focus direction corresponding to eye movement. The larger the gaze offset angle, the larger the angular interval between the blind spot location and the gaze focus direction. For example, if the current visual field is a horizontal angle of -8° to +8° and the gaze focus direction is 0°, and the horizontal gaze offset angle corresponding to the user's eye glancing 1mm to the left is -10°, then the current gaze focus direction is -10° and the blind spot location is -18° to +8°; if the horizontal gaze offset angle corresponding to the user's eye glancing 2mm to the left is -20°, then the current gaze focus direction is +20° and the blind spot location is +12° to +28°.

[0063] In one embodiment, the blind spot location can also be determined by obtaining the head rotation angle based on head posture detection information, and determining the blind spot location based on the head rotation angle. The head rotation angle represents the angle between the user's head facing direction corresponding to the current field of vision area and the current head facing direction after the head is rotated. For example, if the current field of vision area is the area corresponding to the horizontal angle of -8° to +8°, ​​when the user turns 10 degrees to the left, the corresponding blind spot location is -18° to +8°.

[0064] Furthermore, the location of blind spots can also be determined by combining the angle of line of sight deviation and the angle of head rotation, including: The target offset angle is determined based on the line of sight offset angle and the head rotation angle. Determine the visible orientation corresponding to the current field of view area, and use the orientation obtained by offsetting the visible orientation by the target offset angle as the blind zone orientation.

[0065] In one embodiment, the target offset angle is determined based on the weight values ​​corresponding to eye tracking information and head posture detection information, the gaze offset angle, and the head rotation angle, respectively. After offsetting the visible orientation corresponding to the current field of vision area with the target offset angle, the offset orientation is determined as the blind zone orientation.

[0066] For example, if the weights of eye-tracking information and head posture detection information are 0.5 and 0.5 respectively, and the gaze offset angle corresponding to a 1mm leftward movement of the pupil is 10 degrees to the left, and the head rotation angle determined by the head posture detection information is also 10 degrees to the left, then the calculated target offset angle is 0.5*10+0.5*10=10. Then, taking the left edge of the visible orientation corresponding to the current visual area as the starting point, the 10-degree leftward offset is the blind spot orientation. The visible orientation is a horizontal angle of -8° to +8°. When the leftward offset is 10 degrees, the corresponding blind spot orientation is -18° to -8°. Figure 4 As shown, the angular range corresponding to blind zone A is the blind zone orientation.

[0067] It is understandable that the weight values ​​corresponding to eye tracking information and head posture detection information can be the same or different. For example, if head turning action precedes eye movement, then the weight value corresponding to head posture detection information is greater than the weight value corresponding to eye tracking information.

[0068] It should be noted that the eye-tracking module and head posture sensor can capture the user's eye-tracking information and head posture detection information in real time within a short period of time. Even if the user returns to the initial state after a short time, the captured eye-tracking information and head posture detection information can be used to determine the user's gaze deviation angle and head rotation angle in real time, thereby improving the accuracy of detecting blind spot locations.

[0069] In another optional implementation, when a user's eyes are tired, they may unconsciously rotate their pupils, but these pupils will quickly return to their initial position. This is simply a normal physiological reaction and does not require a response. Therefore, to prevent false triggering of the blind spot display, after capturing eye-tracking information and head posture detection information (including gaze duration and head orientation duration), if the gaze duration and / or head orientation duration exceed a preset duration, a step is performed to determine the location of the blind spot based on the gaze offset angle and the head rotation angle.

[0070] In another optional implementation, when a user's eyes or body are fatigued, they may unconsciously and frequently turn their head to relieve the fatigue. However, when the user is actually viewing content in the blind spot, they will not frequently turn their head. Instead, they will focus their pupils on the direction of the content in the blind spot or turn their head towards the blind spot. In this case, frequent head turning is simply a normal physiological reaction of the user, and there is no need to respond to the user's normal physiological reaction to avoid accidental triggering. Based on this, it is also possible to record the number of changes in eye movement and head movement within a preset time period. When the number of changes is less than or equal to the preset number of changes, the step of determining the location of the blind spot based on the gaze offset angle and the head rotation angle is executed.

[0071] Furthermore, by combining eye tracking with head posture detection, the trend of gaze and micro-head movements are captured. After locking the location of the three-dimensional target blind spot, the blind spot video data is filtered from the camera data according to the location of the blind spot, and the blind spot video data is displayed in the blind spot screen display window.

[0072] In this embodiment, the AR glasses are equipped with at least one blind spot camera. The blind spot camera is used to collect video data from different angle ranges. For example, blind spot cameras can be set on the left and right sides of the glasses to collect video data from the left and right blind spots. Blind spot cameras can also be set on the top and bottom of the glasses to collect video data from the top and bottom blind spots. It should be noted that the blind spot cameras in the AR glasses can collect content from blind spots at various angles to achieve shooting without blind spots.

[0073] After determining the location of the blind spot, blind spot video data corresponding to the location is filtered from the video data collected by the blind spot camera. This blind spot video data includes real-world scene information of the area corresponding to the blind spot location. For example, if a vehicle is present in the area corresponding to the blind spot location, the blind spot video data includes real-time video information of that vehicle. In one embodiment, video data corresponding to at least one acquisition angle range is acquired, and then target video data corresponding to a target acquisition angle range matching the blind spot location is acquired. This target video data is used as the blind spot video data, and the target blind spot video data is displayed in the blind spot display window. The display area corresponding to the blind spot display window is smaller than the display area of ​​the head-mounted device's display window.

[0074] Furthermore, after filtering out the blind spot video data, the blind spot video data is displayed in the blind spot display window. This window is used to display the blind spot video data, and users can set the display parameters of the blind spot display window within the AR glasses' display window. By setting these parameters, the blind spot video data can be displayed in the blind spot display window. If the display parameters include a display mode, and the mode is set to overlay display, the blind spot display window is overlaid on top of the real-world window. After displaying the blind spot video data in the blind spot display window, a picture-in-picture effect can be achieved.

[0075] It should be noted that display parameters include not only display mode but also other display parameters. Therefore, in order to highlight the display effect of the blind spot video data while avoiding obstructing the user's view of the real scene, the step of displaying the blind spot video data in the blind spot screen display window includes: Determine the display parameters of the blind spot display window, the display parameters including at least one of the display position, display area, and display transparency; According to the display parameters, the blind spot image display window that will display the blind spot video data will be superimposed on the display window of the head-mounted device.

[0076] In one optional implementation, the display parameters include the display mode, which can be a picture-in-picture overlay display, or selecting an edge area in the display window of the AR glasses and using that edge area as the display position of the blind spot screen display window, so that the blind spot video data is displayed in the same frame as other content. This application embodiment uses the picture-in-picture overlay display mode for explanation and analysis.

[0077] Optionally, the display parameters also include the display position. The display window of the AR glasses displays a virtual image, and the user can view the content after the virtual image is superimposed on the real scene through the display window. At this time, if the position of the blind spot display window is not set reasonably, it is easy to obstruct the user's view of the virtual image and the real scene. Based on this, by selecting a suitable display position for the blind spot display window and setting the blind spot display window at that display position, the obstruction of the content can be reduced. The display position can be the upper left corner of the display window, or the lower left corner of the display window, etc., and is not limited here.

[0078] Optionally, the display parameters also include the display area, which is used to characterize the size of the blind spot display window. If the display area is too large, it may obscure the current display content; if the display area is too small, it may cause the user to not be able to see the blind spot video data. Based on this, by setting the display area of ​​the blind spot display window, the user can see both the real scene information and the blind spot video data. For example, the display area can be set to 10% of the area of ​​the AR glasses' display window.

[0079] Optionally, display parameters also include display transparency. Lower transparency makes blind spot video data clearer but also makes it easier to obscure the current display content. Higher transparency makes blind spot video data harder to see. Therefore, setting display transparency allows for personalized display of the blind spot screen window, improving user experience. Understandably, the blind spot camera on the head-mounted device collects blind spot video data in real time. Through image recognition and content analysis, it identifies target objects and scene elements in the blind spot image, thereby determining the confidence level of the blind spot content. When the content contains valid target content that the user needs to focus on and has environmental reference or safety warning value, the corresponding content confidence level is higher. When the image is an empty background with no valid targets or irrelevant redundant content without safety-related elements, the corresponding content confidence level is lower. The head-mounted device dynamically matches and adjusts the display transparency of the blind spot image window based on the real-time calculated confidence level of the blind spot content. The two are negatively correlated: a higher confidence level indicates more effective targets and stronger reference value within the image, thus reducing the display transparency to ensure key content in the blind spot is clearly identifiable; conversely, a lower confidence level indicates fewer effective targets and weaker reference value, thus increasing the display transparency to reduce image obstruction. For example, when a nearby vehicle is detected in the blind spot video data, if the vehicle is close and there is a risk of movement, the confidence level of the blind spot content is high, and the system controls the blind spot image to be displayed with low transparency. As the vehicle moves away from the user and there is no longer any safety interference or observation need, the system determines that the effective confidence level of the blind spot content decreases and automatically increases the display transparency of the blind spot image window, weakening the display effect of useless blind spot images while avoiding obstruction of the normal field of vision and main display information.

[0080] It should be noted that the display parameters include, but are not limited to, the parameters mentioned above. They may also include the border color of the blind spot display window to highlight the blind spot display window, and the display duration to close the blind spot display window in a timely manner to avoid obstructing the currently displayed content. It is understood that the blind spot video data displayed in the blind spot display window of this application is collected in real time, that is, it is displayed in real time based on the blind spot video collected by the blind spot camera.

[0081] Further, after outputting the blind spot image display window containing blind spot video data, steps S20 to S30 are executed.

[0082] Step S20: During the process of displaying the blind spot video data, the actual field of vision area is determined based on eye tracking information and head posture detection information; Step S30: If the actual field of view overlaps with the target area corresponding to the blind spot, hide the blind spot display window.

[0083] It is understandable that when a user turns their head or eyes, the visible orientation changes, and the real scene information seen by the user through AR glasses also changes. If the changed visible orientation overlaps with the blind spot orientation, it will cause the real scene information seen by the user to overlap with the blind spot video data, affecting the user's normal observation of the real scene information. Based on this, after outputting the blind spot video data, the eye-tracking module and head posture sensor of AR glasses continue to collect the user's eye-tracking information and head posture detection information. Based on the eye-tracking information and head posture detection information, the actual field of view area is determined. The actual field of view area is the field of view area corresponding to the user's head or eyeballs after turning based on the current field of view area. This actual field of view area is different from the current field of view area.

[0084] Furthermore, after determining the actual field of view area, it is determined whether the user is observing the real scene content corresponding to the blind spot area based on the actual field of view area. If the user is already directly observing the real scene content by turning their head, the user's visual orientation has changed, that is, the visual orientation now covers the blind spot area. The user no longer needs to view the blind spot content through picture-in-picture. Based on this, if it is found that the actual field of view area overlaps with the target area corresponding to the blind spot area, the blind spot screen display window is hidden, and the user can directly view the real scene information corresponding to the blind spot area through the display window of the AR glasses.

[0085] In one optional implementation, the line-of-sight offset angle and head rotation angle are obtained, the actual field of view area is determined based on the line-of-sight offset angle and head rotation angle, and then the actual field of view area is compared with the target area corresponding to the blind spot location. When the two areas overlap, the blind spot display window is hidden.

[0086] Optionally, the blind spot display window can be hidden by gradually increasing its transparency to achieve a smooth fade-out effect. Alternatively, the transparency can be gradually increased as the user turns their head or moves their pupils.

[0087] Furthermore, step S20 is followed by: If the actual field of view area does not overlap with the target area corresponding to the blind spot location, then return to the step of displaying the blind spot video data corresponding to the blind spot location in the blind spot screen display window.

[0088] It should be noted that even if the picture-in-picture is hidden due to overlapping fields of view, the data stream from the blind spot camera corresponding to the displayed blind spot video data can still be continuously processed in the background. When the user's field of view shifts again, the blind spot display window immediately displays the latest blind spot video data, rather than the old image before it was hidden. In one implementation, an index is established for the blind spot video data. When the actual field of view is detected to have returned to the original field of view area, i.e., the current field of view area, the latest blind spot video data is retrieved based on the index and displayed in the blind spot display window.

[0089] Understandably, by comparing the user's actual field of view with the target area corresponding to the blind spot in real time, the problem of the main field of view being easily obstructed within the fixed display blind spot is solved. When the fields of view overlap, the blind spot is automatically hidden. When there is no overlap between the actual field of view and the target area corresponding to the blind spot, the video data of the blind spot corresponding to the display blind spot is maintained, balancing blind spot observation and main field of view clarity, and improving the smoothness of the visual experience.

[0090] In this embodiment, by setting an eye-tracking module and a head posture sensor on the head-mounted device, eye-tracking information and head posture detection information are collected to determine the user's current field of vision and blind spot location. Then, the blind spot video data corresponding to the blind spot location collected by the blind spot camera is overlaid on the AR glasses' display window in a picture-in-picture manner. This allows the user to see the real scene information clearly and also to view the blind spot content in real time without turning their head or moving their eyes. While displaying the blind spot video data, the eye-tracking information and head posture detection information are used to determine if the field of vision has changed. If it is determined that the user's current field of vision has changed to the actual field of vision, it is further determined whether the actual field of vision overlaps with the target area corresponding to the blind spot location. If so, the blind spot display window is hidden to conceal the displayed blind spot video data and prevent duplicate blind spot video data from obstructing the user's view of the real scene information.

[0091] Based on the first embodiment, referring to Figure 6 Step S20 includes steps S21 to S22.

[0092] Step S21: Determine the user's actual visual location based on the eye-tracking information and the head posture detection information; Step S22: Determine the actual field of view area based on the actual visible orientation.

[0093] In this embodiment, the gaze offset angle is determined based on the eye tracking information, the head offset angle is determined based on the head posture detection information, the actual offset angle is determined based on the gaze offset angle and the head offset angle, and the actual visible orientation corresponding to the current visual field area is offset by the actual offset angle to obtain the orientation as the actual visible orientation, and then the area corresponding to the actual visible orientation is taken as the actual visual field area. It can be understood that the actual visible orientation is the visible range corresponding to the user's eyeball rotation or head rotation.

[0094] In another optional implementation, the actual visible direction can be determined based on the user's body rotation direction. For example, when the user wears AR glasses and turns towards the blind spot, the corresponding actual visible direction is the blind spot.

[0095] In this embodiment of the application, when the user's head or eyes are actually rotated, the user's actual visual position is determined, and the actual field of view area is determined based on the actual visual position to determine whether it is necessary to hide the display window of the blind spot and avoid obstructing the real scene information.

[0096] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the control method of the head-mounted device of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0097] This application provides a head-mounted device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the control method of the head-mounted device in the first embodiment described above.

[0098] The following is for reference. Figure 7 The diagram illustrates a structural schematic suitable for implementing the head-mounted device of the embodiments of this application. The head-mounted device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The head-mounted device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0099] like Figure 7As shown, the head-mounted device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the head-mounted device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the head-mounted device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show head-mounted devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0100] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0101] The head-mounted device provided in this application, employing the control method of the head-mounted device in the above embodiments, can solve the technical problem of blind spot content display windows obstructing the user's actual field of vision. Compared with the prior art, the beneficial effects of the head-mounted device provided in this application are the same as the beneficial effects of the control method of the head-mounted device provided in the above embodiments, and other technical features of this head-mounted device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0102] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0104] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the control method of the head-mounted device in the above embodiments.

[0105] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0106] The aforementioned computer-readable storage medium may be included in the head-mounted device or may exist independently and not assembled into the head-mounted device.

[0107] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a head-mounted device, enable the head-mounted device to write computer program code for performing the operations of this application in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation that may be implemented in systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0110] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the control method of the head-mounted device described above, which can solve the technical problem of blind spot content display window obstructing the user's actual field of vision. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the control method of the head-mounted device provided in the above embodiments, and will not be repeated here.

[0111] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the control method for a head-mounted device as described above.

[0112] The computer program product provided in this application can solve the technical problem of blind spot content display windows obstructing the user's actual field of vision. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the control method of the head-mounted device provided in the above embodiments, and will not be repeated here.

[0113] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A control method for a head-mounted device, characterized in that, The steps of the control method for the head-mounted device include: In response to the blind spot display command, the video data of the blind spot corresponding to the location of the blind spot is displayed in the blind spot screen display window; During the display of the blind spot video data, the actual field of vision is determined based on eye-tracking information and head posture detection information; If the actual field of view overlaps with the target area corresponding to the blind spot, the blind spot display window is hidden.

2. The control method for a head-mounted device as described in claim 1, characterized in that, The step of displaying the blind spot video data corresponding to the blind spot location in the blind spot display window includes: The current visual field area is determined based on the eye-tracking information and the head posture detection information; The location of the blind spot is determined based on the current field of view; Based on the location of the blind spot, the video data of the blind spot is filtered from the camera data, and the video data of the blind spot is displayed in the blind spot screen display window.

3. The control method for a head-mounted device as described in claim 2, characterized in that, The step of determining the location of the blind spot based on the current field of view includes: The eye-tracking information is used to determine the gaze deviation angle, and the head rotation angle is used to determine the head posture detection information. The location of the blind spot is determined based on the angle of visual deviation and the angle of head rotation.

4. The control method for a head-mounted device as described in claim 3, characterized in that, The step of determining the location of the blind spot based on the line of sight offset angle and the head rotation angle includes: The target offset angle is determined based on the line of sight offset angle and the head rotation angle. Determine the visible orientation corresponding to the current field of view area, and use the orientation obtained by offsetting the visible orientation by the target offset angle as the blind zone orientation.

5. The control method for a head-mounted device as described in claim 1, characterized in that, The step of determining the actual visual field area based on the eye-tracking information and the head posture detection information during the display of the blind spot video data includes: The user's actual visual orientation is determined based on the eye-tracking information and the head posture detection information. The actual field of view area is determined based on the actual visible orientation.

6. The control method for a head-mounted device as described in claim 2, characterized in that, The step of filtering the blind spot video data from the camera data based on the blind spot location and displaying the blind spot video data in the blind spot screen display window includes: Acquire video data corresponding to at least one range of acquisition angles; Acquire target video data corresponding to the target acquisition angle range that matches the blind zone orientation, and use the target video data as the blind zone video data; The target blind spot video data is displayed in the blind spot image display window, and the display area corresponding to the blind spot image display window is smaller than the display area of ​​the head-mounted device's display window.

7. The control method for a head-mounted device as described in claim 6, characterized in that, The step of displaying the blind spot video data corresponding to the blind spot location in the blind spot display window includes: Determine the display parameters of the blind spot display window, the display parameters including at least one of the display position, display area, and display transparency; According to the display parameters, the blind spot image display window that will display the blind spot video data will be superimposed on the display window of the head-mounted device.

8. The control method for a head-mounted device as described in claim 1, characterized in that, After determining the actual visual field area based on the eye-tracking information and the head posture detection information during the process of displaying the blind spot video data, the method further includes: If the actual field of view does not overlap with the target area corresponding to the blind spot, then return to the step of displaying the blind spot video data corresponding to the blind spot location in the blind spot display window.

9. A head-mounted device, characterized in that, The head-mounted device further includes a memory and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the head-mounted device as claimed in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the control method for the head-mounted device as described in any one of claims 1 to 8.

Images