Head-mounted display device, display system and image acquisition device

Abstract

The invention discloses a head-mounted display device, a display system and an image acquisition device, and belongs to the technical field of intelligent wearable devices. The head-mounted display device comprises an eye movement detection assembly, an image acquisition device and a display assembly. The eye movement detection assembly obtains a gazing position of a user; the image acquisition equipment acquires an initial image; the image acquisition device processes the initial image based on the fixation position to obtain a display image; the display component displays the display image, the display image comprises a first part and a second part, and the resolution ratio of the first part is higher than that of the second part; the first part corresponds to the gazing position, so that the gazing position of the user falls within the range of the first part under the condition that the user views the display image.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent wearable devices, and in particular to a head-mounted display device, a display system, and an image acquisition device. Background Art

[0002] With the development of technology, various head-mounted smart wearable devices are becoming more and more popular. Among them, Extended Reality (XR) headsets are relatively common head-mounted smart wearable devices. In related technologies, the screen of a head-mounted smart wearable device is divided into two left and right display screens, which are used to provide viewing images to the left and right eyes respectively. In the process of watching live images based on a head-mounted smart wearable device, the shooting device usually needs to transmit the image data of the left and right cameras to the head-mounted smart wearable device, and then the head-mounted smart wearable device displays the image data of the left and right cameras on the left and right display screens respectively.

[0003] To improve the clarity of live broadcasts, the resolution of live broadcast cameras is relatively high. For example, if two cameras have a resolution of 4K, the amount of data required to transmit two compressed 4K videos simultaneously will still be relatively large. Furthermore, the need to transmit high-resolution data from two cameras during live broadcasts requires a larger network bandwidth. If network bandwidth is limited, the resolution of the two cameras can only be reduced, which affects the clarity of the live broadcast for users. Consequently, the clarity of the live broadcast images viewed by users on head-mounted smart wearable devices is lower. Summary of the Invention

[0004] The embodiments of the present application provide a head-mounted display device, a display system, and an image acquisition device, which can solve the problem of low clarity of live broadcast images viewed by users based on head-mounted smart wearable devices.

[0005] In a first aspect, a head-mounted display device is provided, comprising an eye movement detection component, an image acquisition device, and a display component;

[0006] The eye movement detection component obtains the user's gaze position;

[0007] The image acquisition device acquires an initial image;

[0008] The image acquisition device processes the initial image based on the gaze position to obtain a display image;

[0009] The display component displays the display image, which includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

[0010] In a second aspect, a display system is provided, comprising a head-mounted display device and an image acquisition device, the head-mounted display device and the image acquisition device being wirelessly connected to each other, the head-mounted display device including an eye movement detection component and a display component;

[0011] The head mounted display device is used to obtain the user's gaze position based on the eye movement detection component;

[0012] The head mounted display device is further configured to send the gaze position to the image acquisition device;

[0013] The image acquisition device is used to acquire an initial image;

[0014] The image acquisition device is further configured to process the initial image based on the gaze position to obtain a display image;

[0015] The image acquisition device is further configured to send the display image to the head-mounted display device;

[0016] The head-mounted display device is also used to display the display image based on the display component, and the display image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

[0017] According to a third aspect, an image acquisition device is provided, which is wirelessly connected to a head-mounted display device provided separately, and the image acquisition device receives a user's gaze position transmitted by the head-mounted display device;

[0018] The image acquisition device acquires an initial image;

[0019] The image acquisition device processes the initial image based on the gaze position to obtain a display image;

[0020] The image acquisition device sends the display image to the head-mounted display device;

[0021] The displayed image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that the gaze position falls within the range of the first part.

[0022] In a fourth aspect, a head-mounted display device is provided, which is wirelessly connected to a separately provided image acquisition device, wherein the head-mounted display device includes an eye movement detection component and a display component;

[0023] The head mounted display device obtains the user's gaze position based on the eye movement detection component;

[0024] The head mounted display device sends the gaze position to the image acquisition device;

[0025] The head mounted display device receives the display image sent by the image acquisition device;

[0026] The head-mounted display device displays the display image based on the display component, and the display image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

[0027] In an embodiment of the present application, the user's gaze position is obtained through an eye movement detection component, and the initial image is processed based on the gaze position by an image acquisition device to obtain a display image. Since the display image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part. That is, in the display image, only some areas have a high resolution, and other areas have a low resolution. This helps to reduce the size of the display image, thereby reducing the network bandwidth required for transmitting the display image and reducing the time consumed in transmitting the display image. In addition, since the first part corresponds to the gaze position, when the user views the display image, the user's gaze position falls within the range of the first part, and the resolution of the first part is higher than the resolution of the second part. Therefore, when the user views the initial image, the resolution of the user's gaze area can be improved, thereby improving the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic structural diagram of a head-mounted display device provided in an embodiment of the present application;

[0029] Figure 2 This is a diagram of how to determine the gaze area on the XR device screen based on the eye movement camera;

[0030] Figure 3 is a schematic diagram of a process of processing a first initial image and a second initial image to obtain a first display image and a second display image;

[0031] Figure 4 It is a flowchart of the image fusion process;

[0032] Figure 5 This is a flowchart of a data transmission method based on a head-mounted display device provided in an embodiment of the present application;

[0033] Figure 6 It is a structural diagram of a display system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0035] The terms "first," "second," and the like in this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable, where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein. Furthermore, the objects distinguished by "first" and "second" generally refer to a class and do not limit the number of objects. For example, the first object can be one or more. Furthermore, "or" in this application represents at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three options: Option 1: includes A but not B; Option 2: includes B but not A; and Option 3: includes both A and B. Furthermore, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three options, respectively. The character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0036] The term "indication" in this application can be either a direct indication (or explicit indication) or an indirect indication (or implicit indication). A direct indication can be understood as the sender explicitly informing the receiver of specific information, the operation to be performed, or the result of the request in the instruction sent; an indirect indication can be understood as the receiver determining the corresponding information based on the instruction sent by the sender, or making a judgment and determining the operation to be performed or the result of the request based on the judgment result.

[0037] The head-mounted display device 300, display system and image acquisition device 200 provided in the embodiments of the present application are described in detail below with reference to some embodiments and their application scenarios in conjunction with the accompanying drawings.

[0038] See Figure 1, an embodiment of the present application provides a head-mounted display device 300, including an eye movement detection component 110, an image acquisition device 200 and a display component 120;

[0039] The eye movement detection component 110 obtains the user's gaze position;

[0040] The image acquisition device 200 acquires an initial image;

[0041] The image acquisition device 200 processes the initial image based on the gaze position to obtain a display image;

[0042] The display component 120 displays the display image, which includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

[0043] The head-mounted display device 300 can be any type of smart wearable head-mounted device, such as any type of XR head-mounted display device or a mixed reality (MR) device. The XR head-mounted display device is a wearable device that combines virtual reality (VR) and augmented reality (AR) technologies. In some embodiments of the present application, the head-mounted display device 300 is an XR head-mounted display device with a video see-through (VST) function.

[0044] The eye movement detection component 110 may be any eye movement camera or eye movement sensor in the head-mounted display device 300 for detecting the user's eye movement input. The display component 120 may be any display device in the head-mounted display device 300 for displaying user images. Acquiring the user's gaze position by the eye movement detection component 110 may mean that the eye movement detection component 110 acquires the user's gaze position while the user is viewing an image captured by the image capture device 200 through the head-mounted display device 300.

[0045] It should be noted that the image acquisition device 200 can be separate from the main body 100 of the head-mounted display device 300. The main body 100 includes the eye movement detection component 110, the display component 120, and related wearing components. A user can wear the main body 100 while using the head-mounted display device 300. Alternatively, the image acquisition device 200 can be integrated into the head-mounted display device 300.

[0046] For example, in some embodiments of the present application, when the above-mentioned image acquisition device 200 is an image acquisition device 200 separated from the main part 100 of the head-mounted display device 300, the above-mentioned image acquisition device 200 can be a shooting device in a live broadcast scene. Accordingly, the scene in which the user watches the image captured by the image acquisition device 200 based on the head-mounted display device 300 is: the scene in which the user watches the live broadcast picture based on the head-mounted display device 300.

[0047] For another example, in some embodiments of the present application, when the above-mentioned image acquisition device 200 is an image acquisition device 200 separated from the main part 100 of the head-mounted display device 300, the above-mentioned image acquisition device 200 may also be a shooting device in a drone. Accordingly, the scene in which the user watches the image captured by the image acquisition device 200 based on the head-mounted display device 300 is: the scene in which the user watches the real-time picture captured by the shooting device in the drone based on the head-mounted display device 300 in real time.

[0048] For another example, in some embodiments of the present application, the image acquisition device 200 may also be an image acquisition device 200 integrated into the head-mounted display device 300. Specifically, it may be a camera in the head-mounted display device 300 for the user to capture external images.

[0049] It should be noted that when the image acquisition device 200 is separate from the main body 100 of the head-mounted display device 300, after the image acquisition device 200 acquires the initial image and processes the initial image to obtain the display image, the image acquisition device 200 can transmit the display image to the main body 100 for display via a network connection with the main body 100. Accordingly, when the image acquisition device 200 is integrated into the head-mounted display device 300, after the image acquisition device 200 acquires the initial image and processes the initial image to obtain the display image, the image acquisition device 200 can transmit the display image to the display component 120 for display via a data transmission path within the head-mounted display device 300.

[0050] The first portion corresponds to the gaze position, meaning that the first portion is located at the location indicated by the gaze position. Accordingly, the second portion is the portion other than the first portion. In other words, the first portion is the gaze area in the displayed image, and the second portion is the non-gaze area in the displayed image.

[0051] For ease of understanding, the head-mounted display device 300 in the embodiment of the present application is further explained below by taking the application of the head-mounted display device 300 in a live broadcast scenario as an example.

[0052] The process of the above-mentioned user watching the image captured by the image acquisition device 200 based on the head-mounted display device 300 can be: the process of the user watching the live image based on the VST function of the head-mounted display device 300. The head-mounted display device 300 provided based on the embodiment of the present application can make the live image seen by the user clearer and have a 3D effect, and can reduce the amount of data transmitted between the live camera and the XR headset.

[0053] When a user is watching a live broadcast using the head-mounted display device 300, the eye movement detection component 110 in the head-mounted display device 300 can detect the user's gaze area in real time, and generate gaze position information based on the position of the gaze area in the live broadcast. The eye movement detection component 110 then sends the generated gaze position information to the image acquisition device 200. Since the position of the user's gaze area will not change much in a short period of time while watching the live broadcast, the image acquisition device 200 can process one or more subsequent frames after receiving the gaze position information. It is understandable that since the user's gaze area may change as the content of the image changes while watching the live broadcast, the eye movement detection component 110 can continuously detect the user's gaze area and continuously send the generated gaze position information to the image acquisition device 200. The image acquisition device 200 can process the currently captured image based on the most recently received gaze position information.

[0054] The above-mentioned display image can be a live image. Specifically, the display image can include a left-eye image and a right-eye image for simultaneous display on the head-mounted display device 300. In this way, when the display component 120 displays the display image, the user's left eye can view the left-eye image, and the user's right eye can view the right-eye image. Alternatively, the display image can include only one image. In this case, when the display component 120 displays the image, both eyes of the user can view the same image.

[0055] The processing means in the process of processing the initial image based on the gaze position information may include resolution processing means such as resolution enhancement processing and resolution reduction processing, wherein the resolution enhancement processing may include common processing means for increasing image resolution such as interpolation processing and image fusion. Correspondingly, the resolution reduction processing may be a common processing means for reducing image resolution such as downsampling the image. For example, when the resolution of the image captured by the image acquisition device 200 is high, the resolution reduction processing may be performed on the non-gaze area in the image captured by the image acquisition device 200 using the processing means for reducing the resolution. For another example, when the resolution of the image captured by the image acquisition device 200 is low, the resolution enhancement processing may be performed on the gaze area in the image captured by the image acquisition device 200 using the processing means for enhancing the resolution.

[0056] In the embodiment of the present application, since the eyes of the user wearing the head mounted display device 300 are more sensitive to the gazing area, a high resolution is required, while the non-gazing area is less sensitive and a low resolution can be used, thereby achieving the purpose of improving the user's visual effect while reducing the network bandwidth required for the data transmission process. Figure 1 , is a flowchart of a data transmission process of a head mounted display device 300 provided in an embodiment of the present application. The transmission process specifically includes the following steps:

[0057] The eye movement detection component 110 in the head mounted display device 300 detects the gaze point area of ​​the user's eyes.

[0058] The main body 100 of the head mounted display device 300 transmits the gaze point area of ​​the user's eyes to the image acquisition device 200 via the network.

[0059] After the image acquisition device 200 obtains the gaze point area, the high-resolution camera is used to obtain the image of the gaze point area, and the low-resolution camera of the camera is used to obtain the image of the non-gaze point area to obtain video data, wherein the video data is the above-mentioned initial image.

[0060] The image acquisition device 200 performs video compression encoding processing on the video data based on the video encoding module. During the compression encoding process, the video data is divided into left-eye image compression and right-eye image compression, the gaze point area uses the image data compression of the high-resolution camera, and the non-gaze area uses the image data compression of the low-resolution camera.

[0061] The image acquisition device 200 transmits the compressed and encoded video data to the main body 100 of the head-mounted display device 300 .

[0062] After receiving the video data, the main body 100 decompresses the video data and displays the video data on the left and right display screens of the MR device.

[0063] In this embodiment, the user's gaze position is acquired by the eye movement detection component 110, and the initial image is processed based on the gaze position by the image acquisition device 200 to obtain a display image. Since the display image includes a first portion and a second portion, the resolution of the first portion is higher than that of the second portion. That is, in the display image, only some areas have a higher resolution, while other areas have a lower resolution. This helps reduce the size of the display image, thereby reducing the network bandwidth required for transmitting the display image and reducing the time required to transmit the display image. In addition, since the first portion corresponds to the gaze position, when the user views the display image, the user's gaze position can fall within the range of the first portion. Since the resolution of the first portion is higher than that of the second portion, the resolution of the user's gaze area can be increased during the user's viewing of the initial image, thereby improving the user's visual experience.

[0064] Optionally, the image acquisition device 200 includes a first camera 210 and a second camera 220; the initial image includes a first initial image acquired by the first camera 210 and a second initial image acquired by the second camera 220;

[0065] The first initial image has a first resolution, and the second initial image has a second resolution, wherein the first resolution is higher than the second resolution; wherein the first initial image includes the first portion, and the second initial image includes the second portion;

[0066] The processing of the initial image based on the gaze position to obtain the display image includes: combining the first portion corresponding to the gaze position in the first initial image with the second portion in the second initial image to obtain the display image.

[0067] The specific values ​​of the first resolution and the second resolution can be set as needed. For example, in some embodiments of the present application, the first resolution is 4K and the second resolution is 2K. It will be understood that 4K and 2K in this embodiment are only examples. In fact, the first resolution and the second resolution can also be other values, as long as the resolution of the first camera 210 is higher than the resolution of the second camera 220.

[0068] It is understandable that, in the embodiment of the present application, the displayed image includes only one image. In this case, during the display process of the display component 120, both eyes of the user can view the same image.

[0069] The first portion of the first initial image corresponding to the gaze position may be an area in the first initial image determined based on the gaze position, i.e., the first portion is located at the position indicated by the gaze position. Accordingly, the second portion of the second initial image is located at a location other than the location indicated by the gaze position.

[0070] The shooting angles of the first initial image and the second initial image may be the same or different. When the shooting angles of the first initial image and the second initial image are the same, the first portion of the first initial image corresponding to the gaze position may be directly combined with the second portion of the second initial image to obtain the displayed image. Accordingly, when the shooting angles of the first initial image and the second initial image are different, the first initial image or / and the second initial image may be first subjected to a perspective conversion to convert them to the same perspective. Then, the first portion of the first initial image corresponding to the gaze position having the same perspective may be combined with the second portion of the second initial image to obtain the displayed image.

[0071] In some embodiments of the present application, the first initial image includes the first part, and the second initial image includes the second part; the processing of the first initial image based on the gaze position to obtain the first display image includes: combining the first part corresponding to the gaze position in the first initial image with the second part in the second initial image to obtain the first display image; the processing of the second initial image based on the gaze position to obtain the second display image includes: combining the first part corresponding to the gaze position in the first initial image with the second part in the second initial image to obtain the second display image; the first part in the first display image and the first part in the second display image both correspond to the gaze position, so that when the user views the first display image and the second display image, the user's gaze position falls within the range of the first part in the first display image and / or the first part in the second display image.

[0072] In this embodiment, the display image is obtained by combining the first portion of the first initial image corresponding to the gaze position with the second portion of the second initial image. In this way, the resolution of the gaze area in the obtained display image can be higher than that of the non-gaze area.

[0073] Optionally, the image acquisition device 200 includes a first camera 210 and a second camera 220, and the initial image includes a first initial image acquired by the first camera 210 and a second initial image acquired by the second camera 220; the first initial image has a first resolution, and the second initial image has a second resolution, and the first resolution is higher than the second resolution;

[0074] Processing the initial image based on the gaze position to obtain a display image includes: processing the first initial image based on the gaze position to obtain a first display image; processing the second initial image based on the gaze position to obtain a second display image; the display image includes the first display image and the second display image; wherein the first display image and the second display image both include the first part and the second part, the resolution of the first part in the first display image is higher than the resolution of the second part in the first display image, and the resolution of the first part in the second display image is higher than the resolution of the second part in the second display image;

[0075] The display assembly 120 includes a first display device 121 and a second display device 122 . The first display device 121 displays the first display image, and the second display device 122 displays the second display image.

[0076] Among them, one of the first display device 121 and the second display device 122 is a left-eye display screen, and the other is a right-eye display screen. Accordingly, of the first display image and the second display image, one is a left-eye image, and the other is a right-eye image. In other words, the display images include the left-eye image and the right-eye image for simultaneous display on the head-mounted display device 300.

[0077] It should be noted that, in this embodiment, there may be a viewing angle difference between the first display image and the second display image, and the viewing angle difference between the first display image and the second display image may be equal to or close to the viewing angle difference between the user's eyes, so that the picture viewed by the user has a 3D effect.

[0078] In this embodiment, the display image includes the first display image and the second display image; wherein the first display image and the second display image both include the first part and the second part, the resolution of the first part in the first display image is higher than the resolution of the second part in the first display image, and the resolution of the first part in the second display image is higher than the resolution of the second part in the second display image; the display component 120 includes a first display device 121 and a second display device 122, the first display device 121 displays the first display image, and the second display device 122 displays the second display image. In this way, by generating the first display image and the second display image corresponding to the left and right eyes of the user respectively, it is beneficial to further improve the visual experience of the user when viewing the display image.

[0079] Optionally, the display side of the first display device 121 faces the left eye of the user, and the display side of the second display device 122 faces the right eye of the user;

[0080] An angle between the orientation of the display side of the first display device 121 and the orientation of the display side of the second display device 122 is greater than 5° and less than 150°.

[0081] Specifically, the angle between the display side of the first display device 121 and the display side of the second display device 122 can be equal to the difference in viewing angle between the user's two eyes. For example, the display side of the first display device 121 can face the user's left eye, while the display side of the second display device 122 can face the user's right eye. In this case, the first display device 121 and the second display device 122 can display the same image. Because the display sides of the two display devices are at an angle, the two identical images displayed by them have a viewing angle difference, thereby achieving a 3D effect.

[0082] In this embodiment, by making the angle between the orientation of the display side of the first display device 121 and the orientation of the display side of the second display device 122 greater than 5° and less than 150°, the image displayed by the display component 120 can have a 3D effect when the displayed image only includes one image.

[0083] Optionally, the screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion;

[0084] The sampling angle of view of the first camera 210 is different from the sampling angle of view of the second camera 220, so that a first angle difference is formed between the angles of view of the first initial image and the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference;

[0085] The processing of the first initial image based on the gaze position to obtain a first display image includes: fusing first screen content in the first initial image with first screen content in the second initial image to obtain the first portion; using second screen content in the second initial image as the second portion, or fusing the second screen content in the first initial image with second screen content in the second initial image to obtain the second portion; and combining the first portion with the second portion to obtain the first display image;

[0086] The processing of the second initial image based on the gaze position to obtain the second display image includes: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; using the second screen content in the second initial image as the second part, or fusing the second screen content in the first initial image with the second screen content in the second initial image as the second part, and combining the first part and the second part to obtain the second display image.

[0087] The sampling angle of view may also be referred to as the camera's shooting angle of view. The first angle difference may be equal to or similar to the angle of view difference between the user's two eyes.

[0088] It can be understood that the above-mentioned first initial image and second initial image are two images taken by the first camera 210 and the second camera 220 at the same time and from different perspectives of the same subject. In this way, the first picture content in the first initial image can be the same as the first picture content in the second initial image, and the perspective of the first picture content in the first initial image and the perspective of the first picture content in the second initial image have the first angle difference.

[0089] In some embodiments of the present application, the first initial image is processed based on the gaze position to obtain a first display image, including: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part and the second part to obtain the first display image.

[0090] In some embodiments of the present application, the first initial image is processed based on the gaze position to obtain a first display image, including: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; fusing the second screen content in the first initial image with the second screen content in the second initial image as the second part, and combining the first part and the second part to obtain the second display image.

[0091] In some embodiments of the present application, the second initial image is processed based on the gaze position to obtain a second display image, including: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part with the second part to obtain the second display image.

[0092] In some embodiments of the present application, the second initial image is processed based on the gaze position to obtain a second display image, including: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; fusing the second screen content in the first initial image with the second screen content in the second initial image as the second part, and combining the first part and the second part to obtain the second display image.

[0093] In the above-mentioned embodiments, the fusion of the picture contents of the two images may be performed based on various image storage and processing methods. For example, the features of the two images may be superimposed, or the average of the features of the two images may be taken.

[0094] In this embodiment, the first portion is obtained by fusing the first screen content in the first initial image with the first screen content in the second initial image; the second screen content in the second initial image is used as the second portion, or the second screen content in the first initial image is fused with the second screen content in the second initial image to form the second portion; and the first portion and the second portion are combined to obtain the first display image. Simultaneously, the first portion is obtained by fusing the first screen content in the first initial image with the first screen content in the second initial image; the second screen content in the second initial image is used as the second portion, or the second screen content in the first initial image is fused with the second screen content in the second initial image to form the second portion; and the first portion and the second portion are combined to obtain the second display image. Thus, the second display image is generated.

[0095] Optionally, the screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion;

[0096] The sampling angle of view of the first camera 210 is different from the sampling angle of view of the second camera 220, so that a first angle difference is formed between the angles of view of the first initial image and the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference;

[0097] The processing the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained by the resolution reduction processing with the first part to obtain the first display image; and / or

[0098] The processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image.

[0099] It should be noted that in this embodiment, the two solutions connected by "and / or" can be executed simultaneously, or only one of them can be executed, and the other can be replaced by the combination described in the above embodiment. When the two solutions are executed simultaneously, the generated display image effect is better than when only one of them is executed.

[0100] In some embodiments of the present application, the processing of the first initial image based on the gaze position to obtain the first display image includes: performing resolution reduction processing on the second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained after the resolution reduction processing with the first part to obtain the first display image. The implementation process of processing the second initial image based on the gaze position to obtain the second display image can be: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; using the second screen content in the second initial image as the second part, or fusing the second screen content in the first initial image with the second screen content in the second initial image as the second part, and combining the first part with the second part to obtain the second display image.

[0101] In some embodiments of the present application, the processing of the first initial image based on the gaze position to obtain a first display image includes: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; using the second screen content in the second initial image as the second part, or fusing the second screen content in the first initial image with the second screen content in the second initial image as the second part; and combining the first part with the second part to obtain the first display image. The processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image.

[0102] In some embodiments of the present application, the processing of the first initial image based on the gaze position to obtain the first display image includes: performing down-resolution processing on the second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained by down-resolution processing with the first part to obtain the first display image; the processing of the second initial image based on the gaze position to obtain the second display image includes: performing up-resolution processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image. It should be noted that since this embodiment adopts down-resolution processing for obtaining the first display image and up-resolution processing for obtaining the second display image, the embodiment of the present application has better effect than the previous two embodiments.

[0103] It should be noted that the resolution increase processing in the above-mentioned embodiment may include various means capable of increasing image resolution, such as interpolation processing, image fusion processing, and other processing based on various algorithms capable of increasing image resolution. Correspondingly, the resolution reduction processing may include various means capable of reducing image resolution, such as downsampling processing, and other processing based on various algorithms capable of reducing image resolution.

[0104] The first screen content may be the screen content of the gaze area, and correspondingly, the second screen content may be the screen content of the non-gaze area.

[0105] In this embodiment, the second portion is obtained by downscaling the second content in the first initial image. The first content in the first initial image is used as the first portion, and the second portion obtained through downscaling is combined with the first portion to obtain the first display image. This reduces the resolution of the non-attention area in the first initial image, thereby reducing the size of the first display image. Furthermore, since the first initial image itself has a relatively high resolution, the first content in the first initial image can be directly used as the first portion of the first display image, thereby ensuring that the attention area in the obtained first display image has a relatively high resolution. Accordingly, the first content in the second initial image is upscaled to obtain first content with a higher resolution. This first content with a higher resolution is then fused with the first content in the first initial image to obtain the first portion. The second content in the second initial image is used as the second portion, and the first portion obtained through fusion is combined with the second portion to obtain the second display image. Since the second initial image itself has a relatively low resolution, upscaling the first content in the second initial image and using the first content with a higher resolution as the first portion of the second display image improves the resolution of the attention area in the second display image. At the same time, the second screen content in the second initial image is directly used as the second part of the second display image, so that the size of the second display image can be reduced.

[0106] Optionally, the screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion;

[0107] The sampling angle of view of the first camera 210 is different from the sampling angle of view of the second camera 220, so that a first angle difference is formed between the angles of view of the first initial image and the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference;

[0108] The processing of the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; and combining the second part obtained by fusion with the first part to obtain the first display image; and / or

[0109] The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution as the first part, using the second screen content in the second initial image as the second part, and combining the first part obtained by the resolution-upgrading processing with the second part to obtain the second display image.

[0110] It should be noted that, in this embodiment, the two solutions connected by "and / or" can be executed simultaneously, or only one of them can be executed, and the other can be replaced by the combination described in the above embodiment.

[0111] In some embodiments of the present application, the processing of the first initial image based on the gaze position to obtain the first display image includes: performing resolution reduction processing on the second screen content in the first initial image to obtain the second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second portion; using the first screen content in the first initial image as the first portion; and combining the second portion obtained through fusion with the first portion to obtain the first display image. The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution reduction processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first portion; using the second screen content in the second initial image as the second portion, and combining the first portion obtained through fusion with the second portion to obtain the second display image.

[0112] In some embodiments of the present application, the processing of the first initial image based on the gaze position to obtain the first display image includes: performing resolution reduction processing on the second screen content in the first initial image to obtain the second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; and combining the second part obtained by fusion with the first part to obtain the first display image. The processing of the second initial image based on the gaze position to obtain the second display image includes: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; using the second screen content in the second initial image as the second part, or fusing the second screen content in the first initial image with the second screen content in the second initial image to obtain the second part, and combining the first part with the second part to obtain the second display image.

[0113] In some embodiments of the present application, processing the first initial image based on the gaze position to obtain the first display image includes: performing resolution reduction processing on the second screen content in the first initial image to obtain the second portion, using the first screen content in the first initial image as the first portion, and combining the second portion obtained through the resolution reduction processing with the first portion to obtain the first display image. Processing the second initial image based on the gaze position to obtain the second display image includes: performing resolution reduction processing on the first screen content in the second initial image to obtain the first screen content with the increased resolution as the first portion, using the second screen content in the second initial image as the second portion, and combining the first portion obtained through the resolution reduction processing with the second portion to obtain the second display image.

[0114] In some embodiments of the present application, the processing of the first initial image based on the gaze position to obtain the first display image includes: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first portion; using the second screen content in the second initial image as the second portion, or fusing the second screen content in the first initial image with the second screen content in the second initial image as the second portion; and combining the first portion with the second portion to obtain the first display image. The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with the increased resolution as the first portion, using the second screen content in the second initial image as the second portion, and combining the first portion obtained through the resolution-upgrading processing with the second portion to obtain the second display image.

[0115] In some embodiments of the present application, the processing of the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on the second screen content in the first initial image to obtain the second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; combining the second part obtained by fusion with the first part to obtain the first display image; the processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution increase processing on the first screen content in the second initial image to obtain the first screen content with increased resolution as the first part, using the second screen content in the second initial image as the second part, and combining the first part obtained by the resolution increase processing with the second part to obtain the second display image.

[0116] In this embodiment, the second screen content in the first initial image is processed by downscaling to obtain the second screen content with reduced resolution, and the second screen content with reduced resolution is subjected to image fusion processing with the second screen content in the second initial image to obtain the second part; the first screen content in the first initial image is used as the first part; the second part obtained by fusion is combined with the first part to obtain the first display image, and the first screen content in the second initial image is processed by upscaling to obtain the first screen content with increased resolution as the first part, and the second screen content in the second initial image is used as the second part, and the first part obtained by the upscaling processing is combined with the second part to obtain the second display image. In this way, the generation process of the first display image and the second display image can be realized.

[0117] Optionally, the resolution reduction processing is obtained by sampling the picture content of the initial image; and / or

[0118] The resolution increase processing is obtained by performing interpolation processing on the picture content of the initial image; and / or

[0119] The image fusion processing includes: adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference, and performing image fusion with the other of the first initial image and the second initial image.

[0120] The above-mentioned obtaining by sampling the picture content of the initial image may refer to obtaining by downsampling the picture content of the initial image. For example, sampling may be performed on every other point of the picture content of the initial image.

[0121] The obtaining by performing interpolation processing on the picture content of the initial image may refer to obtaining by performing bilinear interpolation processing on the picture content of the initial image.

[0122] Adjusting the viewing angle of one of the first initial image and the second initial image based on the first angular difference may refer to adjusting the first initial image and the second initial image to the same viewing angle based on the first angular difference. Specifically, the viewing angle of the first initial image may be adjusted, or the viewing angle of the second initial image may be adjusted.

[0123] In some embodiments of the present application, adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference and fusing the image with the other of the first initial image and the second initial image may refer to: adjusting the viewing angle of the first initial image according to the first angle difference, and fusing the first initial image after the adjusted viewing angle with the second initial image.

[0124] In some embodiments of the present application, adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference, and fusing the image with the other of the first initial image and the second initial image may refer to: adjusting the viewing angle of the second initial image according to the first angle difference, and fusing the second initial image after the adjusted viewing angle with the first initial image.

[0125] In this embodiment, the image resolution can be reduced by sampling. Correspondingly, the image resolution can be increased by interpolation. By adjusting the perspectives of the fused images to the same perspective before fusion, the image fusion effect can be improved.

[0126] Optionally, performing image fusion processing on the first picture content after the resolution is increased and the first picture content in the first initial image to obtain the first part includes:

[0127] performing feature extraction on first picture content in the first initial image to obtain first feature data, and performing feature extraction on the first picture content after the resolution is increased to obtain second feature data;

[0128] Taking the product of the first feature data and a disparity matrix to obtain third feature data, wherein the disparity matrix is ​​used to convert the viewing angle of the picture content corresponding to the first feature data to be the same as the viewing angle of the picture content corresponding to the second feature data;

[0129] The second feature data and the third feature data are fused to obtain the first part.

[0130] In some embodiments of the present application, the first camera 210 is a high-resolution 4K camera, and the second camera 220 is a low-resolution 2K camera. There is a certain parallax between the two cameras, which can be expressed by the external parameter T∈R 4×4 Since the resolution of the gaze area of ​​the second initial image captured by the second camera 220 is relatively low, the clarity can be improved through algorithm processing.

[0131] Gaze point coordinates P(x, y)∈R 2, the gaze area accounts for 0.5, then the gaze area R1 of the first camera 210 can be obtained, and the gaze area R2 of the second camera 220 can be obtained according to the external reference T and the area of ​​R1 = σTR1, where σ = 0.5, σ represents the ratio of the image resolution of the second initial image to the image resolution of the first initial image, where σ is used to adjust the resolutions of R1 and R2 to the same resolution, and T is used to convert the perspectives of R1 and R2 to the same perspective.

[0132] Since the resolution of R2 is low, the clarity is poor. The clarity of R2 needs to be improved. In this embodiment of the application, an artificial intelligence (AI) network is used to learn a fused image with higher clarity for display. Specifically, a deep learning network R'2 = f(R1, R2) is constructed, and the image R2 is enhanced by R1 and R2 to obtain a clearer fused image R'2. Figure 4 , the generation process of the fused image includes the following steps:

[0133] The first feature data is obtained by performing convolution processing on the gaze area R1 of the first initial image, wherein the gaze area R1 of the first initial image is the first picture content in the first initial image.

[0134] Then, the first feature data is mapped based on the above-mentioned extrinsic parameter T so that the perspective of the first feature data is consistent with the perspective of the feature data of the second initial image, wherein the extrinsic parameter serves as the above-mentioned disparity matrix, and the mapping process is: taking the product of the first feature data and the disparity matrix to obtain the third feature data.

[0135] A bilinear interpolation process is performed on the gaze area R2 of the second initial image to obtain R2 with increased resolution, wherein the resolution of R2 with increased resolution is equal to the resolution of R1.

[0136] Then, convolution processing is performed on R2 with increased resolution to obtain the third feature data.

[0137] Finally, the second feature data and the third feature data are fused to obtain the first part R′2.

[0138] In this embodiment, first feature data is obtained by performing feature extraction on the first picture content in the first initial image, and second feature data is obtained by performing feature extraction on the first picture content after the resolution is increased; third feature data is obtained by multiplying the first feature data by a disparity matrix, wherein the disparity matrix is ​​used to convert the perspective of the picture content corresponding to the first feature data to the same perspective as the picture content corresponding to the second feature data; the second feature data and the third feature data are fused to obtain the first part, thereby realizing the generation process of the first part in the above-mentioned second display image.

[0139] Optionally, the eye movement detection component 110 includes a first eye movement detector 111 and a second eye movement detector 112, wherein the first eye movement detector 111 obtains a first gaze position of a user's left eye, and the second eye movement detector 112 obtains a second gaze position of a user's right eye, wherein the gaze positions include the first gaze position and the second gaze position;

[0140] The processing of the initial image based on the gaze position to obtain a display image further includes: processing the first initial image based on the first gaze position to obtain a first display image; processing the second initial image based on the second gaze position to obtain a second display image;

[0141] The first part corresponds to both the first gaze position and the second gaze position, so that when the user views the displayed image, the user's first gaze position falls within the range of the first part of the first displayed image, and the user's second gaze position falls within the range of the first part of the second displayed image.

[0142] The first eye movement detector 111 and the second eye movement detector 112 may be various detection devices in the head mounted display device 300 that can detect user eye movement input, for example, various eye movement cameras or eye movement sensors. Figure 2 In some embodiments of the present application, the first eye movement detector 111 is Figure 2 The eye movement camera is located on the left side, and the second eye movement detector 112 is Figure 2 The eye tracking camera is located on the right side.

[0143] It should be noted that the specific implementation process of processing the first initial image based on the first gaze position to obtain the first display image is similar to the specific implementation process of processing the first initial image based on the gaze position to obtain the first display image in the above embodiment. The only difference is that the gaze position in the above embodiment is replaced by the first gaze position. To avoid repetition, it will not be repeated here.

[0144] Correspondingly, the specific implementation process of processing the second initial image based on the second gaze position to obtain the second display image is similar to the specific implementation process of processing the second initial image based on the gaze position to obtain the second display image in the above embodiment. The only difference is that the gaze position in the above embodiment is replaced by the second gaze position. To avoid repetition, it will not be repeated here.

[0145] The first portion corresponding to both the first gaze position and the second gaze position may mean that the first portion in the first display image corresponds to the first gaze position, and the first portion in the second display image corresponds to the second gaze position.

[0146] In this embodiment, the eye movement detection component 110 includes a first eye movement detector 111 and a second eye movement detector 112, the first eye movement detector 111 obtains a first gaze position of the user's left eye, and the second eye movement detector 112 obtains a second gaze position of the user's right eye, and the gaze position includes the first gaze position and the second gaze position; the initial image is processed based on the gaze position to obtain a display image, and further includes: processing the first initial image based on the first gaze position to obtain a first display image; processing the second initial image based on the second gaze position to obtain a second display image. In this way, the degree of matching between the generated display image and the user's gaze area can be further improved, thereby further improving the user's visual experience in the process of viewing the display image.

[0147] Optionally, the head-mounted display device 300 includes a main body 100 and the image acquisition device 200 that can be separated from the main body 100, the main body 100 includes the eye movement detection component 110 and the display component 120; after the image acquisition device 200 processes the initial image to obtain the display image, the image acquisition device 200 sends the display image to the display component 120 based on the wireless communication connection between the image acquisition device 200 and the display component 120, so that the display component 120 displays the display image.

[0148] In some embodiments of the present application, when the above-mentioned image acquisition device 200 is an image acquisition device 200 separated from the main part 100 of the head-mounted display device 300, the above-mentioned image acquisition device 200 can be a shooting device in a live broadcast scene. Accordingly, the scene in which the user watches the image captured by the image acquisition device 200 based on the head-mounted display device 300 is: the scene in which the user watches the live broadcast picture based on the head-mounted display device 300.

[0149] In some embodiments of the present application, when the above-mentioned image acquisition device 200 is an image acquisition device 200 separated from the main part 100 of the head-mounted display device 300, the above-mentioned image acquisition device 200 may also be a shooting device in a drone. Accordingly, the scene in which the user watches the image captured by the image acquisition device 200 based on the head-mounted display device 300 is: the scene in which the user watches the real-time picture captured by the shooting device in the drone based on the head-mounted display device 300 in real time.

[0150] In this embodiment, since only some areas of the displayed image have a high resolution and other areas have a low resolution, it is beneficial to reduce the size of the displayed image, so as to reduce the network bandwidth required for the image acquisition device 200 to send the displayed image to the display component 120 based on the wireless communication connection between the image acquisition device 200 and the display component 120.

[0151] See Figure 5 , Figure 5 The present invention provides a flowchart of a data transmission method based on the head-mounted display device 300 in the above embodiment, which includes the following steps:

[0152] Step 1: See Figure 2 , the eye movement detection component in the XR head display is used to detect the user's gaze area, wherein the eye movement detection component 110 is an eye movement camera.

[0153] Step 2: Transmit the gaze point area detected by the eye movement detection component 110 to the image acquisition device 200 via the network, wherein the image acquisition device 200 may be a live camera.

[0154] Step 3: See Figure 3 , the first camera 210 and the second camera 220 in the image acquisition device 200 respectively shoot the live scene to obtain a high-resolution image and a low-resolution image. Figure 3 The 4K resolution image shown in can be regarded as a high-resolution image, that is, the first initial image as mentioned above. Figure 3 The 2K resolution image shown in can be regarded as a low-resolution image, that is, the second initial image as described above. The image acquisition device 200 synthesizes the left screen image data and the right screen image data according to the gaze position transmitted by the head-mounted display device 300 through the network, wherein the above-mentioned display image includes the left screen image and the right screen image. Among them, the left screen image corresponding to the left screen image data ( Figure 3 ) is the first display image, and the right screen image data corresponds to the right screen image ( Figure 3) is the second display image. The foveation area of ​​the left screen image is the first part of the first display image, and the non-foveation area of ​​the left screen image is the second part of the first display image. The foveation area of ​​the right screen image is the first part of the second display image, and the non-foveation area of ​​the right screen image is the second part of the second display image. The foveation areas of the left and right screen images are both cropped from the high-resolution image, and the non-foveation areas of the left and right screen images are both cropped from the low-resolution image. The foveation areas and the non-foveation areas are then synthesized to obtain the left and right screen images.

[0155] Step 4: The left screen image data corresponding to the first display image and the right screen image data corresponding to the second display image are compressed, such as in the h264 compression format, and then the compressed image data are transmitted to the head-mounted display device 300 via the network.

[0156] Step 5: After the head-mounted display device 300 receives the display image sent by the image acquisition device 200, it decompresses it to obtain left-screen image data and right-screen image data, and then displays the left-screen image data and the right-screen image data in the form of a first display image and a second display image on the first display device 121 and the second display device 122 of the image acquisition device 200, respectively.

[0157] The data transmission method provided in this embodiment can save the amount of image data transmitted from the image acquisition device 200 to the head-mounted display device 300, improve the fluency and clarity of the live broadcast, and at the same time, the live video screen has a 3D stereo effect.

[0158] See Figure 6 , an embodiment of the present application provides a display system, including a head-mounted display device 300 and an image acquisition device 200 that are separately arranged, wherein the head-mounted display device 300 and the image acquisition device 200 are wirelessly connected to each other, and the head-mounted display device 300 includes an eye movement detection component 110 and a display component 120;

[0159] The head mounted display device 300 is used to obtain the user's gaze position based on the eye movement detection component 110;

[0160] The head mounted display device 300 is further configured to send the gaze position to the image acquisition device 200;

[0161] The image acquisition device 200 is used to acquire an initial image;

[0162] The image acquisition device 200 is further configured to process the initial image based on the gaze position to obtain a display image;

[0163] The image acquisition device 200 is further configured to send the display image to the head mounted display device 300;

[0164] The head-mounted display device 300 is also used to display the display image based on the display component 120, and the display image includes a first part and a second part, and the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

[0165] The difference between this embodiment and the above embodiment is that the image acquisition device 200 is an independent device independent of the head-mounted display device 300, that is, the head-mounted display device 300 in this embodiment does not include the image acquisition device 200 in the above embodiment, but is connected to the head-mounted display device 300 through wireless communication via an independent image acquisition device 200.

[0166] It should be noted that the image acquisition device 200 of this embodiment can implement each process of the image acquisition device 200 of the above-mentioned embodiment and has the same beneficial effects. To avoid repetition, they are not described here in detail. Correspondingly, the head-mounted display device 300 of this embodiment can implement each process of the main body 100 of the head-mounted display device 300 of the above-mentioned embodiment and has the same beneficial effects. To avoid repetition, they are not described here in detail.

[0167] Optionally, the image acquisition device 200 includes a first camera 210 and a second camera 220; the initial image includes a first initial image acquired by the first camera 210 and a second initial image acquired by the second camera 220;

[0168] The first initial image has a first resolution, and the second initial image has a second resolution, wherein the first resolution is higher than the second resolution; wherein the first initial image includes the first portion, and the second initial image includes the second portion;

[0169] The processing of the initial image based on the gaze position to obtain the display image includes: combining the first portion corresponding to the gaze position in the first initial image with the second portion in the second initial image to obtain the display image.

[0170] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0171] Optionally, the image acquisition device 200 includes a first camera 210 and a second camera 220, and the initial image includes a first initial image acquired by the first camera 210 and a second initial image acquired by the second camera 220; the first initial image has a first resolution, and the second initial image has a second resolution, and the first resolution is higher than the second resolution;

[0172] Processing the initial image based on the gaze position to obtain a display image includes: processing the first initial image based on the gaze position to obtain a first display image; processing the second initial image based on the gaze position to obtain a second display image; the display image includes the first display image and the second display image; wherein the first display image and the second display image both include the first part and the second part, the resolution of the first part in the first display image is higher than the resolution of the second part in the first display image, and the resolution of the first part in the second display image is higher than the resolution of the second part in the second display image;

[0173] The display assembly 120 includes a first display device 121 and a second display device 122 . The first display device 121 displays the first display image, and the second display device 122 displays the second display image.

[0174] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0175] Optionally, the screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion;

[0176] The sampling angle of view of the first camera 210 is different from the sampling angle of view of the second camera 220, so that a first angle difference is formed between the angles of view of the first initial image and the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference;

[0177] The processing the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained by the resolution reduction processing with the first part to obtain the first display image; and / or

[0178] The processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image.

[0179] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0180] Optionally, the screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion;

[0181] The sampling angle of view of the first camera 210 is different from the sampling angle of view of the second camera 220, so that a first angle difference is formed between the angles of view of the first initial image and the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference;

[0182] The processing of the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; and combining the second part obtained by fusion with the first part to obtain the first display image; and / or

[0183] The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution as the first part, using the second screen content in the second initial image as the second part, and combining the first part obtained by the resolution-upgrading processing with the second part to obtain the second display image.

[0184] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0185] Optionally, the resolution reduction processing is obtained by sampling the picture content of the initial image; and / or

[0186] The resolution increase processing is obtained by performing interpolation processing on the picture content of the initial image; and / or

[0187] The image fusion processing includes: adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference, and performing image fusion with the other of the first initial image and the second initial image.

[0188] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0189] The embodiment of the present application further provides an image acquisition device 200, which is wirelessly connected to a head-mounted display device 300 that is separately provided, and the image acquisition device 200 receives the user's gaze position sent by the head-mounted display device 300;

[0190] The image acquisition device 200 acquires an initial image;

[0191] The image acquisition device 200 processes the initial image based on the gaze position to obtain a display image;

[0192] The image acquisition device 200 sends the display image to the head mounted display device 300;

[0193] The displayed image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that the gaze position falls within the range of the first part.

[0194] The image acquisition device 200 provided in this embodiment can implement each process of the image acquisition device 200 in the above embodiment and has the same beneficial effects. To avoid repetition, they will not be described again here.

[0195] Optionally, the image acquisition device 200 includes a first camera 210 and a second camera 220; the initial image includes a first initial image acquired by the first camera 210 and a second initial image acquired by the second camera 220;

[0196] The first initial image has a first resolution, and the second initial image has a second resolution, wherein the first resolution is higher than the second resolution; wherein the first initial image includes the first portion, and the second initial image includes the second portion;

[0197] The processing of the initial image based on the gaze position to obtain the display image includes: combining the first portion corresponding to the gaze position in the first initial image with the second portion in the second initial image to obtain the display image.

[0198] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0199] Optionally, the image acquisition device 200 includes a first camera 210 and a second camera 220; the initial image includes a first initial image acquired by the first camera 210 and a second initial image acquired by the second camera 220;

[0200] The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion;

[0201] The sampling angle of view of the first camera 210 is different from the sampling angle of view of the second camera 220, so that a first angle difference is formed between the angles of view of the first initial image and the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference;

[0202] The processing the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained by the resolution reduction processing with the first part to obtain the first display image; and / or

[0203] The processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image.

[0204] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0205] Optionally, the image acquisition device 200 includes a first camera 210 and a second camera 220; the initial image includes a first initial image acquired by the first camera 210 and a second initial image acquired by the second camera 220;

[0206] The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion;

[0207] The sampling angle of view of the first camera 210 is different from the sampling angle of view of the second camera 220, so that a first angle difference is formed between the angles of view of the first initial image and the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference;

[0208] The processing of the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; and combining the second part obtained by fusion with the first part to obtain the first display image; and / or

[0209] The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution as the first part, using the second screen content in the second initial image as the second part, and combining the first part obtained by the resolution-upgrading processing with the second part to obtain the second display image.

[0210] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0211] Optionally, the resolution reduction processing is obtained by sampling the picture content of the initial image; and / or

[0212] The resolution increase processing is obtained by performing interpolation processing on the picture content of the initial image; and / or

[0213] The image fusion processing includes: adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference, and performing image fusion with the other of the first initial image and the second initial image.

[0214] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0215] The embodiment of the present application further provides a head-mounted display device 300, which is wirelessly connected to a separately arranged image acquisition device 200, and includes an eye movement detection component 110 and a display component 120;

[0216] The head mounted display device 300 obtains the user's gaze position based on the eye movement detection component 110;

[0217] The head mounted display device 300 sends the gaze position to the image acquisition device 200;

[0218] The head mounted display device 300 receives the display image sent by the image acquisition device 200;

[0219] The head-mounted display device 300 displays the display image based on the display component 120, and the display image includes a first part and a second part, and the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

[0220] The head-mounted display device 300 provided in this embodiment can implement various processes of the main body part 100 in the head-mounted display device 300 in the above embodiment and has the same beneficial effects. To avoid repetition, they will not be described here.

[0221] Optionally, the eye movement detection component 110 includes a first eye movement detector 111 and a second eye movement detector 112, wherein the first eye movement detector 111 obtains a first gaze position of a user's left eye, and the second eye movement detector 112 obtains a second gaze position of a user's right eye, wherein the gaze positions include the first gaze position and the second gaze position;

[0222] The display image includes a first display image and a second display image, wherein the first display image and the second display image each include the first portion and the second portion, a resolution of the first portion in the first display image is higher than a resolution of the second portion in the first display image, and a resolution of the first portion in the second display image is higher than a resolution of the second portion in the second display image;

[0223] The first part corresponds to both the first gaze position and the second gaze position, so that when the user views the displayed image, the user's first gaze position falls within the range of the first part of the first displayed image, and the user's second gaze position falls within the range of the first part of the second displayed image.

[0224] The specific implementation process of this embodiment is similar to the implementation process in the above embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0225] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0226] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0227] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A head-mounted display device, characterized in that: Including eye movement detection components, image acquisition equipment and display components; The eye movement detection component obtains the user's gaze position; The image acquisition device acquires an initial image; The image acquisition device processes the initial image based on the gaze position to obtain a display image; The display component displays the display image, the display image including a first portion and a second portion, the resolution of the first portion being higher than the resolution of the second portion; The first portion corresponds to the gaze position, so that when the user views the displayed image, the gaze position of the user falls within the range of the first portion.

2. The head-mounted display device according to claim 1, wherein: The image acquisition device includes a first camera and a second camera; the initial image includes a first initial image acquired by the first camera and a second initial image acquired by the second camera; The first initial image has a first resolution, and the second initial image has a second resolution, wherein the first resolution is higher than the second resolution; wherein the first initial image includes the first portion, and the second initial image includes the second portion; The processing of the initial image based on the gaze position to obtain the display image includes: combining the first portion corresponding to the gaze position in the first initial image with the second portion in the second initial image to obtain the display image.

3. The head-mounted display device according to claim 1, wherein: The image acquisition device includes a first camera and a second camera, and the initial image includes a first initial image acquired by the first camera and a second initial image acquired by the second camera; The first initial image has a first resolution, the second initial image has a second resolution, and the first resolution is higher than the second resolution; Processing the initial image based on the gaze position to obtain a display image includes: processing the first initial image based on the gaze position to obtain a first display image; processing the second initial image based on the gaze position to obtain a second display image; the display image includes the first display image and the second display image; wherein the first display image and the second display image both include the first part and the second part, the resolution of the first part in the first display image is higher than the resolution of the second part in the first display image, and the resolution of the first part in the second display image is higher than the resolution of the second part in the second display image; The display assembly includes a first display device and a second display device, the first display device displays the first display image, and the second display device displays the second display image.

4. The head-mounted display device according to claim 3, wherein: The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion; The sampling angle of view of the first camera is different from the sampling angle of view of the second camera, so that a first angle difference is formed between the angle of view of the first initial image and the angle of view of the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference; The processing of the first initial image based on the gaze position to obtain a first display image includes: fusing first screen content in the first initial image with first screen content in the second initial image to obtain the first portion; using second screen content in the second initial image as the second portion, or fusing the second screen content in the first initial image with second screen content in the second initial image to obtain the second portion; and combining the first portion with the second portion to obtain the first display image; The processing of the second initial image based on the gaze position to obtain the second display image includes: fusing the first screen content in the first initial image with the first screen content in the second initial image to obtain the first part; using the second screen content in the second initial image as the second part, or fusing the second screen content in the first initial image with the second screen content in the second initial image as the second part, and combining the first part and the second part to obtain the second display image.

5. The head mounted display device according to claim 3, wherein: The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion; The sampling angle of view of the first camera is different from the sampling angle of view of the second camera, so that a first angle difference is formed between the angle of view of the first initial image and the angle of view of the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference; The processing the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained by the resolution reduction processing with the first part to obtain the first display image; and / or The processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image.

6. The head mounted display device according to claim 3, wherein: The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion; The sampling angle of view of the first camera is different from the sampling angle of view of the second camera, so that a first angle difference is formed between the angle of view of the first initial image and the angle of view of the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference; The processing of the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; and combining the second part obtained by fusion with the first part to obtain the first display image; and / or The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution as the first part, using the second screen content in the second initial image as the second part, and combining the first part obtained by the resolution-upgrading processing with the second part to obtain the second display image.

7. The head mounted display device according to claim 5 or 6, characterized in that: The resolution reduction processing is obtained by sampling the picture content of the initial image; and / or The resolution increase processing is obtained by performing interpolation processing on the picture content of the initial image; and / or The image fusion processing includes: adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference, and performing image fusion with the other of the first initial image and the second initial image.

8. The head mounted display device according to claim 5, wherein: The performing image fusion processing on the first picture content after the resolution is increased and the first picture content in the first initial image to obtain the first part includes: performing feature extraction on first picture content in the first initial image to obtain first feature data, and performing feature extraction on the first picture content after the resolution is increased to obtain second feature data; Taking the product of the first feature data and a disparity matrix to obtain third feature data, wherein the disparity matrix is ​​used to convert the viewing angle of the picture content corresponding to the first feature data to be the same as the viewing angle of the picture content corresponding to the second feature data; The second feature data and the third feature data are fused to obtain the first part.

9. The head mounted display device according to claim 3, wherein: The eye movement detection component includes a first eye movement detector and a second eye movement detector, the first eye movement detector obtains a first gaze position of the user's left eye, and the second eye movement detector obtains a second gaze position of the user's right eye, the gaze positions including the first gaze position and the second gaze position; The processing of the initial image based on the gaze position to obtain a display image further includes: processing the first initial image based on the first gaze position to obtain a first display image; processing the second initial image based on the second gaze position to obtain a second display image; The first part corresponds to both the first gaze position and the second gaze position, so that when the user views the displayed image, the user's first gaze position falls within the range of the first part of the first displayed image, and the user's second gaze position falls within the range of the first part of the second displayed image.

10. The head mounted display device according to claim 1, wherein: The head-mounted display device includes a main body and an image acquisition device that can be separated from the main body, the main body including the eye movement detection component and the display component; after the image acquisition device processes the initial image to obtain the display image, the image acquisition device sends the display image to the display component based on the wireless communication connection between the image acquisition device and the display component, so that the display component displays the display image.

11. A display system, characterized in that: The head-mounted display device comprises a head-mounted display device and an image acquisition device which are separately arranged, wherein the head-mounted display device and the image acquisition device are wirelessly connected to each other, and the head-mounted display device comprises an eye movement detection component and a display component; The head mounted display device is used to obtain the user's gaze position based on the eye movement detection component; The head mounted display device is further configured to send the gaze position to the image acquisition device; The image acquisition device is used to acquire an initial image; The image acquisition device is further configured to process the initial image based on the gaze position to obtain a display image; The image acquisition device is further configured to send the display image to the head-mounted display device; The head-mounted display device is also used to display the display image based on the display component, and the display image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

12. The display system according to claim 11, wherein: The image acquisition device includes a first camera and a second camera; the initial image includes a first initial image acquired by the first camera and a second initial image acquired by the second camera; The first initial image has a first resolution, and the second initial image has a second resolution, wherein the first resolution is higher than the second resolution; wherein the first initial image includes the first portion, and the second initial image includes the second portion; The processing of the initial image based on the gaze position to obtain the display image includes: combining the first portion corresponding to the gaze position in the first initial image with the second portion in the second initial image to obtain the display image.

13. The display system according to claim 11, wherein: The image acquisition device includes a first camera and a second camera, and the initial image includes a first initial image acquired by the first camera and a second initial image acquired by the second camera; The first initial image has a first resolution, the second initial image has a second resolution, and the first resolution is higher than the second resolution; Processing the initial image based on the gaze position to obtain a display image includes: processing the first initial image based on the gaze position to obtain a first display image; processing the second initial image based on the gaze position to obtain a second display image; the display image includes the first display image and the second display image; wherein the first display image and the second display image both include the first part and the second part, the resolution of the first part in the first display image is higher than the resolution of the second part in the first display image, and the resolution of the first part in the second display image is higher than the resolution of the second part in the second display image; The display assembly includes a first display device and a second display device, the first display device displays the first display image, and the second display device displays the second display image.

14. The display system according to claim 13, wherein: The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion; The sampling angle of view of the first camera is different from the sampling angle of view of the second camera, so that a first angle difference is formed between the angle of view of the first initial image and the angle of view of the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference; The processing the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained by the resolution reduction processing with the first part to obtain the first display image; and / or The processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image.

15. The display system according to claim 13, wherein: The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion; The sampling angle of view of the first camera is different from the sampling angle of view of the second camera, so that a first angle difference is formed between the angle of view of the first initial image and the angle of view of the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference; The processing of the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; and combining the second part obtained by fusion with the first part to obtain the first display image; and / or The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution as the first part, using the second screen content in the second initial image as the second part, and combining the first part obtained by the resolution-upgrading processing with the second part to obtain the second display image.

16. The display system according to claim 14 or 15, characterized in that: The resolution reduction processing is obtained by sampling the picture content of the initial image; and / or The resolution increase processing is obtained by performing interpolation processing on the picture content of the initial image; and / or The image fusion processing includes: adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference, and performing image fusion with the other of the first initial image and the second initial image.

17. An image acquisition device wirelessly connected to a separately arranged head-mounted display device, characterized in that: The image acquisition device receives the gaze position of the user sent by the head mounted display device; The image acquisition device acquires an initial image; The image acquisition device processes the initial image based on the gaze position to obtain a display image; The image acquisition device sends the display image to the head-mounted display device; The displayed image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that the gaze position falls within the range of the first part.

18. The image acquisition device according to claim 17, characterized in that: The image acquisition device includes a first camera and a second camera; the initial image includes a first initial image acquired by the first camera and a second initial image acquired by the second camera; The first initial image has a first resolution, and the second initial image has a second resolution, wherein the first resolution is higher than the second resolution; wherein the first initial image includes the first portion, and the second initial image includes the second portion; The processing of the initial image based on the gaze position to obtain the display image includes: combining the first portion corresponding to the gaze position in the first initial image with the second portion in the second initial image to obtain the display image.

19. The image acquisition device according to claim 17, wherein: The image acquisition device includes a first camera and a second camera; the initial image includes a first initial image acquired by the first camera and a second initial image acquired by the second camera; The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion; The sampling angle of view of the first camera is different from the sampling angle of view of the second camera, so that a first angle difference is formed between the angle of view of the first initial image and the angle of view of the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference; The processing the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain the second part, using the first screen content in the first initial image as the first part, and combining the second part obtained by the resolution reduction processing with the first part to obtain the first display image; and / or The processing of the second initial image based on the gaze position to obtain a second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution, performing image fusion processing on the first screen content with increased resolution and the first screen content in the first initial image to obtain the first part; using the second screen content in the second initial image as the second part, and combining the first part obtained by fusion with the second part to obtain the second display image.

20. The image acquisition device according to claim 17, wherein: The image acquisition device includes a first camera and a second camera; the initial image includes a first initial image acquired by the first camera and a second initial image acquired by the second camera; The screen content of the first initial image and the screen content of the second initial image both include first screen content corresponding to the first portion and second screen content corresponding to the second portion; The sampling angle of view of the first camera is different from the sampling angle of view of the second camera, so that a first angle difference is formed between the angle of view of the first initial image and the angle of view of the second initial image, that is, the first picture content in the first initial image is the same as the first picture content in the second initial image, and the angle of view of the first picture content in the first initial image and the angle of view of the first picture content in the second initial image have the first angle difference; The processing of the first initial image based on the gaze position to obtain a first display image includes: performing resolution reduction processing on second screen content in the first initial image to obtain second screen content with reduced resolution, performing image fusion processing on the second screen content with reduced resolution and the second screen content in the second initial image to obtain the second part; using the first screen content in the first initial image as the first part; and combining the second part obtained by fusion with the first part to obtain the first display image; and / or The processing of the second initial image based on the gaze position to obtain the second display image includes: performing resolution-upgrading processing on the first screen content in the second initial image to obtain the first screen content with increased resolution as the first part, using the second screen content in the second initial image as the second part, and combining the first part obtained by the resolution-upgrading processing with the second part to obtain the second display image.

21. The image acquisition device according to claim 19 or 20, characterized in that: The resolution reduction processing is obtained by sampling the picture content of the initial image; and / or The resolution increase processing is obtained by performing interpolation processing on the picture content of the initial image; and / or The image fusion processing includes: adjusting the viewing angle of one of the first initial image and the second initial image according to the first angle difference, and performing image fusion with the other of the first initial image and the second initial image.

22. A head-mounted display device wirelessly connected to a separately arranged image acquisition device, characterized in that: The head-mounted display device includes an eye movement detection component and a display component; The head mounted display device obtains the user's gaze position based on the eye movement detection component; The head mounted display device sends the gaze position to the image acquisition device; The head mounted display device receives the display image sent by the image acquisition device; The head-mounted display device displays the display image based on the display component, and the display image includes a first part and a second part, the resolution of the first part is higher than the resolution of the second part; the first part corresponds to the gaze position, so that when the user views the display image, the user's gaze position falls within the range of the first part.

23. The head mounted display device according to claim 22, wherein: The eye movement detection component includes a first eye movement detector and a second eye movement detector, the first eye movement detector obtains a first gaze position of the user's left eye, and the second eye movement detector obtains a second gaze position of the user's right eye, the gaze positions including the first gaze position and the second gaze position; The display image includes a first display image and a second display image, wherein the first display image and the second display image each include the first portion and the second portion, a resolution of the first portion in the first display image is higher than a resolution of the second portion in the first display image, and a resolution of the first portion in the second display image is higher than a resolution of the second portion in the second display image; The first part corresponds to both the first gaze position and the second gaze position, so that when the user views the displayed image, the user's first gaze position falls within the range of the first part of the first displayed image, and the user's second gaze position falls within the range of the first part of the second displayed image.

Images