Microdisplay system and head-mounted device

Abstract

This application provides a micro-display system and a head-mounted display device, relating to the field of micro-display technology. The micro-display system includes an image acquisition module, an image processing module, a display module, and a configuration module. The image acquisition module acquires the current spatial position information of the head-mounted display device worn by the user in real time. Using configured first image parameters, it can achieve partitioned acquisition of different areas within the same image frame, improving processing efficiency and accuracy, thereby reducing image link latency. The image processing module uses the current spatial position information and second image parameters to dynamically adjust the image update method and, based on a target image processing strategy and current image data, quickly generates an image of the current real-world scene, reducing image link latency and achieving low-latency end-to-end image link display. This improves the user experience when using an MR head-mounted display device, addressing issues such as dizziness and motion mismatch caused by latency, and enhances immersion and real-time interaction.

Description

Technical Field

[0001] This invention relates to the field of display technology, and more specifically, to a micro-display system and head-mounted display device. Background Technology

[0002] Mixed Reality (MR) is an immersive technology that blends the virtual world with the real world. It combines the advantages of Virtual Reality (VR) and Augmented Reality (AR), enabling virtual objects and real environments to coexist and interact in the same space and have spatial awareness capabilities. It aims to provide users with a realistic virtual world with multiple senses.

[0003] Currently, most MR headsets employ Video See-Through (VST) display technology. Specifically, the camera in the MR headset captures a video stream of the real-world scene within the user's field of vision, and then combines the video stream with pre-generated virtual images to create an image displayed on the MR headset screen, thus providing the user with an immersive mixed reality experience.

[0004] However, when combining video streams with virtual images, there is an unavoidable delay in the image link. If the delay in the image link exceeds a certain duration, it can lead to a poor user experience, such as dizziness, mismatched movements, and poor immersion. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing a micro-display system and head-mounted display device, thereby resolving the technical problems existing in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, embodiments of this application provide a micro-display system, which includes: an image acquisition module, an image processing module, a display module, and a configuration module;

[0008] The output of the image acquisition module is connected to the input of the image processing module, and the output of the image processing module is communicatively connected to the input of the display module.

[0009] The configuration module is communicatively connected to both the image acquisition module and the image processing module.

[0010] The configuration module is used to receive a first image parameter and a second image parameter input by the user, transmit the first image parameter to the image acquisition module, and transmit the second image parameter to the image processing module;

[0011] The image acquisition module is used to acquire the current spatial position information of the head-mounted display device worn by the user, and to acquire current image data according to the first image parameters. The current spatial position information and the current image data are transmitted to the image processing module. The current image data includes: center image and edge image. The current spatial position information includes: current azimuth angle and / or current pitch angle.

[0012] The image processing module is used to obtain a target image processing strategy based on the current spatial location information and the second image parameters, generate a current real-world scene image based on the target image processing strategy and the current image data, and transmit the current real-world scene image to the display module.

[0013] The display module is used to overlay the current real-world scene image with a pre-generated virtual image.

[0014] Optionally, the image acquisition module includes: an angle acquisition unit and a shooting unit;

[0015] The angle acquisition unit is used to acquire the current spatial position information of the head-mounted display device;

[0016] The shooting unit is used to partition the image acquisition area in the shooting unit according to the first image parameters to obtain the central area of ​​the field of view and the edge area of ​​the field of view, and to acquire the central image located in the central area of ​​the field of view and the edge image located in the edge area of ​​the field of view according to a preset image acquisition mode.

[0017] Optionally, the step of acquiring a central image located in the center region of the field of view and an edge image located in the edge region of the field of view according to a preset image acquisition mode includes:

[0018] The central image is captured using the global shutter mode;

[0019] The edge image is captured using the rolling shutter mode.

[0020] Optionally, obtaining the target image processing strategy based on the current spatial location information and the second image parameters includes:

[0021] Based on the current spatial location information and the second image parameters, the current swing amplitude of the head-mounted display device is determined;

[0022] Based on the current swing amplitude of the head-mounted display device and the pre-built mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is determined.

[0023] Optionally, with the vertical centerline as a reference, the second image parameter includes: azimuth threshold;

[0024] Determining the current swing amplitude of the head-mounted display device based on the current spatial location information and the second image parameters includes:

[0025] If the absolute value of the current azimuth angle is less than or equal to the azimuth angle threshold, then the current swing amplitude of the head-mounted display device is determined to be the first swing amplitude.

[0026] If the absolute value of the current azimuth angle is greater than the azimuth angle threshold and less than or equal to the preset first maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude, and the second swing amplitude is greater than the first swing amplitude.

[0027] If the absolute value of the current azimuth angle is greater than the maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude, which is greater than the second swing amplitude.

[0028] Optionally, with the horizontal centerline as a reference, the second image parameter includes: a pitch angle threshold;

[0029] Determining the current swing amplitude of the head-mounted display device based on the current spatial location information and the second image parameters includes:

[0030] If the absolute value of the current pitch angle is less than or equal to the pitch angle threshold, then the current swing amplitude of the head-mounted display device is determined to be the first swing amplitude.

[0031] If the absolute value of the current pitch angle is greater than the pitch angle threshold and less than or equal to the preset second maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude, and the second swing amplitude is greater than the first swing amplitude.

[0032] If the absolute value of the current pitch angle is greater than the maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude, which is greater than the second swing amplitude.

[0033] Optionally, determining the target image processing strategy corresponding to the current swing amplitude based on the current swing amplitude of the head-mounted display device and a pre-built mapping relationship between the swing amplitude and the image processing strategy includes:

[0034] If the current swing amplitude of the head-mounted display device is the first swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the first image processing strategy, wherein the first image processing strategy is used to indicate that the center image is not updated.

[0035] If the current swing amplitude of the head-mounted display device is the second swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the second image processing strategy, wherein the second image processing strategy is used to indicate that the previous image data is used to update the current image data;

[0036] If the current swing amplitude of the head-mounted display device is the third swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the third image processing strategy, wherein the third image processing strategy is used to indicate that the current image data collected is used for updating.

[0037] Optionally, generating the current real-world scene image based on the target image processing strategy and the current image data includes:

[0038] If the target image processing strategy is the first image processing strategy, then the current real-world scene image is generated based on the previous image data of the current image data;

[0039] If the target image processing strategy is the second image processing strategy, then the current real-world scene image is generated based on the current spatial location information and the previous image data of the current image data;

[0040] If the target image processing strategy is the third image processing strategy, then the current image data is processed to generate the current real-world scene image.

[0041] Optionally, generating the current real-world scene image based on the current spatial location information and the previous image data of the current image data includes:

[0042] Based on the current spatial position information, the current pixel translation amount is obtained, and the pixel translation amount includes: row pixel translation amount and column pixel translation amount;

[0043] The previous image data is shifted according to the current pixel shift amount to generate the current real-world scene image.

[0044] Optionally, the micro-display system further includes: an interface module; the interface module is connected to the input of the image processing module via the output of the image acquisition module.

[0045] The interface module is used to transmit the current image data acquired by the image acquisition module to the image processing module in a multi-channel parallel reading mode.

[0046] Secondly, embodiments of this application also provide a head-mounted display device, including the micro-display system described in the first aspect above.

[0047] The beneficial effects of this application are:

[0048] This application provides a micro-display system and a head-mounted display device. The micro-display system includes an image acquisition module, an image processing module, a display module, and a configuration module. The image acquisition module acquires the current spatial position information of the head-mounted display device worn by the user in real time, and acquires current image data according to a first image parameter. That is, by using the configured first image parameter, it can realize the partitioned acquisition of different regions in the same image, meet diverse image processing needs, improve processing efficiency and accuracy, and thus reduce image link latency. Then, the image processing module uses the current spatial position information and a second image parameter to determine a target image processing strategy. That is, by using the second image parameter, it can dynamically adjust the image update method, and quickly generate the current real scene image based on the target image processing strategy and the current image data, reducing image processing time, thereby reducing image link latency and improving the immersive experience. Finally, the display module overlays the current real scene image with a pre-generated virtual image to achieve a display effect that combines virtual and reality, enhancing the immersive experience of the user during use. Therefore, in the micro-display system provided in this application, by optimizing the entire image link of image acquisition (i.e., partitioned acquisition) and processing (i.e., dynamically adjusting the image processing strategy), low-latency display is achieved end-to-end in the image link, which significantly improves the user's immersive experience and real-time interaction, thereby improving the user's poor experience caused by latency, such as dizziness and mismatched movements when using MR head-mounted displays, and enhancing immersion and real-time interaction. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the structure of a micro-display system provided in an embodiment of this application;

[0051] Figure 2 A schematic diagram of a center image located in the visual center region and an edge image located in the visual edge region, provided for embodiments of this application;

[0052] Figure 3 This application provides a schematic diagram of the structure of an image acquisition module in a micro-display system.

[0053] Figure 4 A schematic diagram of the horizontal partitioning and calibration angle of the imaging unit in a micro-display system provided in this application embodiment;

[0054] Figure 5 A schematic diagram of the vertical partitioning and calibration angle of the imaging unit in a micro-display system provided in this application embodiment;

[0055] Figure 6 A schematic diagram illustrating the working principle of a micro-display system provided in an embodiment of this application;

[0056] Figure 7 A schematic diagram of the horizontal partitioning of a shooting unit in a micro-display system provided in an embodiment of this application;

[0057] Figure 8 A schematic diagram illustrating the working principle of another micro-display system provided in an embodiment of this application;

[0058] Figure 9 A schematic diagram of a vertical partition of a shooting unit in a micro-display system provided in an embodiment of this application;

[0059] Figure 10 A schematic diagram illustrating the working principle of yet another micro-display system provided in this application embodiment;

[0060] Figure 11 A schematic diagram illustrating the working principle of another micro-display system provided in an embodiment of this application;

[0061] Figure 12 A schematic diagram illustrating the working principle of yet another micro-display system provided in this application embodiment;

[0062] Figure 13 A schematic diagram illustrating the working principle of another micro-display system provided in an embodiment of this application;

[0063] Figure 14 This is a schematic diagram of another micro-display system provided in an embodiment of this application;

[0064] Figure 15 A schematic diagram of the delay of a conventional micro-display system provided in an embodiment of this application;

[0065] Figure 16 This application provides a schematic diagram of the time delay after optimization of a micro-display system.

[0066] Figure 17 This is another delayed schematic diagram of an optimized micro-display system provided in an embodiment of this application;

[0067] Figure 18This is a schematic diagram of the structure of a head-mounted display device provided in an embodiment of this application.

[0068] Icons: 100-Micro display system; 1-Image acquisition module; 2-Image processing module; 3-Display module; 4-Configuration module; 11-Angle acquisition unit; 12-Shooting unit; 5-Interface module; 200-Head-mounted display device. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0070] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0071] First, the background technology involved in this application will be introduced.

[0072] Currently, most MR headsets use VST display technology. Specifically, the camera in the MR headset captures a real-time video stream of the real-world scene within the user's field of vision, and then combines the real-time video stream with pre-generated virtual images to create an image displayed on the MR headset screen, thus providing the user with an immersive mixed reality experience.

[0073] However, latency in the image link is unavoidable during the synthesis of real-time video streams and pre-generated virtual images. This latency causes a mismatch between the user's vision and actions, triggering dizziness and significantly reducing the quality of the immersive experience. To mitigate this problem, MR headsets need to control image latency within a certain range to ensure a good user experience. According to research on human perception, latency below 20 milliseconds is generally considered not to cause significant discomfort or abnormal sensations. However, the image latency of currently available MR headsets generally exceeds 30 milliseconds, making it difficult to achieve a satisfactory mixed reality experience.

[0074] To address the aforementioned issues, this application proposes a low-latency micro-display system, which includes an image acquisition module, an image processing module, a display module, and a configuration module. By performing system-level optimization on the entire image link of image acquisition, transmission, processing, and display, the image link latency is significantly reduced, thereby improving the user experience caused by large image link latency, such as visual dizziness and motion mismatch, and enhancing the immersion and real-time interaction during use.

[0075] Optionally, refer to Figure 1 The diagram shown is a structural schematic of the micro-display system provided in this application. The micro-display system 100 includes: an image acquisition module 1, an image processing module 2, a display module 3, and a configuration module 4.

[0076] Optionally, the microdisplay system provided in this application can support virtual reality (VR), augmented reality (AR), and mixed reality (MR) and is suitable for various scenarios such as cycling navigation, translation, and shopping.

[0077] For example, the image acquisition module 1 may include various sensors (such as cameras, depth sensors, etc.) to capture images, spatial depth, lighting conditions and other data of the real scene in which the user is located in real time, providing a foundation for subsequent virtual and reality fusion.

[0078] Image processing module 2 can be an electronic device with image processing function, and display module 3 can be a display screen that supports different display materials, such as liquid crystal, LED, OLED, micro LED and Q-LED, and supports different display types, including spatial color display and temporal color display.

[0079] Configuration module 4 can be an electronic device with display function and user input receiving function. Users can input different configuration parameters through the configuration module.

[0080] Continue to refer to Figure 1 As shown, the output of the image acquisition module 1 is connected to the input of the image processing module 2, and the output of the image processing module 2 is communicatively connected to the input of the display module 3.

[0081] Configuration module 4 is communicatively connected to image acquisition module 1 and image processing module 2, respectively;

[0082] Configuration module 4 is used to receive first image parameters and second image parameters input by the user, transmit the first image parameters to the image acquisition module, and transmit the second image parameters to the image processing module. That is, the user can input the first image parameters through the configuration module to optimize the acquisition process of the image acquisition module; and input the second image parameters through the configuration module to optimize the processing process of the image processing module, so as to reduce the delay of the image acquisition and processing process.

[0083] Image acquisition module 1 is used to acquire the current spatial position information of the head-mounted display device worn by the user, and to acquire current image data according to the first image parameters. The current spatial position information and current image data are transmitted to the image processing module. The current image data includes: center image and edge image. The current spatial position information includes: current azimuth angle and / or current pitch angle.

[0084] In one feasible approach, the current spatial position information of the user's head-mounted display device, such as the current azimuth and / or current pitch angle, is acquired in real time via an image acquisition module. Current image data is then acquired based on first image parameters, where the current image data represents the user's real-world environment and includes both center and edge images. (Reference) Figure 2 As shown, the center image refers to the image in the center of the field of view, and the edge image refers to the image in the edge of the field of view. That is, by using the configured first image parameters, it is possible to achieve partitioned acquisition of different regions in the same image, meet diverse image processing needs, improve processing efficiency and accuracy, and thus reduce image link latency.

[0085] Image processing module 2 is used to obtain the target image processing strategy based on the current spatial location information and the second image parameters, and generate the current real scene image based on the target image processing strategy and the current image data, and transmit the current real scene image to the display module.

[0086] In one feasible approach, the image processing module uses the current spatial location information and the second image parameters to determine the target image processing strategy. That is, by using the second image parameters, the image update method can be dynamically adjusted, and based on the target image processing strategy and the current image data, the current real-world scene image can be quickly generated, reducing image processing time and achieving the goal of reducing image link latency and improving immersive experience.

[0087] Display module 3 is used to overlay the current real scene image with the pre-generated virtual image, that is, to achieve a display effect that combines virtual and reality by overlaying the current real scene image with the pre-generated virtual image, thereby enhancing the immersive experience of the user.

[0088] Therefore, in the micro-display system provided in this application, by optimizing the entire image link of image acquisition (i.e., partitioned acquisition) and processing (i.e., dynamically adjusting the image processing strategy), low-latency display is achieved end-to-end in the image link, which significantly improves the user's immersive experience and real-time interaction, thereby improving the user's poor experience caused by latency, such as dizziness and mismatched movements when using MR head-mounted displays, and enhancing immersion and real-time interaction.

[0089] In summary, this application provides a micro-display system comprising an image acquisition module, an image processing module, a display module, and a configuration module. The image acquisition module acquires the current spatial location information of the user's head-mounted display device in real time and collects current image data based on a first image parameter. This first image parameter allows for partitioned acquisition of different areas within the same image frame, meeting diverse image processing needs, improving processing efficiency and accuracy, and reducing image link latency. Then, the image processing module uses the current spatial location information and a second image parameter to determine a target image processing strategy. This second image parameter allows for dynamic adjustment of the image update method, and based on the target image processing strategy and current image data, quickly generates an image of the current real-world scene, reducing image processing time and achieving the goal of reducing image link latency and improving the immersive experience. Finally, the display module overlays the current real-world scene image with a pre-generated virtual image, achieving a combined virtual and real-world display effect, enhancing the user's immersive experience. Therefore, in the micro-display system provided in this application, by optimizing the entire image link of image acquisition (i.e., partitioned acquisition) and processing (i.e., dynamically adjusting the image processing strategy), low-latency display is achieved end-to-end in the image link, which significantly improves the user's immersive experience and real-time interaction, thereby improving the user's poor experience caused by latency, such as dizziness and mismatched movements when using MR head-mounted displays, and enhancing immersion and real-time interaction.

[0090] Optionally, refer to Figure 3As shown, the image acquisition module 1 includes: an angle acquisition unit 11 and an image capture unit 12; for example, the angle acquisition unit includes: an azimuth sensor and a pitch sensor, that is, the azimuth sensor is used to acquire the horizontal rotation angle of the head-mounted display device, that is, the azimuth angle, whose value range is usually 0°~360°; and the pitch sensor is used to acquire the rotation angle of the head-mounted display device around the horizontal axis (X-axis), that is, the pitch angle, which is usually in the range of -90°~+90°.

[0091] The angle acquisition unit 11 is used to acquire the current spatial position information of the head-mounted display device, namely the current azimuth angle and the current pitch angle. Based on the current azimuth angle and the current pitch angle, the attitude information of the head-mounted display device in three-dimensional space can be determined.

[0092] The shooting unit 12 is used to divide the image acquisition area in the shooting unit into partitions according to the first image parameters to obtain the central region of the field of view and the edge region of the field of view, and to acquire the central image located in the central region of the field of view and the edge image located in the edge region of the field of view according to the preset image acquisition mode.

[0093] For example, such as Figures 4-5 As shown, the image acquisition area in the shooting unit can be divided into three ranges: the sampling range i*j, the actual display resolution p*q, and the visual center area m*n. The area outside the visual center area can be called the field of view edge area.

[0094] Specifically, the sampling range i*j is the outermost circle, the corresponding angle range in the horizontal direction is denoted as α, and the corresponding angle range in the vertical direction is denoted as θ;

[0095] The actual display resolution p*q is the center circle, the corresponding angle range in the horizontal direction is denoted as β, and the corresponding angle range in the vertical direction is denoted as φ;

[0096] The visual center region m*n is the inner circle, with the corresponding angle range in the horizontal direction denoted as γ and the corresponding angle range in the vertical direction denoted as δ. The visual center region focuses on the area that the user is most interested in.

[0097] Therefore, it is possible Figures 4-5 As shown, these angle values ​​can be obtained through product calibration, and the visual center area is configured as m*n through the first image parameter as the first stage of the shooting unit partition, and the remaining part is the second stage of the shooting unit partition, where m≤p≤i and n≤q≤j.

[0098] Optionally, in the embodiments provided in this application, the image acquisition area of ​​the shooting unit is partitioned according to the second image acquisition parameters, that is, the field of view is divided into a central region and an edge region, and the central image located in the central region and the edge image located in the edge region are acquired according to a preset image acquisition mode. In this way, the image processing efficiency can be improved and the accuracy of the visual center region can be enhanced.

[0099] Optionally, according to a preset image acquisition mode, a central image located in the center region of the field of view and edge images located in the edge region of the field of view are acquired, including:

[0100] Use the global shutter mode to capture the center image; use the rolling shutter mode to capture the edge image.

[0101] Global shutter mode means that all pixels begin to be exposed at the same time, resulting in no motion blur or interline distortion in the image.

[0102] Rolling shutter mode refers to the exposure of pixels line by line.

[0103] Optionally, to improve image acquisition efficiency, this application proposes a regionalized hybrid shutter, that is, using different shutter modes for different regions. Specifically, a global shutter mode is used to acquire the central image located in the visual center region (such as the user's area of ​​focus) of the same image frame, and a rolling shutter mode is used to acquire the edge images located in the visual edge region of the same image frame. This sampling of global shutter and rolling shutter hybrid mode achieves a balance between visual quality, power consumption, and processing latency, reducing overall power consumption and latency accumulation.

[0104] Therefore, the regionalized hybrid shutter method proposed in this application is adopted, which divides the field of view into zones. The dynamic high-frequency zone, i.e. the center of the field of view, is forced to use a global shutter, while other zones use a rolling shutter to shorten the exposure time.

[0105] Optionally, the imaging unit includes: a photosensitive module, a dynamic scheduling and parallelization module, a photoelectric conversion circuit module, a preprocessing module, a compression and downsampling module, and a parallel multi-channel interface;

[0106] The photosensitive module is used to control the image sensor to perform shutter operations and supports regional hybrid shutter mode, which can balance latency accumulation and power consumption;

[0107] The photoelectric conversion circuit module is used to convert optical signals into electrical signals.

[0108] The dynamic scheduling and parallelization module prioritizes the retrieval of the center image acquired in the global shutter mode, also known as the first-stage data. This center image is then cached in the preprocessing module and begins conventional ISP processing, such as prioritizing dynamic content in the processing chain. Next, it retrieves the edge images acquired using the rolling shutter mode, also known as the second-stage data. Unlike previous methods, it does not require processing the entire frame using the rolling shutter mode, which results in latency accumulation. It also does not require processing the entire frame using the global shutter mode, which results in high power consumption. Furthermore, it does not require waiting line by line until the visual center area is reached before starting data processing, reducing inter-line waiting latency. It prioritizes the processing of the center image located in the visual center area and can also process the edge images located in the visual edge area in parallel, further reducing latency.

[0109] After processing, the central image located in the visual center area is preferentially transmitted to the subsequent image processing module through a parallel multi-channel interface, and then the edge images located in the visual edge area are transmitted to free up bus bandwidth. At the same time, a compression downsampling module is configured to further reduce the amount of data transmitted, which also reduces the amount of data that the subsequent image processing module needs to process, thus reducing the amount of data processed. The multi-channel parallel reading architecture allows pixel data from different channels to be transmitted simultaneously and stitched in real time, avoiding the single-channel bandwidth bottleneck.

[0110] The following examples will illustrate how to dynamically adjust image processing strategies.

[0111] Optionally, refer to Figure 6 As shown, based on the current spatial location information and the second image parameters, the target image processing strategy is obtained, including:

[0112] S101. Determine the current swing amplitude of the head-mounted display device based on the current spatial location information and the second image parameters.

[0113] S102. Based on the current swing amplitude of the head-mounted display device and the pre-built mapping relationship between the swing amplitude and the image processing strategy, determine the target image processing strategy corresponding to the current swing amplitude.

[0114] Swing amplitude is used to characterize the intensity of motion of the head-mounted display device in space.

[0115] The mapping relationship between swing amplitude and image processing strategy records the image processing strategies corresponding to different swing amplitudes. For example, if the swing amplitude B1 is the first swing amplitude, i.e., a smaller swing, then the target image processing strategy corresponding to the swing amplitude B1 can be determined as the first image processing strategy.

[0116] In one feasible approach, the current azimuth angle, current pitch angle, and second image parameters of the head-mounted display (HMD) can be combined to detect the current swing amplitude of the HMD. If the current swing amplitude is the first swing amplitude (i.e., a small swing), the target image processing strategy corresponding to the current swing amplitude can be retrieved from the mapping relationship between swing amplitude and image processing strategy. Therefore, different image processing strategies are adopted for different swing amplitudes, improving image processing efficiency and reducing latency.

[0117] Optionally, refer to Figure 7 As shown, with the vertical centerline as the reference, the second image parameter includes: azimuth threshold.

[0118] refer to Figure 8 As shown, step S101 above includes:

[0119] S201. If the absolute value of the current azimuth angle is less than or equal to the azimuth angle threshold, then the current swing amplitude of the head-mounted display device is determined as the first swing amplitude.

[0120] S202. If the absolute value of the current azimuth angle is greater than the azimuth angle threshold and less than or equal to the preset first maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude.

[0121] The second swing amplitude is greater than the first swing amplitude.

[0122] The first maximum offset angle is (α-β) / 2, where α is the total horizontal viewing angle of the sampling range, β is the total horizontal viewing angle of the actual display resolution, and (α-β) / 2 is half of the range of the "edge area" that extends from the actual display area to the left and right.

[0123] S203. If the absolute value of the current azimuth angle is greater than the maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude.

[0124] The third swing amplitude is greater than the second swing amplitude.

[0125] In one feasible way, continue to refer to Figure 7 As shown, with the vertical centerline as the reference, and an azimuth threshold λ set in the horizontal direction, the current swing amplitude of the head-mounted display is determined based on the absolute value of the current azimuth angle α. Specifically:

[0126] (1) If the absolute value of the current azimuth angle is less than or equal to the azimuth angle threshold, i.e., |a|≤λ, then the current swing amplitude of the head-mounted display device is determined to be the first swing amplitude, i.e., the smaller swing.

[0127] (2) If the absolute value of the current azimuth angle is greater than the azimuth angle threshold and less than or equal to the preset first maximum offset angle, i.e., λ<|a|≤(α-β) / 2, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude, i.e., medium swing.

[0128] (3) If the absolute value of the current azimuth angle is greater than the maximum offset angle, i.e. (α-β) / 2<|a|, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude, i.e., a larger swing.

[0129] Optionally, refer to Figure 9 As shown, with the horizontal centerline as the reference, the second image parameters include: pitch angle threshold;

[0130] refer to Figure 10 As shown, step S101 above includes:

[0131] S301. If the absolute value of the current pitch angle is less than or equal to the pitch angle threshold, then the current swing amplitude of the head-mounted display device is determined as the first swing amplitude.

[0132] S302. If the absolute value of the current pitch angle is greater than the pitch angle threshold and less than or equal to the preset second maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude.

[0133] The second swing amplitude is greater than the first swing amplitude.

[0134] The second maximum offset angle is (θ-φ) / 2, where θ is the viewing angle range of the sampling range in the vertical direction, φ is the viewing angle range of the actual display resolution in the vertical direction, and (θ - φ) / 2 is half of the "edge area" range that the actual display resolution extends upward and downward.

[0135] S303. If the absolute value of the current pitch angle is greater than the maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude.

[0136] The third swing amplitude is greater than the second swing amplitude.

[0137] In one feasible way, continue to refer to Figure 9 As shown, with the horizontal centerline as the reference, a pitch angle threshold η is set in the vertical direction. Then, the current swing amplitude of the head-mounted display is determined based on the absolute value of the current pitch angle b. Specifically:

[0138] (1) If the absolute value of the current pitch angle is less than or equal to the pitch angle threshold, i.e., |b|≤η, then the current swing amplitude of the head-mounted display device is determined to be the first swing amplitude, i.e., the smaller swing.

[0139] (2) If the absolute value of the current pitch angle is greater than the azimuth threshold and less than or equal to the preset maximum offset angle, i.e. η<|b|≤(θ-φ) / 2, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude, i.e., medium swing.

[0140] (3) If the absolute value of the current pitch angle is greater than the maximum offset angle, i.e. (θ-φ) / 2<|a|, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude, i.e., the larger swing.

[0141] Optionally, refer to Figure 11 As shown, step S102 above includes:

[0142] S401. If the current swing amplitude of the head-mounted display device is the first swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the first image processing strategy.

[0143] The first image processing strategy is used to indicate that the center image should not be updated.

[0144] S402. If the current swing amplitude of the head-mounted display device is the second swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the second image processing strategy.

[0145] The second image processing strategy is used to instruct the previous image data to be updated using the current image data.

[0146] S403. If the current swing amplitude of the head-mounted display device is the third swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is the third image processing strategy.

[0147] The third image processing strategy is used to instruct updates to be performed using the acquired current image data.

[0148] In one feasible approach, if the current oscillation amplitude of the head-mounted display is a first oscillation amplitude, the target image processing strategy corresponding to the first oscillation amplitude can be queried based on the mapping relationship between the oscillation amplitude and the image processing strategy. This means that the previous image data of the current image data is used for updating. Therefore, when a slight up-and-down movement of the head-mounted display is detected, the image content remains almost unchanged, avoiding unnecessary image acquisition and processing.

[0149] If the current swing amplitude of the head-mounted display is the second swing amplitude, then based on the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the second swing amplitude can be found to be the second image processing strategy. That is, the center image of the visual center region is updated, but the current image data is not re-acquired from the shooting unit. Instead, the previous image data that directly retrieves the current image data is processed (such as cropping, rotation, translation, or affine transformation). In this way, while ensuring visual continuity, the image acquisition frequency is reduced, and the system power consumption and resource consumption are lowered.

[0150] If the current swing amplitude of the head-mounted display is the third swing amplitude, then based on the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the third swing amplitude can be found to be the third image processing strategy, that is, the currently acquired image data is used for updating. Therefore, when a large up-and-down swing of the head-mounted display is detected, the previous image data using the current image data can no longer accurately reflect the current viewpoint. Therefore, it is necessary to re-acquire the current image data from the imaging unit to obtain the true image of the current viewpoint to ensure display accuracy.

[0151] Therefore, in this application, different image update strategies are executed based on the current swing amplitude of the head-mounted display device. That is, it dynamically determines whether to update the center image of the visual center area, whether to re-acquire the image from the shooting unit, or to use the previous image data of the cached current image data for image synthesis or transformation, so as to reduce the image processing latency.

[0152] Optionally, refer to Figure 12 As shown, based on the target image processing strategy and the current image data, an image of the current real-world scene is generated, including:

[0153] S501. If the target image processing strategy is the first image processing strategy, then generate the current real-world scene image based on the previous image data of the current image data.

[0154] In one feasible approach, if the target image processing strategy is the first image processing strategy, the previous image data can be directly used as the current real-world scene image. That is, the current real-world scene image retains the original frame, and the image content in the visual center area is not updated, saving processing resources and reducing latency in the image processing process.

[0155] S502. If the target image processing strategy is the second image processing strategy, then generate the current real-world scene image based on the current spatial location information and the previous image data of the current image data.

[0156] In another feasible approach, if the target image processing strategy is the second image processing strategy, the current real-world scene image can be generated based on the current spatial location information and the previous image data of the current image data. That is, the image content of the visual center area is updated, but the image data is not retrieved from the source shooting unit. Instead, the previous image data is directly retrieved and processed, such as by simulating the current viewpoint change through image transformation (such as translation, rotation, and cropping). In this way, while ensuring visual continuity, the image acquisition frequency can be reduced, and the system power consumption and resource consumption can be reduced.

[0157] S503. If the target image processing strategy is the third image processing strategy, then process the current image data to generate the current real-world scene image.

[0158] In another possible approach, if the target image processing strategy is the third image processing strategy, the current image data is directly obtained from the shooting unit and used as the current real-world scene image to ensure display accuracy.

[0159] Optionally, refer to Figure 13 As shown, step S502 above includes:

[0160] S601. Based on the current spatial location information, obtain the current pixel translation amount.

[0161] Pixel shift includes: row pixel shift or column pixel shift.

[0162] S602. Shift the previous image data according to the current pixel shift amount to generate the current real-world scene image.

[0163] The first method, which uses the vertical centerline as a reference to shift the previous image data horizontally, involves the following process:

[0164] Continue to refer to Figure 7 As shown, based on the geometric relationship of triangles, angle α corresponds to i column pixels, and the number of column pixels corresponding to each unit angle is equivalent to: Angle β corresponds to p column pixels, and the number of column pixels corresponding to each unit angle is equivalent to: The equivalent number of pixel columns corresponding to the (α-β) / 2 angle (maximum boundary) is: or .

[0165] Assuming rightward is the positive direction, if the current azimuth angle α > 0, the column pixel translation amount can be calculated as follows: That is, shift the previous image data to the right. The number of pixels (rounded down); if the current azimuth angle α < 0, the column pixel translation can be calculated as follows: This means shifting the previous image data to the left. Pixel units (rounded down).

[0166] The second method, which uses the horizontal center line as a reference to shift the previous image data vertically, involves the following process:

[0167] refer to Figure 9 As shown, based on the geometric relationship of triangles, angle θ corresponds to row j pixels, and the number of rows corresponding to each unit angle is equivalent to: Angle φ corresponds to q rows of pixels, and the number of rows corresponding to each unit angle is equivalent to: The equivalent number of rows corresponding to the angle (θ-φ) / 2 is: or .

[0168] Assuming upward is the positive direction, if the current pitch angle b > 0, then the row pixel translation amount can be calculated as follows: This means shifting the previous image data upwards. For each row unit (rounded down), if the current pitch angle b < 0, the row pixel translation amount can be calculated. That is, shift the previous image data downwards. Units per row (rounded down).

[0169] Optionally, refer to Figure 14 As shown, another micro-display system provided in this application is included. The micro-display system 100 further includes an interface module 5.

[0170] Interface module 5 is connected to the input of image processing module via the output of image acquisition module;

[0171] Interface module 5 is used to transmit the current image data acquired by the image acquisition module to the image processing module in a multi-channel parallel reading mode.

[0172] For example, interface module 5 can be an interface based on the MIPI CSI-3 standard.

[0173] In one feasible approach, to suit high-resolution, high-frame-rate image transmission scenarios, an interface module is proposed to be set up between the image acquisition module and the image processing module. The interface module adopts a multi-channel parallel reading mode, such as dividing the center image into one sub-data stream and the edge image into another sub-data stream, and transmitting the sub-data streams in parallel through multiple communication channels. The pixel data transmitted by each channel is stitched together in real time at the receiving end to reconstruct a complete image frame. This can effectively avoid the data transmission bottleneck caused by the bandwidth limitation of a single channel and significantly improve the data transmission efficiency.

[0174] Optionally, in the micro-display system provided in this application, low-latency display of the image link is achieved through methods such as partitioned acquisition, parallel transmission, visual center priority processing, and dynamic image update strategies. This improves the user experience, such as visual dizziness and motion mismatch caused by large image link delays, and enhances the immersion and real-time interaction during use.

[0175] Optionally, in the micro-display system provided in this application, the imaging unit is transformed from passive light-sensing acquisition to active light-sensing low-latency acquisition, and partitioned sampling is adopted, that is, priority is given to the center image of the visual center area at the front end of the link, light sensing-reading-preprocessing, reducing latency and leaving more time margin for back-end image processing and synthesis, thereby improving immersion and real-time performance; and a multi-channel parallel reading architecture is adopted, with parallel reading reducing transmission latency; only the row and column data images required for the visual center are processed, thereby reducing the amount of data processed and reducing image processing latency. At the same time, combined with partitioned sampling and threshold judgment, the refresh rate on the display side is increased, thereby reducing system latency.

[0176] Optionally, refer to Figure 15 As shown, the latency in the traditional image acquisition module to the display module can be divided into three parts, with the process proceeding sequentially as follows: event initiation - image acquisition module - interface module - image processing module - display module. (Refer to...) Figure 15 As shown, taking two consecutive frames of sampling as an example, the system delay Tdelay = t1 + t2 + t3 + t4. Here, t1 represents the delay time of the imaging unit in the image acquisition module from sensing to reading, preprocessing, and output; t2 represents the delay time from the interface module to the image processing module; t3 represents the delay time of the image processing module performing a series of image processing operations as required; and t4 represents the delay time for the processed images to be overlaid and displayed.

[0177] I-t1 represents the delay time of the image acquisition module itself from sensing to reading to preprocessing to output in the previous frame; I-t2 represents the delay time from the interface module to the image processing module in the previous frame; I-t3 represents the delay time of the image processing module in the previous frame performing a series of image processing operations as required; I-t4 represents the delay time of overlaying and displaying the processed image from the previous frame; and I-display represents the total reception and display time of the previous frame. II-t1 represents the delay time of the image acquisition module itself from sensing to reading to preprocessing to output in the current frame; II-t2 represents the delay time from the interface module to the image processing module in the current frame; II-t3 represents the delay time of the image processing module in the current frame performing a series of image processing operations as required; II-t4 represents the delay time of overlaying and displaying the processed image from the current frame; and II-display represents the total reception and display time of the current frame.

[0178] Therefore, from the above Figure 15 From this, we can obtain that the delay of a traditional micro-display system is Tdelay = t1 + t2 + t3 + t4.

[0179] Optionally, refer to Figure 16 The diagram shows the delay effect after optimizing the image acquisition and parallel transmission of the micro-display system. The process along the image link from the image acquisition module to the display module remains unchanged, and is as follows: event initiation - image acquisition module - interface module - image processing module - display module. Taking two consecutive frames as an example, when the sampling range and resolution are the same, i.e. i=p, j=q, with a sampling range of 1920*1080, a resolution of 1920*1080, a frame rate of 60, and a visual center area of ​​800*600, the system delay Tdelay1<t1+t2+t3+t4. The delay time t1 is shortened. In this example, the shape of the visual center area can be changed, with row 600 positioned in the middle of row 1080, with 240 rows above and below. Therefore, compared with the traditional sequential execution method, the row waiting time can be reduced by (240 / 1080) * t1, approximately 0.22t1. The delay time t2 remains unchanged. The amount of data processed in t3 becomes (800*600) / (1920*1080), reducing the delay time by approximately 0.23t3. The I-t1 of the previous frame is shortened, while the I-t2 and I-t3 of the previous frame remain unchanged. The I-t4 of the previous frame remains unchanged, and the time when the I-display of the previous frame receives valid data and begins to display is advanced. The II-t1 of the current frame is shortened, while the II-t2 of the current frame remains unchanged. The II-t3 of the current frame is shortened, while the II-t4 of the current frame remains unchanged. The time when the II-display of the current frame receives valid data and begins to display is advanced.

[0180] Optionally, refer to Figure 17 As shown, in Figure 16 Based on this, the delay effect diagram is generated after optimizing the image acquisition, parallel transmission, processing and display of the micro-display system.

[0181] When the sampling range is greater than the same resolution, i.e. i > p, j > q, with a sampling range of 2200*1200, a resolution of 1920*1080, a frame rate of 60, and a visual center area of ​​800*600, the process on the image link from the image acquisition module to the display module remains unchanged, in the following order: event initiation - image acquisition module - interface module - image processing module - display module. Taking two consecutive frames of sampling as an example, the system delay Tdelay1 < t1 + t2 + t3 + t4. The delay time t1 is shortened. In this example, the shape of the visual center area can be changed, with line 600 positioned in the middle of line 1080, with 240 lines above and below. Therefore, compared to the traditional sequential execution method, the line waiting time can be reduced by (240 / 1080) * t1, approximately 0.22t1. The delay time t2 remains unchanged, and the amount of data processed in t3 becomes (800*600) / (1920*1080), reducing the delay time by approximately 0.23t3. The I-t1 of the previous frame is shortened, the I-t2 of the previous frame remains unchanged, the I-t3 of the previous frame is shortened, and the I-t4 of the previous frame is shortened. The moment when I-display receives valid data and starts displaying in the previous frame is advanced. Corresponding to I-display-1 and I-display-2 in the figure, they do not need to wait for t1+t2 time and can directly retrieve data from the storage unit for processing and display. Moreover, the refresh rate of I-display is higher than that of the previous frame. Figure 16 The optimization effect is doubled; the current frame's II-t1 is shortened, the current frame's II-t2 remains unchanged, the current frame's II-t3 is shortened, the current frame's II-t4 is shortened, and the time when the current frame's II-display receives valid data and starts displaying is advanced. Corresponding to II-display-1 and II-display-2 in the diagram, they no longer need to wait for t1+t2; they can directly retrieve data, process it, and display it, resulting in a higher refresh rate. Figure 16 The optimization effect is doubled.

[0182] Optionally, refer to Figure 18 As shown, this application also provides a head-mounted display device, the head-mounted display device 200 including the micro-display system 100 provided in the above embodiments.

[0183] Optionally, using the head-mounted display device provided in this application can significantly reduce image link latency, thereby improving the user's visual dizziness and motion mismatch caused by large image link latency, and enhancing the immersion and real-time interaction during use.

[0184] Optionally, the present invention also provides a program product, such as a computer-readable storage medium, including a program that, when executed by a processor, is used to perform the above-described method embodiments.

[0185] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0186] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0188] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A micro-display system, characterized in that, The micro-display system includes: an image acquisition module, an image processing module, a display module, and a configuration module; The output of the image acquisition module is connected to the input of the image processing module, and the output of the image processing module is communicatively connected to the input of the display module. The configuration module is communicatively connected to both the image acquisition module and the image processing module. The configuration module is used to receive a first image parameter and a second image parameter input by the user, transmit the first image parameter to the image acquisition module, and transmit the second image parameter to the image processing module; The image acquisition module is used to acquire the current spatial position information of the head-mounted display device worn by the user, and to acquire current image data according to the first image parameters. The current spatial position information and the current image data are transmitted to the image processing module. The current image data includes: center image and edge image. The current spatial position information includes: current azimuth angle and / or current pitch angle. The image processing module is used to obtain a target image processing strategy based on the current spatial location information and the second image parameters, generate a current real-world scene image based on the target image processing strategy and the current image data, and transmit the current real-world scene image to the display module. The display module is used to overlay the current real-world scene image with a pre-generated virtual image.

2. The system according to claim 1, characterized in that, The image acquisition module includes: an angle acquisition unit and a shooting unit; The angle acquisition unit is used to acquire the current spatial position information of the head-mounted display device; The shooting unit is used to partition the image acquisition area in the shooting unit according to the first image parameters to obtain the central area of ​​the field of view and the edge area of ​​the field of view, and to acquire the central image located in the central area of ​​the field of view and the edge image located in the edge area of ​​the field of view according to a preset image acquisition mode.

3. The system according to claim 2, characterized in that, The step of acquiring a central image located in the center region of the field of view and an edge image located in the edge region of the field of view according to a preset image acquisition mode includes: The central image is captured using the global shutter mode; The edge image is captured using the rolling shutter mode.

4. The system according to claim 1, characterized in that, The step of obtaining the target image processing strategy based on the current spatial location information and the second image parameters includes: Based on the current spatial location information and the second image parameters, the current swing amplitude of the head-mounted display device is determined; Based on the current swing amplitude of the head-mounted display device and the pre-built mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is determined.

5. The system according to claim 4, characterized in that, Based on the vertical centerline, the second image parameters include: azimuth threshold; Determining the current swing amplitude of the head-mounted display device based on the current spatial location information and the second image parameters includes: If the absolute value of the current azimuth angle is less than or equal to the azimuth angle threshold, then the current swing amplitude of the head-mounted display device is determined to be the first swing amplitude. If the absolute value of the current azimuth angle is greater than the azimuth angle threshold and less than or equal to the preset first maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude, and the second swing amplitude is greater than the first swing amplitude. If the absolute value of the current azimuth angle is greater than the maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude, which is greater than the second swing amplitude.

6. The system according to claim 4, characterized in that, Based on the horizontal centerline, the second image parameters include: pitch angle threshold; Determining the current swing amplitude of the head-mounted display device based on the current spatial location information and the second image parameters includes: If the absolute value of the current pitch angle is less than or equal to the pitch angle threshold, then the current swing amplitude of the head-mounted display device is determined to be the first swing amplitude. If the absolute value of the current pitch angle is greater than the pitch angle threshold and less than or equal to the preset second maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the second swing amplitude, and the second swing amplitude is greater than the first swing amplitude. If the absolute value of the current pitch angle is greater than the maximum offset angle, then the current swing amplitude of the head-mounted display device is determined to be the third swing amplitude, which is greater than the second swing amplitude.

7. The system according to claim 4, characterized in that, The step of determining the target image processing strategy corresponding to the current swing amplitude based on the current swing amplitude of the head-mounted display device and a pre-built mapping relationship between the swing amplitude and the image processing strategy includes: If the current swing amplitude of the head-mounted display device is the first swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the first image processing strategy, wherein the first image processing strategy is used to indicate that the center image is not updated. If the current swing amplitude of the head-mounted display device is the second swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the second image processing strategy, wherein the second image processing strategy is used to indicate that the previous image data is used to update the current image data; If the current swing amplitude of the head-mounted display device is the third swing amplitude, then according to the mapping relationship between the swing amplitude and the image processing strategy, the target image processing strategy corresponding to the current swing amplitude is obtained as the third image processing strategy, wherein the third image processing strategy is used to indicate that the current image data collected is used for updating.

8. The system according to claim 7, characterized in that, The step of generating a current real-world scene image based on the target image processing strategy and the current image data includes: If the target image processing strategy is the first image processing strategy, then the current real-world scene image is generated based on the previous image data of the current image data; If the target image processing strategy is the second image processing strategy, then the current real-world scene image is generated based on the current spatial location information and the previous image data of the current image data; If the target image processing strategy is the third image processing strategy, then the current image data is processed to generate the current real-world scene image.

9. The system according to claim 8, characterized in that, The step of generating the current real-world scene image based on the current spatial location information and the previous image data of the current image data includes: Based on the current spatial position information, the current pixel translation amount is obtained, and the pixel translation amount includes: row pixel translation amount and column pixel translation amount; The previous image data is shifted according to the current pixel shift amount to generate the current real-world scene image; The micro-display system further includes: an interface module; the interface module is connected to the input of the image processing module via the output of the image acquisition module. The interface module is used to transmit the current image data acquired by the image acquisition module to the image processing module in a multi-channel parallel reading mode.

10. A head-mounted display device, characterized in that, The microdisplay system includes any one of claims 1-9.

Images