Screen display adjustment method, head-mounted display device and storage medium

Abstract

This invention discloses a screen display adjustment method, a head-mounted display device, and a storage medium. The method includes: dividing the image captured by the camera acquisition device into multiple sub-regions and determining the sharpness detection result of each sub-region; determining the sub-region to be adjusted and the corresponding pixel adjustment parameters of the sub-region by determining the sharpness detection result of each sub-region, thereby adjusting the pixel height of the unclear sub-region. After adjusting the pixel height of the unclear sub-region, the sharpness of the entire image area can be made uniform, thereby improving the display effect of the image captured by the head-mounted display device.

Description

Technical Field

[0001] This invention relates to the field of head-mounted display technology, and more particularly to a screen display adjustment method, a head-mounted display device, and a storage medium. Background Technology

[0002] With the widespread use of head-mounted display devices (such as VR, AR, XR, MR, etc.), people's requirements for the clarity of the displayed content are increasing. In related technologies, the display screen of head-mounted display devices adopts a flat design. However, because the optical module is curved during imaging, the sharpness at the edges of the screen is lower than that at the center of the screen during actual imaging. This results in different imaging sharpness in different areas of the display screen, leading to poor display effects of the head-mounted display devices. Summary of the Invention

[0003] This application provides a screen display adjustment method, a head-mounted display device, and a storage medium, aiming to improve the display effect of the head-mounted display device.

[0004] This application provides a screen display adjustment method, which includes:

[0005] Acquire the image captured by the camera acquisition device of the head-mounted display device, and determine the sharpness detection result of a sub-region in the image;

[0006] The pixel adjustment parameters of the sub-region to be adjusted are determined based on the sharpness detection results.

[0007] The pixel height of the sub-region to be adjusted is adjusted based on the pixel adjustment parameters.

[0008] Optionally, the step of determining the sharpness detection result of a sub-region in the image includes:

[0009] The image is divided into multiple sub-regions;

[0010] Each of the sub-regions is processed sequentially through the input layer, convolutional layer, pooling layer, fully connected layer and output layer of the sharpness detection model to obtain the output result of the output layer;

[0011] The sharpness detection result of each sub-region is determined based on the output results.

[0012] Optionally, before the step of processing each of the sub-regions sequentially through the input layer, convolutional layer, pooling layer, fully connected layer, and output layer of the sharpness detection model to obtain the output result of the output layer, the method further includes:

[0013] Acquire clear and unclear samples of the image;

[0014] An initial sharpness detection model is trained using the clear and unclear samples of the imaging image.

[0015] Obtain the output of the loss function corresponding to the initial sharpness detection model;

[0016] The classification accuracy is determined based on the output of the loss function.

[0017] When the classification accuracy reaches the preset classification accuracy, the sharpness detection model is obtained.

[0018] Optionally, after the step of adjusting the pixel height of the sub-region to be adjusted based on the pixel adjustment parameters, the method further includes:

[0019] By stitching together the sub-regions with adjusted pixel heights and the sub-regions in the image that have not undergone pixel adjustment, a stitched image is obtained.

[0020] The stitched image is updated to the imaging image, and the process is returned to determine the sharpness detection result of the sub-region in the imaging image.

[0021] Optionally, the step of determining the pixel adjustment parameters of the sub-region to be adjusted based on the sharpness detection result includes:

[0022] The sub-regions whose sharpness detection results are unclear are identified as the sub-regions to be adjusted;

[0023] Determine the pixel height corresponding to the sub-region to be adjusted;

[0024] The electrical performance parameters of the lifting and moving mechanism corresponding to the sub-region to be adjusted are determined based on the pixel height.

[0025] The preset pixel adjustment parameters associated with the electrical performance parameters are used to determine the pixel adjustment parameters of the sub-region to be adjusted.

[0026] Optionally, the step of determining the pixel height corresponding to the sub-region to be adjusted includes:

[0027] Obtain the preset pixel value and the current pixel value corresponding to the sub-region to be adjusted;

[0028] The pixel height of the sub-region to be adjusted is determined based on the preset pixel value and the current pixel value.

[0029] Optionally, the head-mounted display device is equipped with at least two camera acquisition devices; the step of acquiring the image captured by the camera acquisition devices of the head-mounted display device includes:

[0030] Determine the shooting parameters corresponding to the camera acquisition devices located at different positions of the head-mounted display device, wherein the shooting parameters include at least the shooting angle and the shooting time;

[0031] Determine the shooting area corresponding to the camera acquisition device;

[0032] The camera acquisition device is controlled to capture images of the shooting area based on corresponding shooting parameters, thereby obtaining the image captured by the camera acquisition device.

[0033] In addition, to achieve the above objectives, the present invention also provides a head-mounted display device comprising: a memory, a processor, and a screen display adjustment program stored in the memory and executable on the processor, wherein the screen display adjustment program, when executed by the processor, implements the steps of the screen display adjustment method described above.

[0034] Optionally, the head-mounted display device includes: at least two camera acquisition devices;

[0035] A pixel block adjustment module, comprising a lifting and moving mechanism, is used to adjust the pixel height of the pixel block corresponding to the sub-region.

[0036] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a screen display adjustment program thereon, which, when executed by a processor, implements the steps of the screen display adjustment method described above.

[0037] This application provides a screen display adjustment method, a head-mounted display device, and a storage medium. The method divides the image captured by the camera acquisition device into multiple sub-regions and determines the sharpness detection result of each sub-region. By determining the sharpness detection result of each sub-region, the method identifies the sub-region to be adjusted and the corresponding pixel adjustment parameters, thereby adjusting the pixel height of the unclear sub-regions. After adjusting the pixel height of the unclear sub-regions, the sharpness of the entire image area becomes more uniform, thus improving the display effect of the head-mounted display device. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of an embodiment of the pixel block adjustment device involved in the present application;

[0040] Figure 3 This is a flowchart illustrating the first embodiment of the screen display adjustment method of the present invention.

[0041] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings are only one embodiment and not the entirety of the invention. Detailed Implementation

[0042] Currently, most mainstream head-mounted display devices have flat screens, but the optical modules used for imaging are curved. This characteristic of head-mounted display devices inevitably results in different image clarity in different areas of the screen. For example, the image clarity is highest at the center of the field of view, and the image clarity is worse closer to the edge, resulting in poor display effect of head-mounted display devices.

[0043] Therefore, to address the problem of insufficient edge sharpness caused by the use of flat screens in current head-mounted displays, this invention proposes a screen display adjustment method. This application divides the image captured by the camera acquisition device into multiple sub-regions and determines the sharpness detection result of each sub-region. By determining the sharpness detection result of each sub-region, the sub-region to be adjusted and the corresponding pixel adjustment parameters are determined, thereby adjusting the pixel height of the unclear sub-regions. After adjusting the pixel height of the unclear sub-regions, the sharpness of the entire image area becomes more uniform, thus improving the display effect of the head-mounted display.

[0044] In addition, this application adds a pixel block adjustment module to the original head-mounted display device structure. This pixel block adjustment module includes: a locking mechanism for locking the pixel block by lateral extension and retraction; and a lifting and moving mechanism for moving the pixel block vertically when it is not locked. The virtual reality display screen is composed of pixel blocks, and each pixel block is adjusted vertically by the aforementioned pixel block adjustment module. The vertical adjustment of the pixel blocks on the display screen is performed; content is displayed based on the adjusted pixel block height.

[0045] Each pixel on the display screen can fluctuate up and down on its original plane. By changing the distance between each pixel on the display screen and the lens of the virtual reality product, that is, changing the optical path difference of each light-emitting point in the optical path of the virtual reality product, the overall image quality can be improved. This can further enhance the image clarity of the virtual reality product, providing optical designers with new ideas in the design process. It innovatively solves the current problem that limits the further improvement of image quality. By the fluctuation of each pixel on the screen, the image display can be dynamically adjusted, improving the output image quality of the virtual reality product, making the optical display effect of the virtual reality product more realistic, and giving users a more realistic virtual reality experience.

[0046] This effectively improves the imaging quality of virtual reality products during the design and debugging stages, including optical design and production verification. Without affecting the overall optical design (Fresnel or ultra-short-throw optical folding path), it ensures optimal display effects in both the center and edge areas of the virtual reality display screen, further enhancing the imaging quality and clarity of virtual reality products.

[0047] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0048] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a head-mounted display device in the hardware operating environment of the embodiments of this application.

[0049] like Figure 1 As shown, the head-mounted display device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0050] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the head-mounted display device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0051] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a screen display adjustment program.

[0052] exist Figure 1In the head-mounted display device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the head-mounted display device of this application can be set in the head-mounted display device, and the head-mounted display device calls the screen display adjustment program stored in the memory 1005 through the processor 1001 and performs the following operations:

[0053] Acquire the image captured by the camera acquisition device of the head-mounted display device, and determine the sharpness detection result of a sub-region in the image;

[0054] The pixel adjustment parameters of the sub-region to be adjusted are determined based on the sharpness detection results.

[0055] The pixel height of the sub-region to be adjusted is adjusted based on the pixel adjustment parameters.

[0056] In one embodiment, the processor 1001 may call a screen display adjustment program stored in the memory 1005 and further perform the following operations:

[0057] The image is divided into multiple sub-regions;

[0058] Each of the sub-regions is processed sequentially through the input layer, convolutional layer, pooling layer, fully connected layer and output layer of the sharpness detection model to obtain the output result of the output layer;

[0059] The sharpness detection result of each sub-region is determined based on the output results.

[0060] In one embodiment, the processor 1001 may call a screen display adjustment program stored in the memory 1005 and further perform the following operations:

[0061] Based on the pixel height of the sub-region to be adjusted, the pixel-adjusted sub-region is obtained;

[0062] By stitching together the pixel-adjusted sub-regions with the unadjusted sub-regions in the image, a stitched image is obtained.

[0063] The stitched image is updated to the imaging screen, and the process returns to the step of dividing the imaging screen into multiple sub-regions.

[0064] In one embodiment, the processor 1001 may call a screen display adjustment program stored in the memory 1005 and further perform the following operations:

[0065] Acquire clear and unclear samples of the image;

[0066] An initial sharpness detection model is trained using the clear and unclear samples of the imaging image.

[0067] Obtain the output of the loss function corresponding to the initial sharpness detection model;

[0068] The classification accuracy is determined based on the output of the loss function.

[0069] When the classification accuracy reaches the preset classification accuracy, the sharpness detection model is obtained.

[0070] In one embodiment, the processor 1001 may call a screen display adjustment program stored in the memory 1005 and further perform the following operations:

[0071] The sub-regions whose sharpness detection results are unclear are identified as the sub-regions to be adjusted;

[0072] Determine the pixel height corresponding to the sub-region to be adjusted;

[0073] The electrical performance parameters of the lifting and moving mechanism corresponding to the sub-region to be adjusted are determined based on the pixel height.

[0074] The preset pixel adjustment parameters associated with the electrical performance parameters are used to determine the pixel adjustment parameters of the sub-region to be adjusted.

[0075] In one embodiment, the processor 1001 may call a screen display adjustment program stored in the memory 1005 and further perform the following operations:

[0076] Obtain the preset pixel value and the current pixel value corresponding to the sub-region to be adjusted;

[0077] The pixel height of the sub-region to be adjusted is determined based on the preset pixel value and the current pixel value.

[0078] In one embodiment, the processor 1001 may call a screen display adjustment program stored in the memory 1005 and further perform the following operations:

[0079] Determine the shooting parameters corresponding to the camera acquisition devices located at different positions of the head-mounted display device, wherein the shooting parameters include at least the shooting angle and the shooting time;

[0080] Determine the shooting area corresponding to the camera acquisition device;

[0081] The camera acquisition device is controlled to capture images of the shooting area based on corresponding shooting parameters, thereby obtaining the image captured by the camera acquisition device.

[0082] Optionally, the head-mounted display device of this application further includes: a pixel block adjustment module, as shown in the reference. Figure 2 In one embodiment of the pixel block adjustment module, the pixel block adjustment module includes:

[0083] A locking and fixing mechanism is used to lock and fix pixel blocks by lateral extension and retraction;

[0084] The lifting and moving mechanism is used to move the pixel block up and down when the pixel block is not locked in place, that is, to adjust the pixel height of the pixel block corresponding to the sub-region.

[0085] The locking mechanism for fixing the pixel blocks secures them by lateral extension and retraction, preventing vertical or horizontal displacement while they are fixed. The lifting mechanism moves the pixel blocks vertically when they are not locked, altering their position. This allows each pixel to move up and down within its original plane, ensuring safety and preventing compression or wear. By changing the distance between each pixel and the lens of the virtual reality product, and altering the optical path difference of each light-emitting point in the product's optical path, the overall image quality is improved, further enhancing the image clarity. This effectively improves the image quality of virtual reality products during the design and production verification stages. Without affecting the overall optical design (Fresnel or ultra-short-throw optics pancake), it ensures optimal display effects in both the center and edge areas of the virtual reality display screen, further improving the image quality and clarity of the virtual reality product. It should be noted that, in order to ensure the best display effect for virtual reality products during the design, production and debugging stages before leaving the factory, the function of adjusting the vertical position of the pixel blocks is not available to users during actual use after product delivery.

[0086] For example, both the locking and fixing mechanism and the lifting and moving mechanism are magnetic components.

[0087] Among them, the locking and fixing mechanism for fixing the pixel block and the lifting and moving mechanism for moving the pixel block up and down are both magnetic components. Based on the electromagnetic effect, the electromagnetic force of the magnetic components is changed by changing the current flowing through the locking and fixing mechanism and the lifting and moving mechanism, thereby realizing the fixing of the pixel block and the moving of the pixel block up and down without contact.

[0088] For example, the locking and fixing mechanism includes: a first electromagnet group located on the left side of the pixel block, and a second electromagnet group located on the right side of the pixel block, wherein the locking forces generated by the first electromagnet group and the second electromagnet group are mutually balancing forces.

[0089] The first electromagnet group of the locking and fixing mechanism is located on the left side of the pixel block, and the second electromagnet group of the locking and fixing mechanism is located on the right side of the pixel block. The locking forces generated by the first electromagnet group and the second electromagnet group are mutually balancing forces, both acting on the pixel block, thereby achieving non-contact fixing of the pixel block.

[0090] For example, the electromagnet assembly includes a first electromagnet that moves up and down together with the pixel block, and a second electromagnet fixed on a preset fixing block on the side of the pixel block.

[0091] The electromagnet assembly includes a first electromagnet that moves up and down together with the pixel block, and a second electromagnet fixed to a preset fixing block on the side of the pixel block. The preset fixing block can be the second electromagnet itself, vertically fixed to the base of the pixel block. In this embodiment, for the electromagnet...

[0092] The method of fixing the iron to the pixel block is not limited. In one embodiment, the first electromagnet and the second electromagnet are two vertically parallel electromagnets, and the length of the first electromagnet is shorter than the length of the second electromagnet, so that the pixel block can be fixed in place no matter what height it is moved up or down.

[0093] For example, the lifting and moving mechanism includes: a third electromagnet that moves up and down together with the pixel block, and a fourth electromagnet fixed on the base of the pixel block.

[0094] Similarly, the lifting and moving mechanism includes a third electromagnet that moves up and down together with the pixel block, and a fourth electromagnet fixed on the base of the pixel block.

[0095] In one embodiment, reference is made to Figure 2 In this system, 1 represents a single pixel block, 2 represents a locking mechanism for fixing the pixel block, 3 represents the third electromagnet of the lifting and moving mechanism, and 4 represents the fourth electromagnet of the lifting and moving mechanism. Initially, electromagnets 3 and 4 are both unenergized, meaning they have no magnetism.

[0096] The pixel block 1 is fixed in its default position by the locking mechanism 2 and cannot move up or down. When it is necessary to adjust the undulation of the pixel block 1, electromagnets 3 and 4 are energized. The different magnitudes of the current flowing through electromagnets 3 and 4 will cause the pixel block 1 to tend to move upward or downward. In this embodiment, the magnetic force is set as a repulsive force. By adjusting the magnitude of the repulsive force (current) between the two poles of the electromagnets, the pixel block 1 can be moved upward or downward. After the electromagnets 3 and 4 are energized, the locking mechanism 2 is engaged.

[0097] Mechanism 2 unlocks, and pixel block 1 moves up or down a certain distance under the influence of magnetism. Once pixel block 1 is adjusted to the ideal position, locking mechanism 2 locks, fixing pixel block 1 in its adjusted ideal position. Finally, electromagnets 3 and 4 are de-energized, and the adjustment is complete.

[0098] In one embodiment, the head-mounted display device of this application further includes at least two camera acquisition devices. For example, when there are two camera acquisition devices, these two camera acquisition devices can be arranged opposite to each other.

[0099] When there are four acquisition devices, corresponding camera acquisition devices can be set in four directions of the head-mounted display device. For example, four camera acquisition devices are used on the face side: the camera acquisition device on the top is used to capture the image of the middle and lower half of the image; the camera acquisition device on the bottom is used to capture the image of the middle and upper half of the image; the camera acquisition device on the left is used to capture the image of the middle and right half of the image; and the camera acquisition device on the right is used to capture the image of the middle and left half of the image. The four images captured by the above camera acquisition devices can be used for full-field image sharpness detection.

[0100] This application provides a virtual reality display screen, which is composed of pixel blocks, and each pixel block is adjusted up and down by the aforementioned pixel block adjustment module.

[0101] Currently, most virtual reality (VR) products on the market use rectangular or polygonal flat screens. The light emanating from the screen, carrying image information, passes through lenses before entering the user's eye. Due to the refraction of light by the lenses, the image is sharpest at the center. As the image transitions from the center to the edges, distortion and blurring increase significantly, affecting the user's viewing experience. VR product screens are typically designed as flat by optical engineers during optical path simulations. This leads to a situation where, during the design process, the image quality at the edges is poor when the center field of view is clear, and vice versa. The final design result is that the image quality at both the center and edges of the displayed image is below the theoretically best possible value; it's a trade-off between the two.

[0102] If the distance between each pixel on the screen (i.e., the light source) and the lens in the design of a virtual reality product can be changed, the image quality can be significantly improved during the initial optical path design of the VR product. This helps optical design engineers correct aberrations, distortions, field curvature, chromatic aberration, and other issues in the overall image during the initial design phase. During actual assembly and testing, the height of pixels in areas of unclear imaging on the screen can also be dynamically adjusted to further improve the image quality and clarity of the virtual reality product.

[0103] Compared to other flat screens, the display screen in this embodiment is approximately 15mm thicker. This increased thickness is due to the pixel lifting mechanism and the reserved adjustable pixel distance. Each pixel on the screen is divided into a tiny square, beneath which is an adjustable pixel height telescopic mechanism. Combined with accompanying software and algorithms, and through computer analysis and control, it automatically raises or lowers individual pixels or pixel arrays within the target area by a certain distance.

[0104] In this embodiment, a virtual reality display screen is proposed, in which each pixel on the screen can move up and down on the plane where the original pixel is located. By changing the distance between each pixel on the screen and the lens of the VR product, that is, changing the optical path difference of each light-emitting point in the optical path of the VR product, the overall image quality is improved. The screen is initially a single flat screen, but each pixel contained therein can be moved up and down by ±6mm by a tiny adjustment mechanism underneath. The displacement method is magnetic displacement, and the voltage of each magnetic lifting mechanism can be controlled by software to move a single pixel up or down relative to the initial plane.

[0105] This application provides a screen display adjustment method, referring to... Figure 3 In one embodiment of the screen display adjustment method, the method is applied to a head-mounted display device as described above, including:

[0106] Step S110: Obtain the image captured by the camera acquisition device of the head-mounted display device, and determine the sharpness detection result of the sub-region in the image;

[0107] In this embodiment, each pixel can be adjusted vertically on its original plane. By changing the distance between each pixel and the lens of the virtual reality product, and altering the optical path difference of each light-emitting point in the optical path of the virtual reality product, the overall image quality is improved. Content is then displayed on the screen based on the adjusted pixel block height, further enhancing the image clarity of the virtual reality product. This effectively improves the image quality of the virtual reality product during the design and debugging stages, including optical design and production verification. Without affecting the overall optical scheme, it ensures optimal display effects in both the central and edge areas of the virtual reality display screen, further enhancing the image quality and clarity of the virtual reality product.

[0108] Optionally, to improve the clarity of the image, this application uses a camera acquisition device positioned close to the face on the head-mounted display device, specifically for capturing the image from the optical module. Optionally, the number of these camera acquisition devices can be determined based on actual conditions. A higher number of camera acquisition devices results in a clearer image. For example, when there is only one camera acquisition device, it can be controlled to capture different display areas of the image on the screen from different shooting angles. After correcting the display areas, the images are stitched together to obtain the final image.

[0109] For example, when there are at least two camera acquisition devices, the step of acquiring the image captured by the camera acquisition device of the head-mounted display device includes:

[0110] Step S111: Determine the shooting parameters corresponding to the camera acquisition devices located at different positions of the head-mounted display device. The shooting parameters include at least the shooting angle and the shooting time.

[0111] Step S112: Determine the shooting area corresponding to the camera acquisition device;

[0112] Step S113: Control the camera acquisition device to capture images of the shooting area based on the corresponding shooting parameters, and obtain the image captured by the camera acquisition device.

[0113] In this embodiment, for example, when there are four camera acquisition devices, considering the shooting angle, in order to detect the image sharpness across the entire field of view, this application employs four camera acquisition devices on the face side (top, bottom, left, and right): the top camera acquisition device captures the image of the middle and lower half of the image; the bottom camera acquisition device captures the image of the middle and upper half of the image; the left camera acquisition device captures the image of the middle and right half of the image; and the right camera acquisition device captures the image of the middle and left half of the image. The four images captured by the above camera acquisition devices can then be used for full-field-of-view image sharpness detection.

[0114] Optionally, the camera acquisition device can be controlled to capture images of the corresponding shooting area of ​​the head-mounted display device based on a corresponding shooting angle and shooting time, and the captured area is determined as the image captured by the corresponding camera acquisition device. The shooting angle and shooting time can be preset according to actual conditions. To improve the display effect of subsequent images, the camera acquisition device must maintain a consistent shooting time.

[0115] Optionally, after acquiring the image, the sharpness detection result of a sub-region in the image can be determined. For example, determining the sharpness detection result of the sub-region in the image includes:

[0116] Step S114: Divide the image into multiple sub-regions;

[0117] Step S115: Each of the sub-regions is processed sequentially through the input layer, convolutional layer, pooling layer, fully connected layer and output layer of the sharpness detection model to obtain the output result of the output layer;

[0118] Step S116: Determine the sharpness detection result of each of the sub-regions based on the output results.

[0119] In this embodiment, the image captured by the camera acquisition device can be used as input to divide the entire field of view into several sub-regions. Then, a sharpness detection model is used to obtain the sharpness detection result corresponding to each sub-region. The number of sub-regions depends on the required image detection accuracy and can be set manually or dynamically. Optionally, the sharpness detection result includes both sharp and blurry results. Alternatively, the sharpness detection result can be divided into sharpness levels, with lower levels indicating less sharpness and higher levels indicating sharpness. The sharpness detection result of each sub-region is determined based on the sharpness level.

[0120] Optionally, this application can process the sub-region sequentially through the input layer, convolutional layer, pooling layer, fully connected layer and output layer of the sharpness detection model to obtain the output result of the output layer, and then determine the sharpness detection result of the sub-region based on the output result of the output layer.

[0121] For example, before processing the sub-region through the input layer, convolutional layer, pooling layer, fully connected layer, and output layer of the sharpness detection model to obtain the output result of the output layer, the sharpness detection model needs to be trained. The training of the sharpness detection model in this application includes the following steps:

[0122] Step S210: Obtain clear and unclear image samples;

[0123] Step S220: Using the clear image samples and the unclear image samples, train an initial sharpness detection model;

[0124] Step S230: Obtain the output result of the loss function corresponding to the initial sharpness detection model;

[0125] Step S240: Determine the classification accuracy based on the output of the loss function;

[0126] Step S250: When the classification accuracy reaches the preset classification accuracy, the sharpness detection model is obtained.

[0127] In this embodiment, this application proposes a sharpness detection model based on the aforementioned convolutional neural network technology. During the model training phase, a large number of known clear and unclear image samples are used as training samples and input into the convolutional neural network for training. Gradient descent is used to continuously reduce the loss function, ultimately achieving a relatively ideal classification accuracy, enabling the model to determine whether an image is sharp. When using this model, for the input image, after different stages of computation such as convolutional layers and pooling layers, the final output is a judgment of the image sharpness, for example, 1 for sharp and 0 for unclear. Based on the judgment result of the convolutional neural network model, unclear display areas can be adjusted to achieve the desired image sharpness effect. The trained convolutional neural network model can be ultimately built into the operating system as a system file, allowing users to directly call it during actual use, and the system can update it with the latest and more accurate model via OTA (Over-The-Air).

[0128] Step S120: Determine the pixel adjustment parameters of the sub-region to be adjusted based on the sharpness detection results.

[0129] In this embodiment, before determining the pixel adjustment parameters of the sub-region to be adjusted, it is necessary to first determine the sub-region to be adjusted. Optionally, the sub-region to be adjusted can be determined based on the image display clarity of the display screen. The determination of the sub-region to be adjusted depends on the algorithm automatically determining it based on the image clarity and adjusting the height of the pixels corresponding to the unclear parts of the image. That is, the pixels of the sub-region to be adjusted are adjusted up and down until the algorithm detects that the image of the sub-region to be adjusted has reached the highest clarity, at which point the adjustment of the pixels of the sub-region to be adjusted is stopped.

[0130] Optionally, the sub-region to be adjusted is determined by comparing the theoretical sharpness with the actual sharpness of the image currently displayed on the screen. After determining the sub-region to be adjusted, the step of determining the pixel adjustment parameters of the sub-region to be adjusted based on the sharpness detection result includes:

[0131] Step S121: The sub-regions whose sharpness detection results are unclear are identified as the sub-regions to be adjusted;

[0132] Step S122: Determine the pixel height corresponding to the sub-region to be adjusted.

[0133] In this embodiment, pixel height is the physical distance representing one pixel on the screen. Assuming a 10*10cm screen has a resolution of 100*100, then the pixel height is 0.1cm. If the 10*10cm screen has a resolution of 1000*1000, then the pixel height is 0.01cm. If a character occupies 4 pixels, then the actual size of the character is 0.02*0.02cm².

[0134] For example, a preset pixel value and a current pixel value corresponding to the sub-region to be adjusted can be obtained, and the pixel height of the sub-region to be adjusted can be determined based on the preset pixel value and the current pixel value.

[0135] Step S123: Determine the electrical performance parameters of the lifting and moving mechanism corresponding to the sub-region to be adjusted based on the pixel height;

[0136] Step S124: Determine the pixel adjustment parameters of the sub-region to be adjusted by associating the electrical performance parameters with the preset pixel adjustment parameters.

[0137] In this embodiment, each sub-region has a corresponding lifting and moving mechanism, the specific structure of which can be referred to in the first embodiment and will not be repeated here. Each pixel height corresponds to a specific electrical performance parameter, which can be current, voltage, etc. Each electrical performance parameter has an associated preset pixel adjustment parameter, which is then used as the pixel adjustment parameter for the sub-region to be adjusted. Optionally, there can be multiple electrical performance parameters, and the preset pixel adjustment parameter corresponding to each electrical performance parameter can be obtained separately, thereby allowing for targeted adjustment and improving adjustment accuracy.

[0138] Step S130: Adjust the pixel height of the sub-region to be adjusted based on the pixel adjustment parameters.

[0139] In this embodiment, after determining that the actual sharpness of the image displayed on the display screen is less than the theoretical sharpness, the pixel count of the sub-region to be adjusted is increased; after determining that the actual sharpness of the image displayed on the display screen is greater than or equal to the theoretical sharpness, the pixel count of the sub-region to be adjusted is decreased.

[0140] Determining the sub-region to be adjusted relies on an algorithm that compares the theoretical sharpness of the image currently displayed on the screen with the actual sharpness of the displayed image, automatically identifying the sub-region to be adjusted corresponding to the unclear area. In this embodiment, the strategy for adjusting the sub-region by raising and lowering pixels is as follows: after determining that the actual sharpness of the image displayed on the screen is less than the theoretical sharpness, the pixels of the sub-region to be adjusted are increased; after determining that the actual sharpness of the image displayed on the screen is greater than or equal to the theoretical sharpness, the pixels of the sub-region to be adjusted are decreased. This adjusts the image of the sub-region to be adjusted towards the highest sharpness until the algorithm detects that the image of the sub-region to be adjusted has reached the highest sharpness, at which point the adjustment of the pixels in the sub-region to be adjusted stops.

[0141] For example, after the step of until the image display sharpness reaches the optimal sharpness of the display screen, the method further includes:

[0142] Obtain the target adjustment region and the target action to adjust the target adjustment region, and adjust the pixel points of the target adjustment region by raising and lowering based on the target action.

[0143] In addition to determining the sub-region to be adjusted by comparing sharpness using algorithms and then adjusting it accordingly, considering that algorithms cannot be completely accurate, have a probability of misjudgment and error, and that unreasonable algorithm settings may not conform to the user's viewing experience, this embodiment first obtains the target adjustment region and the target action to adjust the target adjustment region. Based on the target action, the vertical movement of pixels in the target adjustment region is adjusted. For example, the area whose pixel height is to be adjusted can be manually input in the software, along with the positive and negative height values ​​relative to the original plane. The algorithm can then automatically adjust the height difference between pixels through a dynamic adjustment mechanism under each individual pixel on the screen.

[0144] In this embodiment, each pixel of the display screen of the head-mounted display device can be raised and lowered. The content is displayed based on the adjusted pixel height.

[0145] Each pixel on the display screen can fluctuate up and down on its original plane. By changing the distance between each pixel on the display screen and the lens of the virtual reality product, that is, changing the optical path difference of each light-emitting point in the optical path of the virtual reality product, the overall image quality can be improved. This can further enhance the image clarity of the virtual reality product, providing optical designers with new ideas in the design process. It innovatively solves the current problem that limits the further improvement of image quality. By the fluctuation of each pixel on the screen, the image display can be dynamically adjusted, improving the output image quality of the virtual reality product, making the optical display effect of the virtual reality product more realistic, and giving users a more realistic virtual reality experience.

[0146] This effectively improves the imaging quality of virtual reality products during the design and debugging stages, including optical design and production verification. Without affecting the overall optical design (Fresnel or ultra-short-throw optical folding path), it ensures optimal display effects in both the center and edge areas of the virtual reality display screen, further enhancing the imaging quality and clarity of virtual reality products.

[0147] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a screen display adjustment program, which, when executed by a processor, implements the steps of the screen display adjustment method described above.

[0148] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the embodiments of this application.

[0150] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of adjusting a screen display, characterized by, The application is applied to a head-mounted display device provided with at least two camera acquisition devices and a pixel point block adjusting module, the pixel point block adjusting module comprises: a locking and fixing mechanism for locking and fixing a pixel point block through horizontal extension and contraction; a lifting and moving mechanism for moving the pixel point block up and down when the pixel point block is not locked and fixed, so as to adjust the pixel height of the pixel point block corresponding to a sub-region, and the method comprises: determining the shooting parameters corresponding to the camera acquisition devices located at different positions of the head-mounted display device, the shooting parameters at least comprising a shooting angle and a shooting time; determining the shooting regions corresponding to the camera acquisition devices; controlling the camera acquisition devices to shoot the shooting regions based on the corresponding shooting parameters, obtaining the imaging pictures collected by the camera acquisition devices, and dividing the imaging pictures into multiple sub-regions; processing each sub-region through an input layer, a convolution layer, a pooling layer, a full connection layer and an output layer of a definition detection model in sequence to obtain the output result of the output layer; determining the definition detection result of each sub-region according to the output result; determining the pixel adjustment parameter of the sub-region to be adjusted according to the definition detection result; adjusting the pixel height of the sub-region to be adjusted based on the pixel adjustment parameter; the step of determining the pixel adjustment parameter of the sub-region to be adjusted according to the definition detection result comprises: determining the sub-region with unclear definition as the sub-region to be adjusted; determining the pixel height corresponding to the sub-region to be adjusted; determining the electrical performance parameter of the lifting and moving mechanism corresponding to the sub-region to be adjusted according to the pixel height; determining the pixel adjustment parameter of the sub-region to be adjusted by associating the preset pixel adjustment parameter with the electrical performance parameter.

2. The method of claim 1, wherein, Before the step of processing each sub-region through the input layer, the convolution layer, the pooling layer, the full connection layer and the output layer of the definition detection model in sequence to obtain the output result of the output layer, the method further comprises: obtaining clear imaging sample and unclear imaging sample; training an initial definition detection model by using the clear imaging sample and the unclear imaging sample; obtaining the output result of the loss function corresponding to the initial definition detection model; determining the classification accuracy according to the output result of the loss function; obtaining the definition detection model when the classification accuracy reaches a preset classification accuracy.

3. The method of claim 1, wherein, After the step of adjusting the pixel height of the sub-region to be adjusted based on the pixel adjustment parameter, the method further comprises: splicing the sub-region with adjusted pixel height and the sub-region without pixel adjustment in the imaging picture to obtain a spliced image; updating the spliced image as the imaging picture and returning to execute the step of determining the definition detection result of the sub-region in the imaging picture.

4. The method of claim 1, wherein, The step of determining the pixel height corresponding to the sub-region to be adjusted comprises: obtaining the preset pixel value and the current pixel value corresponding to the sub-region to be adjusted; determining the pixel height of the sub-region to be adjusted according to the preset pixel value and the current pixel value.

5. A head-mounted display device, comprising: The head-mounted display device comprises a memory, a processor, and a screen display adjustment program stored on the memory and executable on the processor, and the screen display adjustment program, when executed by the processor, implements the steps of the screen display adjustment method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a screen display adjustment program, and the screen display adjustment program, when executed by a processor, implements the steps of the screen display adjustment method according to any one of claims 1-4.

Images