Image display method and device, electronic equipment and storage medium
By using motion parameters and depth information in a head-mounted device to determine whether to process images, and directly display or processed images, the image display delay problem is solved, efficiency is improved and dizziness is reduced, and it is suitable for image display of extended real-life devices .
Patent Information
- Application Number
- CN202411051819.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-22
AI Technical Summary
When using head-mounted extended reality devices, image display delay causes the user to feel dizzy. The prior art requires the acquisition and processing of each frame of images when the user's head is moved, resulting in a longer time to display the image.
By acquiring the motion parameters of the head-mounted device, the depth information of the first image and the depth information of the second image, it is determined whether the first image is processed, and the processed image is directly displayed or displayed, reducing the flow from motion to image display .
While ensuring image quality, image display delay is reduced, image display efficiency is improved, user dizziness is avoided, and it is suitable for 3D scene reconstruction and mode switching scenes.
Smart Images

Figure CN119002853A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of electronic equipment, and specifically relates to an image display method, device, electronic equipment and storage medium. Background Art
[0002] Extended Reality (XR) includes Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR) and other forms. It combines reality and virtuality through computer technology to bring users an "immersive feeling" of seamless transition between the virtual world and the real world.
[0003] However, when using a head-mounted device with an extended reality function, since the head-mounted device displays images in real time based on the movement of the user's head, and after the user's head starts moving, the head-mounted device needs to capture and process the images, it takes some time to display the images, which will cause delays in the image display and make the user feel dizzy. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide an image display method, device, electronic device and storage medium, which can reduce the delay of image display and improve the efficiency of image display.
[0005] In a first aspect, an embodiment of the present application provides an image display method, applied to a head-mounted device, the method comprising: acquiring a first image; based on first information, displaying the first image or the processed first image; wherein the first information comprises at least one of the following: motion parameters of the head-mounted device, depth information of the first image, and depth information of a second image, the second image being any frame image before the first image.
[0006] In a second aspect, an embodiment of the present application provides an image display device, the device comprising: an acquisition module and a display module. The acquisition module is used to acquire a first image. The display module is used to display the first image or the processed first image based on first information; wherein the first information includes at least one of the following: motion parameters of a head mounted device, depth information of the first image, and depth information of a second image, the second image being any frame image before the first image.
[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the image display method described in the first aspect are implemented.
[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the image display method described in the first aspect are implemented.
[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the image display method as described in the first aspect.
[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the image display method as described in the first aspect.
[0011] In an embodiment of the present application, a head-mounted device acquires a first image, and displays the first image or the processed first image based on first information, wherein the first information includes at least one of the following: motion parameters of the head-mounted device, depth information of the first image, and depth information of a second image, wherein the second image is any frame image before the first image. In this solution, after acquiring the first image, the head-mounted device determines whether to process the first image based on the motion parameters of the head-mounted device, the first image, and the depth information of any frame image before the first image, and then displays the first image or the processed first image. Compared with the related art in which each frame image is processed before being displayed, this solution reduces the process from motion to image display while ensuring the image quality of some images, thereby reducing the delay in image display, improving the efficiency of image display, and avoiding dizziness in users. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 This is one of the flowcharts of the image display method provided in the embodiment of the present application;
[0013] Figure 2 This is the second flow chart of the image display method provided in the embodiment of the present application;
[0014] Figure 3 This is the third flow chart of the image display method provided in the embodiment of the present application;
[0015] Figure 4 This is the fourth flow chart of the image display method provided in the embodiment of the present application;
[0016] Figure 5 This is the fifth flow chart of the image display method provided in the embodiment of the present application;
[0017] Figure 6 This is one of the processing diagrams of the image display method provided in the embodiment of the present application;
[0018] Figure 7 This is the second schematic diagram of the processing process applied to the image display method provided in the embodiment of the present application;
[0019] Figure 8 It is one of the structural schematic diagrams of the image display device provided in the embodiment of the present application;
[0020] Fig. 9 This is the second structural schematic diagram of the image display device provided in the embodiment of the present application;
[0021] Fig.10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0022] Fig.11 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0023] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0024] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0025] The terms "at least one (item)", "at least one of" and the like in the specification and claims of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".
[0026] The image display method provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0027] The image display method provided in the embodiment of the present application can be applied to scenarios in which the head-mounted device performs scene reconstruction, scene switching, or mode switching during the process in which a user uses a head-mounted device.
[0028] For example, when a user just turns on the head-mounted device or moves the head-mounted device, in existing solutions, the head-mounted device needs to first capture an image, then process the image, and then display the processed image, which makes the motion-to-photon (MTP) time longer, resulting in a delay in image display, causing the user to feel dizzy.
[0029] To this end, an embodiment of the present application provides an image display method, which acquires a first image, and displays the first image or the processed first image based on first information, wherein the first information includes at least one of the following: motion parameters of a head-mounted device, depth information of the first image, and depth information of a second image, wherein the second image is any frame image before the first image. In this solution, after acquiring the first image, the head-mounted device determines whether to process the first image based on the motion parameters of the head-mounted device, the first image, and the depth information of any frame image before the first image, and then displays the first image or the processed first image. Compared with the prior art in which each frame image is processed before being displayed, this solution reduces the process from motion to image display, thereby reducing the delay in image display, improving the efficiency of image display, and avoiding dizziness in users.
[0030] The execution subject of the image display method provided in the embodiment of the present application may be an image display device. Exemplarily, the image display method device may be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device in the embodiment of the present application may be a head-mounted device. The image display method provided in the embodiment of the present application will be described exemplarily below by taking the execution subject as a head-mounted device as an example.
[0031] Figure 1 is a flow chart of an image display method provided in an embodiment of the present application, such as Figure 1 As shown, the image display method provided in the embodiment of the present application may include the following steps 101 and 102.
[0032] Step 101: A head mounted device acquires a first image.
[0033] In the embodiment of the present application, the head-mounted device may be an electronic device with XR functions. For example, the head-mounted device may be AR glasses, MR virtual reality equipment, a cycling helmet display device, a head-mounted head-up display (HUD) device, etc.
[0034] In the embodiment of the present application, the above-mentioned head-mounted device uses video see through (VST) technology to superimpose virtual images on real-time video streams in the real world, so that users can view augmented reality content through the head-mounted device.
[0035] In an embodiment of the present application, the head mounted device is provided with an image acquisition unit for acquiring images.
[0036] In the embodiment of the present application, the above-mentioned image acquisition unit may include at least one of the following: a color image sensor, a spatial composition sensor, and a depth image sensor.
[0037] In the embodiment of the present application, the color image sensor is used to collect color images. For example, the image collected by the color image sensor may be an image in RGB format or YUV format or any image format.
[0038] In the embodiment of the present application, the above-mentioned spatial composition sensor is used to collect images with a wider range than that collected by the color image sensor, and the collected images can be color images or black and white images.
[0039] In the embodiment of the present application, the above-mentioned depth sensor is used to collect depth information of the image.
[0040] For example, the head mounted device may be equipped with a color image sensor, a spatial composition sensor and a depth sensor, so that the head mounted device can collect black and white images, color images and depth information of the images, so that the collected images are more informative.
[0041] For example, the head mounted device may be configured with two color image sensors, four spatial composition sensors and a depth sensor. Furthermore, the spatial composition sensor may be a fisheye camera, the two color image sensors may be color cameras, and the two color image sensors may be configured at positions corresponding to the two eyes of the user.
[0042] It should be noted that, in actual use, a head-mounted device whose image acquisition unit includes different sensors can be selected for use according to the application scenario.
[0043] For example, in a scenario where depth information is not required, a head mounted device configured with a color image sensor, or a spatial composition sensor, or a color image sensor and a spatial composition sensor is selected. In a scenario where color information is not required, a head mounted device configured with a spatial composition sensor, or a depth sensor, or a spatial composition sensor and a depth sensor is selected. In a scenario where large composition is not required, a head mounted device configured with a color sensor, or a color sensor and a depth sensor is selected.
[0044] In an embodiment of the present application, the head mounted device is further configured with an image processing unit, and the image processing unit is connected to the above-mentioned image acquisition unit.
[0045] Exemplarily, the above-mentioned image processing unit may be an image signal processing (ISP) unit.
[0046] In an embodiment of the present application, the head-mounted device captures a first image through an image acquisition unit, and the image acquisition unit then sends the first image to an image processing unit, which determines whether to perform image processing on the first image, that is, determines whether to display the first image or display the processed first image.
[0047] It should be noted that the first image may be any frame of image acquired by the head-mounted device, that is, each frame of image acquired by the head-mounted device may be displayed by the image display method provided in the embodiment of the present application. Alternatively, the first image may be the N+1th frame of image acquired by the head-mounted device, that is, starting from the N+1th frame of image acquired by the head-mounted device, the image display method provided in the embodiment of the present application is used for display.
[0048] It should be noted that N is an integer greater than 1, and N can be set or adjusted according to actual needs, which is not limited in the embodiment of the present application. For example, N can be 3.
[0049] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 2 As shown, before the above step 101, the image display method provided in the embodiment of the present application may further include the following step 100.
[0050] Step 100: The head mounted device starts acquiring images and displays each acquired frame of the image until the Nth frame of the image is acquired and displayed.
[0051] In the embodiment of the present application, the head-mounted device starts from the first frame image, and displays one frame image each time it captures one frame image until the Nth frame image is acquired and displayed, and then starts from the acquired N+1th frame image and displays it using the image display method provided in the embodiment of the present application.
[0052] In the embodiment of the present application, the head mounted device is also configured with a display unit, which may be a display screen.
[0053] In the embodiment of the present application, when the head mounted device is just turned on or the head mounted device displays a new scene or the head mounted device switches to a new mode to display a scene, the image acquisition unit starts to acquire images in real time, and each time the image acquisition unit acquires a frame of image, it sends the frame of image to the display unit in real time, and the display unit displays the frame of image. For the first N frames of image acquired, the head mounted device displays them frame by frame in this manner.
[0054] In this way, when the head mounted device is just turned on or starts to display a new scene or starts to display a scene in a new mode, the first N frames of images acquired are not processed but directly displayed, which can quickly realize image display and reduce the delay of image display. In addition, directly displaying the first N frames of images acquired can avoid the influence of the unstable situation of the head mounted device when acquiring the first few frames of images on the display form.
[0055] Step 102: The head mounted device displays the first image or the processed first image based on the first information.
[0056] In the embodiment of the present application, the above-mentioned first information may include at least one of the following: a motion parameter of the head mounted device, depth information of the first image, and depth information of the second image.
[0057] In the embodiment of the present application, the second image may be any frame image before the first image.
[0058] Optionally, in an embodiment of the present application, the second image can be determined according to actual needs. Exemplarily, the second image can be the first frame image, the second frame image, or the fourth frame image before the first image. For example, assuming that the first image is the N+1th frame image acquired by the head-mounted device, and the second image is the first frame image before the first image, then the second image is the Nth frame image acquired by the head-mounted device.
[0059] Optionally, in the embodiment of the present application, the second image can be determined according to the frame rate of the image acquisition unit. Furthermore, when the frame rate of the image acquisition unit is high, the second image determined by the head mounted device can be an image acquired at a time point far before the first image. When the frame rate of the image acquisition unit is low, the second image determined by the head mounted device can be an image acquired at a time point near the first image.
[0060] Exemplarily, when the frame rate of the image acquisition unit is 25 frames / second, the head-mounted device can determine that the second image is the second frame image before the first image. When the frame rate of the image acquisition unit is 50 frames / second, the head-mounted device can determine that the second image is the fifth frame image before the first image. When the frame rate of the image acquisition unit is 100 frames / second, the head-mounted device can determine that the second image is the tenth frame image before the first image.
[0061] In the embodiment of the present application, the motion parameters of the head-mounted device are the motion parameters of the head-mounted device during the acquisition process from the second image to the first image.
[0062] In the embodiment of the present application, the above-mentioned motion parameters may include a rotation direction and a rotation angle.
[0063] In the embodiment of the present application, the head mounted device may be configured with an inertial measurement unit (IMU). During the image acquisition process, the inertial measurement unit in the head mounted device may obtain the motion parameters of the head mounted device in real time.
[0064] Exemplarily, the inertial measurement unit may acquire the motion parameters of the head mounted device at a frequency of 1 kHz, that is, the inertial measurement unit acquires the motion parameters of the head mounted device once every 1 ms.
[0065] Exemplarily, the inertial measurement unit may include an angular velocity sensor, which may be a gyroscope or other sensors capable of measuring angular velocity.
[0066] Exemplarily, during the image acquisition process, the gyroscope in the above-mentioned head-mounted device can collect the angular velocity values of the head-mounted device rotating in three directions (x-axis, y-axis, and z-axis) in space in real time, and obtain the rotation direction of the head-mounted device and the rotation angle in each rotation direction, that is, obtain the motion parameters of the head-mounted device.
[0067] In the embodiment of the present application, during the image acquisition process, the head-mounted device can record the acquisition time point of each motion parameter, and record the acquisition time point of each frame of the image. The head-mounted device can determine the motion parameters of the acquisition time point between the acquisition time point of the second image and the acquisition time point of the first image (including the acquisition time point of the first image and the acquisition time point of the second image), and use the above-determined motion parameters as the motion parameters of the head-mounted device during the acquisition process from the second image to the first image.
[0068] Exemplarily, assuming that the acquisition time point of the second image is the 10th ms, the acquisition time point of the first image is the 15th ms, and the head-mounted device acquires motion parameters of the head-mounted device at the 10th ms, 11th ms, 12th ms, 13th ms, 14th ms and 15th ms respectively, the head-mounted device can use the motion parameters acquired at the 10th ms, 11th ms, 12th ms, 13th ms, 14th ms and 15th ms as the motion parameters of the head-mounted device.
[0069] Optionally, in the embodiment of the present application, the frequency of the head mounted device acquiring motion parameters may be greater than or equal to the frequency of acquiring images. In this way, it can be ensured that the motion parameters of the head mounted device acquired by the head mounted device are closer to the actual state of the head mounted device and have higher accuracy.
[0070] In the embodiment of the present application, the head-mounted device may determine whether to display the first image or the processed first image according to the motion parameters of the head-mounted device. Alternatively, the head-mounted device may determine whether to display the first image or the processed first image according to the depth information of the first image and the depth information of the second image. Alternatively, the head-mounted device may determine whether to display the first image or the processed first image according to the motion parameters of the head-mounted device, the depth information of the first image, and the depth information of the second image.
[0071] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 3 As shown, the first information may include motion parameters of the head mounted device, and the motion parameters may include a rotation direction and a rotation angle. In this case, step 102 may be implemented by following step 102a or step 102b.
[0072] Step 102a: When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, the head mounted device displays the processed first image.
[0073] It should be noted that the first threshold can be set and adjusted according to actual needs, and the embodiment of the present application does not limit this. For example, the first threshold can be 5 degrees.
[0074] In an embodiment of the present application, during the acquisition process from the second image to the first image, if the rotation angle of the head-mounted device in each rotation direction is less than the first threshold, indicating that the head-mounted device is basically unchanged during the acquisition process from the second image to the first image, the head-mounted device can process the first image and display the processed first image.
[0075] Exemplarily, in the process of capturing from the second image to the first image, assuming that the rotation angle of the head-mounted device in the x-axis direction is 3 degrees, the rotation angle in the y-axis direction is 0 degrees, and the rotation angle in the z-axis direction is 1 degree, and the first threshold is 5 degrees, it can be determined that the rotation angle of the head-mounted device in each rotation direction is less than 5 degrees. Assuming that the second image is the second frame image before the first image, it means that in the process of capturing three consecutive frames of images, the rotation angle of the head-mounted device in each rotation direction is relatively small. Therefore, the head-mounted device can process the first image and display the processed first image.
[0076] In the embodiment of the present application, the head mounted device processes the first image including but not limited to noise reduction processing, beauty processing, focus processing, etc.
[0077] Exemplarily, the noise reduction processing may be to reduce noise in the first image.
[0078] It should be noted that the above-mentioned noise reduction processing can be implemented by any method that can achieve the image noise reduction effect, and the embodiments of the present application are not limited to this.
[0079] Exemplarily, the beautification processing may be to adjust the first image using preset beautification parameters to obtain an image with better effect.
[0080] Exemplarily, the above-mentioned focusing process may be to adjust the focus of the first image to a specific entity in the first image.
[0081] The specific entity may be an entity preset in the first image, or may be an entity that the user's eyes are gazing at in the first image.
[0082] Exemplarily, the head-mounted device may also be configured with an eye movement sensor for detecting the movement of the user's eyes, and capable of determining the entity that the user's eyes are looking at, that is, determining a specific entity, and the head-mounted device then focuses the first image based on the determined specific entity.
[0083] It is worth noting that when the number of images between the second image and the first image is different, the head mounted device may process the first image differently.
[0084] Optionally, in an embodiment of the present application, if the second image is an image acquired at a time point far before the first image, the number of images included from the second image to the first image is relatively large, indicating that the rotation angle of the head-mounted device in each direction is less than the first threshold value in a relatively long period of time, and the processing of the first image by the head-mounted device may include relatively more operations. If the second image is an image acquired at a time point close to the first image, the number of images included from the second image to the first image is relatively small, indicating that the rotation angle of the head-mounted device in each direction is less than the first threshold value in a relatively short period of time, and the processing of the first image by the head-mounted device may include relatively fewer operations.
[0085] For example, if the second image is the second frame image before the first image, then there are a total of 3 frames of images from the second image to the first image, indicating that the rotation angle of the head-mounted device in each rotation direction is small during the acquisition of the three consecutive frames of images, and the processing of the first image by the head-mounted device may include any one of noise reduction processing, beauty processing, and focus processing. If the second image is the fourth frame image before the first image, then there are a total of 5 frames of images from the second image to the first image, indicating that the rotation angle of the head-mounted device in each direction is small during the acquisition of the five consecutive frames of images, and the processing of the first image by the head-mounted device may include at least two of noise reduction processing, beauty processing, and focus processing.
[0086] Optionally, in an embodiment of the present application, the head-mounted device can pre-set the correspondence between the flag bits and the algorithms, and set the flag bits for regulation in different situations.
[0087] Exemplarily, assuming that the second image is the second frame image before the first image, there are a total of three frames of images from the second image to the first image, indicating that during the acquisition of the three consecutive frames of images, the rotation angle of the head-mounted device in each rotation direction is less than the first threshold. The flag bits adjusted by the head-mounted device include flag bit 1, and flag bit 1 corresponds to noise reduction processing. Therefore, the head-mounted device performs noise reduction processing on the first image.
[0088] Exemplarily, assuming that the second image is the fourth frame image before the first image, there are a total of five frames of images from the second image to the first image, indicating that during the acquisition of five consecutive frames of images, the rotation angle of the head-mounted device in each rotation direction is less than the first threshold. The flag bits adjusted by the head-mounted device include flag bit 1, flag bit 2, and flag bit 3, and flag bit 1 corresponds to noise reduction processing, flag bit 2 corresponds to beauty processing, and flag bit 3 corresponds to focus processing. Therefore, the head-mounted device performs noise reduction processing, beauty processing, and focus processing on the first image.
[0089] In an embodiment of the present application, the image processing unit of the above-mentioned head-mounted device determines that the first image needs to be processed based on the motion parameters of the head-mounted device, and then processes the first image by the image processing unit, and then sends the processed first image to the display unit, which displays the processed first image.
[0090] In this way, since the rotation angle of the head-mounted device in each rotation direction during the acquisition process from the second image to the first image is less than the first threshold, all the images included from the second image to the first image will not basically change. Therefore, when the image before the first image has been displayed, the first image is processed and then displayed. This not only reduces the delay time of image display and improves the image display efficiency while ensuring the image quality of the displayed image, but also the displayed image does not basically change in the user's view. Therefore, even if it is displayed after processing, it will not cause visual delay and can avoid dizziness of the user.
[0091] Optionally, in the embodiment of the present application, the above step 102a can be implemented by the following steps 102a1 and 102a2.
[0092] Step 102a1: When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, the head mounted device processes the first image in a target processing manner, and performs scene recognition on the first image to obtain scene information.
[0093] In the embodiment of the present application, the above-mentioned target processing method may include any one of the following: noise reduction processing, beauty processing, and focus processing.
[0094] In the embodiment of the present application, the scene information is used to represent the scene of the first image. For example, the scene information includes night scene, indoor, outdoor, backlight, cloudy, etc.
[0095] In the embodiment of the present application, the head mounted device processes the first image in a target processing manner through the above-mentioned image processing unit, and performs scene recognition on the first image to obtain scene information.
[0096] Exemplarily, a scene recognition algorithm may be preset in the above-mentioned image processing unit. After the image processing unit processes the first image in a target processing manner, the image processing unit performs scene recognition on the first image through the scene recognition algorithm to obtain scene information.
[0097] For example, the image processing unit performs noise reduction processing on the first image, and performs scene recognition on the first image after the noise reduction processing, and obtains scene information that the scene is a night scene.
[0098] Step 102a2: The head mounted device displays the first image processed by the target processing method based on the display parameters corresponding to the scene information.
[0099] In the embodiment of the present application, the above display parameters are parameters for adjusting the first image when displaying the first image.
[0100] Exemplarily, the above display parameters include but are not limited to brightness, contrast, saturation, color temperature, clarity, exposure, etc.
[0101] It should be noted that the above display parameters may be pre-set and stored in the head mounted device, and the above display parameters correspond one-to-one to the scene information.
[0102] Exemplarily, the night scene corresponds to display parameter a, the backlight corresponds to display parameter b, the cloudy day corresponds to display parameter c, and so on.
[0103] For example, if the scene is a night scene, the display parameters corresponding to the night scene may be brightness compensation parameters; if the scene is backlight, the display parameters corresponding to the backlight may be exposure compensation parameters; if the scene is outdoor, the display parameters corresponding to outdoor may be contrast compensation parameters; if the scene is indoor cloudy, the display parameters corresponding to cloudy may be color temperature compensation parameters.
[0104] In the embodiment of the present application, the image processing unit of the head-mounted device adjusts the first image processed by the target processing method based on the display parameters corresponding to the scene information, and then sends the adjusted first image to the display unit for display.
[0105] In this way, the head-mounted device can display the image with the most appropriate display parameters according to the scene information of the image, thereby improving the quality of the displayed image.
[0106] Step 102b: When the rotation angle of the head mounted device in any rotation direction is greater than or equal to the first threshold, the head mounted device displays the first image, or displays the first image in a display manner of a previous frame of the first image.
[0107] Optionally, in an embodiment of the present application, during the acquisition process from the second image to the first image, if the rotation angle of the head-mounted device in any rotation direction during the acquisition of at least two consecutive frames of images is greater than or equal to a first threshold, indicating that the head-mounted device has moved during the acquisition process from the second image to the first image, the head-mounted device may not process the first image and directly display the first image.
[0108] Exemplarily, assuming that the second image is the first frame image before the first image, there are two frames of images between the second image and the first image, and in the process of the head-mounted device capturing the second image and the first image, the rotation angles in the x-axis direction and the y-axis direction are greater than the first threshold, then the difference between the first image and the second image is large, therefore, the head-mounted device does not process the first image and directly displays the first image.
[0109] Optionally, in an embodiment of the present application, during the acquisition process from the second image to the first image, if the rotation angle of the head-mounted device in any rotation direction is greater than or equal to the first threshold only during the acquisition of a certain frame of image, it may be a false detection or the head-mounted device has just started to prepare for movement, and the first image is not much different from the previous frame of image. Therefore, the head-mounted device can display the first image in the display mode of the previous frame of image of the first image.
[0110] Exemplarily, assuming that the second image is the second frame image before the first image, there are a total of three frames of images from the second image to the first image, and the rotation angle of the head-mounted device in the x-axis direction and the y-axis direction is greater than the first threshold only during the process of capturing the first image, and the angle in each rotation direction when capturing the other two frames of images is less than the first threshold. It may be that the head-mounted device has just started to prepare for movement, and the first image is not much different from the previous frame image. Therefore, the head-mounted device can display the first image in the display mode of the previous frame image of the first image.
[0111] Thus, in the process of acquiring the first image from the second image, if the rotation angle of the head-mounted device in any rotation direction is greater than the first threshold value in the process of acquiring at least two consecutive frames of images, the difference between the first image and the previous frame of image may be relatively large. Therefore, when the image before the first image has been displayed, the first image is not processed and the first image is directly displayed, which can reduce the delay of image display, improve the efficiency of image display, and avoid dizziness of the user. If the rotation angle of the head-mounted device in any rotation direction is greater than the first threshold value only in the process of acquiring a single frame of image, it may be a false detection or the head-mounted device has just started to prepare for movement, and the difference between the first image and the previous frame of image may not be large. Therefore, the head-mounted device can display the first image in the display mode of the previous frame of image of the first image, so that the image display is relatively smooth, which can improve the efficiency of image display while ensuring the image display quality.
[0112] Furthermore, the above-mentioned inertial measurement unit may also include an acceleration sensor.
[0113] Exemplarily, during the image acquisition process, the gyroscope can obtain the angular velocity values of the head-mounted device in three directions, and the acceleration sensor can obtain the acceleration values of the head-mounted device in three directions. The rotation angle in each rotation direction can be determined according to the angular velocity values in each direction, and when the acceleration values in each direction are negative, it indicates that the head-mounted device tends to be stationary. Therefore, when the rotation angle of the head-mounted device in any direction is greater than or equal to the first threshold, it can also be determined whether the acceleration value in any direction is negative. If so, the head-mounted device can display the processed first image; if not, the head-mounted device can display the first image.
[0114] The second implementation method: Combination Figure 1 ,like Figure 4 As shown, the first information may include depth information of the first image and depth information of the second image. In this case, step 102 may be implemented by following step 102e or step 102f.
[0115] Step 102e: When the depth information of the first image matches the depth information of the second image, the head mounted device displays the processed first image.
[0116] In the embodiment of the present application, the depth information is the distance between each pixel in the image and the image acquisition unit that acquires the image.
[0117] It should be noted that, for the description of the second image, reference may be made to the relevant description of the above embodiment, which will not be repeated in this embodiment.
[0118] In an embodiment of the present application, when a total of two frames of images are included from the second image to the first image, the head-mounted device compares the depth information of the first image with the depth information of the second image to see whether they match. Then, the depth information of the first image matches the depth information of the second image, which means that the similarity between the depth information of the first image and the depth information of the second image is greater than a second threshold.
[0119] In an embodiment of the present application, when there are more than two frames of images from the second image to the first image, the head-mounted device divides every two frames of images into a group according to the acquisition order, and compares whether the depth information of the two frames of images in each group matches. Then, the depth information of the first image matches the depth information of the second image, which means that the depth information of the two frames of images in each group matches, that is, the similarity between the depth information of the two images in each group is greater than the second threshold.
[0120] It should be noted that the second threshold can be set and adjusted according to actual needs, and the embodiment of the present application does not limit this. For example, the second threshold can be 95%.
[0121] Exemplarily, taking the second threshold of 95% as an example, assuming that the second image is the previous frame image of the first image, and both the first image and the second image include 100,000 pixels, the head-mounted device compares the depth value of each pixel in the first image with the depth value of the pixel at the same position in the second image. Assuming that 98,000 pixels have the same depth values, the head-mounted device can determine that the similarity between the depth information of the first image and the depth information of the second image is 98%, which is greater than the second threshold of 95%, that is, it is determined that the depth information of the first image matches the depth information of the second image. The head-mounted device then processes the first image and displays the processed first image.
[0122] Step 102f: When the depth information of the first image does not match the depth information of the second image, the head mounted device displays the first image, or displays the first image in a display manner of a previous frame of the first image.
[0123] In an embodiment of the present application, when a total of two frames of images are included from the second image to the first image, the depth information of the first image and the depth information of the second image do not match each other, which means that the similarity between the depth information of the first image and the depth information of the second image is less than or equal to a second threshold.
[0124] Exemplarily, taking the second threshold of 95% as an example, assuming that the second image is the previous frame image of the first image, and both the first image and the second image include 100,000 pixels, the head-mounted device compares the depth value of each pixel in the first image with the depth value of the pixel at the same position in the second image. Assuming that 90,000 pixels have the same depth values, the head-mounted device can determine that the similarity between the depth information of the first image and the depth information of the second image is 90%, which is less than the second threshold of 95%, that is, it is determined that the depth information of the first image matches the depth information of the second image. The head-mounted device does not process the first image and directly displays the first image.
[0125] In the embodiment of the present application, when the second image to the first image includes more than two frames of images, the mismatch between the depth information of the first image and the depth information of the second image means that the depth information of at least one group of two frames of images does not match.
[0126] Optionally, in the embodiment of the present application, the depth information of the two frames of images of the at least one group does not match includes: the depth information of the two images of two consecutive groups does not match, that is, the similarity between the depth information of the two images of the two consecutive groups is less than or equal to the second threshold. In this case, the head mounted device directly displays the first image.
[0127] Optionally, in the embodiment of the present application, the depth information of the two frames of images of the at least one group does not match includes: the depth information of the two images of any group does not match, that is, the similarity between the depth information of the two images of any group is less than or equal to a second threshold. In this case, the head mounted device displays the first image in a display manner of a previous frame of the first image.
[0128] In this way, the head-mounted device determines how to display the first image by comparing the depth information of the image, which can also reduce the delay time of image display, improve image display efficiency, and avoid user dizziness. In addition, there is no need to collect IMU data for processing, which can also reduce the data processing volume of the head-mounted device.
[0129] It should be noted that the above embodiment records how to directly determine how to display the first image based on the motion parameters of the head-mounted device, or the depth information of the first image and the depth information of the second image. In other embodiments of the present application, the head-mounted device may first determine the state of the head-mounted device based on the motion parameters of the head-mounted device, or the depth information of the first image and the depth information of the second image, and then determine how to display the first image based on the state of the head-mounted device.
[0130] Optionally, in the embodiment of the present application, the above step 102 can also be implemented by the following steps 201 and 202.
[0131] Step 201: The head mounted device determines a state of the head mounted device based on first information.
[0132] In the embodiment of the present application, the state of the head-mounted device is the state of the head-mounted device during the acquisition process from the second image to the first image.
[0133] In the embodiment of the present application, the state of the head mounted device may include a stationary state or a moving state.
[0134] Exemplarily, the state of the head mounted device is a static state, which can be understood as the head mounted device substantially not moving during the acquisition process from the second image to the first image. For example, the rotation angle of the head mounted device in each rotation direction is less than the first threshold.
[0135] Exemplarily, the state of the head mounted device is a moving state, which can be understood as the head mounted device moving during the acquisition process from the second image to the first image. For example, the rotation angle of the head mounted device in any rotation direction is greater than or equal to the first threshold.
[0136] In the embodiment of the present application, the above step 201 may include the following three implementation methods.
[0137] A first implementation manner: when the first information includes motion parameters of the head mounted device, the head mounted device may determine the state of the head mounted device according to the motion parameters of the head mounted device.
[0138] Optionally, in an embodiment of the present application, the above-mentioned motion parameters include a rotation direction and a rotation angle. If the rotation angle of the head-mounted device in each rotation direction is less than a first threshold value, the head-mounted device determines that the state of the head-mounted device is a stationary state during the process from capturing the second image to capturing the first image. If the rotation angle of the head-mounted device in any rotation direction is greater than or equal to the first threshold value, the head-mounted device determines that the state of the head-mounted device is a moving state during the process from capturing the second image to capturing the first image.
[0139] For example, taking the rotation angle of the head-mounted device in the x-axis direction as 3 degrees, the rotation angle in the y-axis direction as 0 degrees, the rotation angle in the z-axis direction as 1 degree, and the first threshold as 5 degrees, it can be determined that the rotation angle of the head-mounted device in each rotation direction is less than 5 degrees, then the head-mounted device can determine that the state of the head-mounted device is a stationary state.
[0140] For example, taking the rotation angle of the head-mounted device in the x-axis direction as 10 degrees, the rotation angle in the y-axis direction as 0 degrees, the rotation angle in the z-axis direction as 45 degrees, and the first threshold as 5 degrees, it can be determined that the rotation angles of the head-mounted device in the x-axis direction and the z-axis direction are greater than 5 degrees, then the head-mounted device can determine that the state of the head-mounted device is a moving state.
[0141] In this way, the head-mounted device determines whether the head-mounted device is in a stationary state or a moving state through the motion parameters obtained during the image acquisition process, and can correspond the state of the head-mounted device to the acquired image so as to flexibly determine the image display mode according to the state.
[0142] A second implementation manner: when the first information includes depth information of the first image and depth information of the second image, the head-mounted device may determine the state of the head-mounted device according to the depth information of the first image and the depth information of the second image.
[0143] Optionally, in the embodiment of the present application, when there are two frames of images from the second image to the first image, if the depth information of the first image matches the depth information of the second image, the head-mounted device determines that the state of the head-mounted device is a stationary state during the process from capturing the second image to capturing the first image. Alternatively, if the depth information of the first image does not match the depth information of the second image, the head-mounted device determines that the state of the head-mounted device is a moving state during the process from capturing the second image to capturing the first image.
[0144] Optionally, in the embodiment of the present application, when there are more than two frames of images from the second image to the first image, the head-mounted device groups every two frames of images according to the acquisition order, and if the depth information of the two frames of images in each group matches, the head-mounted device determines that the state of the head-mounted device is a stationary state during the process from acquiring the second image to acquiring the first image. Alternatively, if the depth information of the two frames of images in at least one group does not match, the head-mounted device determines that the state of the head-mounted device is a moving state during the process from acquiring the second image to acquiring the first image.
[0145] In this way, the head-mounted device can also determine the state of the head-mounted device based on the depth information of the collected image. There is no need to collect and process IMU data, which can reduce the data processing volume of the head-mounted device and facilitate the subsequent display of images in different ways according to the state of the head-mounted device.
[0146] The third implementation method: The head-mounted device can combine motion parameters and image depth information to determine the state of the head-mounted device.
[0147] Exemplarily, when the state of the head mounted device determined according to the motion parameters is the same as the state of the head mounted device determined according to the depth information of the first image and the depth information of the second image, the state of the head mounted device is directly determined. When the state of the head mounted device determined according to the motion parameters is different from the state of the head mounted device determined according to the depth information of the first image and the depth information of the second image, the state of the head mounted device determined according to the motion parameters shall prevail.
[0148] In this way, the accuracy of the determined state of the head mounted device can be improved, thereby improving the accuracy of the ultimately determined display mode.
[0149] Step 202: The head mounted device displays the first image or the processed first image based on the state of the head mounted device.
[0150] In the embodiment of the present application, the first image may be displayed in different display modes when the state of the head mounted device is different.
[0151] Optionally, in the embodiment of the present application, the above step 202 can be implemented by the following steps 202a and 202b.
[0152] Step 202a: When the head mounted device is in a stationary state, the head mounted device displays the processed first image.
[0153] It should be noted that the implementation process of the above step 202a can refer to the description of the relevant steps of the head mounted device displaying the processed first image in the above embodiment, and the embodiment of the present application will not be repeated here.
[0154] Step 202b: When the state of the head mounted device is a moving state, the head mounted device displays the first image, or displays the first image in a display manner of a previous frame image of the first image.
[0155] It should be noted that the implementation process of the above-mentioned step 202b can refer to the description of the relevant steps of displaying the first image by the head-mounted device in the above-mentioned embodiment, or displaying the first image using the display method of the previous frame image of the first image, and the embodiments of the present application will not go into details about this.
[0156] In this way, the head-mounted device first determines whether the head-mounted device is moving according to the first information, and then displays the processed first image when the head-mounted device is stationary. When the head-mounted device is moving, the first image is directly displayed. This not only enables flexible display of images, but also reduces the delay in image display and improves the efficiency of image display compared to the related art in which each frame of the image is processed before display, and can avoid dizziness in users.
[0157] In the image display method provided by the embodiment of the present application, when the head-mounted device acquires the first N frames of images, it does not need to process them. Each frame of image acquired directly displays the frame of image. When the N+1th frame of image (i.e., the above-mentioned first image) is acquired, it is determined whether to process the first image according to the motion parameters of the head-mounted device, or the image information of the first image and the image information of the second image, and then the first image or the processed first image is displayed. In this way, compared with the related art in which each frame of image is processed before display, this solution reduces the process from motion to image display while ensuring the image quality of some images, shortens the MTP to within 20ms, reduces the delay of image display, improves the efficiency of image display, and can avoid dizziness of users. In addition, the image display method of this solution is also applicable to 3D scene reconstruction. For example, when the head-mounted device is moved or turned on, its 3D scene reconstruction is accompanied by the increase of preview image data and continuously enriches the reconstruction details of the 3D scene. This solution is also applicable to mode switching, scene switching, etc., and no processing is performed when the scene is displayed at the beginning, and the processing operation is slowly increased to simulate the human eye from blur to clarity, while reducing the delay of image display.
[0158] Figure 5 is a flow chart of an image display method provided in an embodiment of the present application. The method may include steps 11 to 17.
[0159] Step 11: The head mounted device starts acquiring images and displays each acquired frame of the image.
[0160] Step 12: When the N+1th frame image is acquired, the head mounted device acquires the motion parameters of the head mounted device during the acquisition process from the second image to the N+1th frame image.
[0161] In the embodiment of the present application, the above-mentioned motion parameters include rotation direction and rotation angle.
[0162] It should be noted that the above-mentioned N+1th frame image is the first image in the above-mentioned embodiment.
[0163] Step 13: If the rotation angle of the head mounted device in each rotation direction is less than the first threshold, the head mounted device processes the N+1th frame image and displays the processed N+1th frame image.
[0164] Step 14: If the rotation angle of the head mounted device in any rotation direction is greater than or equal to the first threshold, the head mounted device directly displays the N+1th frame image, or displays the N+1th frame image in the display mode of the Nth frame image.
[0165] For example, Figure 6 Schematic diagram of the processing process of the image display method provided in the embodiment of the present application. Figure 6 In the embodiment, the head-mounted device collects images, and according to the motion parameters of the head-mounted device (including the rotation direction and rotation angle), a control mechanism is determined, wherein the rotation angle of the head-mounted device in each rotation direction is less than 5 degrees during the collection of 3 consecutive frames of images (i.e., the images between the second image and the first image), the rotation angle of the head-mounted device in each rotation direction is less than 5 degrees during the collection of 5 consecutive frames of images, and the rotation angle of the head-mounted device in each direction is greater than or equal to 5 degrees during the collection of 2 consecutive frames of images. The above control mechanism is applicable to each frame of the image, and is used to determine whether to process the image and what kind of processing to perform. For example, during the acquisition of three consecutive frames of images, if the rotation angle of the head-mounted device in each rotation direction is less than 5 degrees, the last frame of the three consecutive frames (i.e., the first image mentioned above) is subjected to noise reduction and beauty processing, and then the processed last frame is displayed; during the acquisition of five consecutive frames of images, if the rotation angle of the head-mounted device in each direction is less than 5 degrees, the last frame of the five consecutive frames is subjected to noise reduction, beauty processing, focus processing, and scene recognition processing, and then the processed last frame is displayed; during the acquisition of two consecutive frames of images, if the rotation angle of the head-mounted device in each direction is greater than or equal to 5 degrees, no processing is performed on the last frame of the two consecutive frames, and the last frame is directly displayed.
[0166] For example, Figure 7 It is a schematic diagram of a processing process applied to an image display method provided according to an embodiment of the present application.
[0167] exist Figure 7 In the embodiment, the image acquisition unit of the head mounted device starts to acquire images, and displays the first frame of image when the first frame of image is acquired, displays the second frame of image when the second frame of image is acquired, and displays the third frame of image when the third frame of image is acquired, that is, the value of N in the above embodiment is 3.
[0168] When the 4th frame image (i.e., the first image mentioned above) is captured, the head-mounted device can judge, based on the 4th frame image and the previous two frames of images, i.e., the second image mentioned above is the second frame image before the first image, that during the capture of three consecutive frames of images (i.e., the capture process from the second image to the first image mentioned above), the rotation angle of the head-mounted device in each rotation direction is small, that is, the head-mounted device is in a stationary state. Then the head-mounted device can perform noise reduction and beauty processing on the 4th frame image, and display the processed 4th frame image.
[0169] When the 5th frame image (i.e., the first image mentioned above) is captured, the head-mounted device can judge, based on the 5th frame image and the previous two frames of images, i.e., the second image mentioned above is the second frame image before the first image, that during the capture of three consecutive frames of images (i.e., the capture process from the second image to the first image mentioned above), the rotation angle of the head-mounted device in each rotation direction is small, that is, the head-mounted device is in a stationary state. Then the head-mounted device can perform noise reduction and beauty processing on the 5th frame image, and display the processed 5th frame image.
[0170] When the 6th frame image (i.e., the first image mentioned above) is captured, the head-mounted device can judge that the rotation angle of the head-mounted device in each rotation direction is small during the capture of 5 consecutive frames of images (i.e., the capture process from the second image to the first image mentioned above), that is, the head-mounted device is in a stationary state, then the head-mounted device can perform noise reduction, beautification, focus and scene recognition processing on the 6th frame image, and display the processed 6th frame image.
[0171] When the 7th frame image (i.e., the first image mentioned above) is captured, the head-mounted device can judge that the rotation angle of the head-mounted device in any rotation direction is larger during the capture of only one frame image among two consecutive frames of images based on the 7th frame image and the previous frame image, i.e., the second image mentioned above is the first frame image before the first image, that is, the head-mounted device is in motion. Then, the head-mounted device can perform noise reduction, beautification, focus and scene recognition processing on the 7th frame image according to the display mode of the 6th frame image, and display the processed 7th frame image.
[0172] When the 8th frame image (i.e., the first image mentioned above) is collected, the head-mounted device can judge that during the collection of two consecutive frames of images, the rotation angle of the head-mounted device in any rotation direction during the collection of each frame image is large, i.e., the head-mounted device is in motion, based on the 8th frame image and the previous frame image, i.e., the second image is the first frame image before the first image. Then, the head-mounted device does not process the 8th frame image, and directly displays the original image of the 8th frame image. By analogy, displaying images through the image display method provided by the present application can reduce the delay of image display and improve the efficiency of image display.
[0173] Step 15: When the N+1th frame image is obtained, the head mounted device compares the depth information of the second image with the depth information of the N+1th frame image.
[0174] Step 16: When the depth information of the N+1th frame image matches the depth information of the second image, the head mounted device displays the processed N+1th frame image.
[0175] Step 17: When the depth information of the N+1th frame image does not match the depth information of the second image, the head mounted device directly displays the N+1th frame image, or displays the N+1th frame image in the display mode of the Nth frame image.
[0176] It should be noted that there is no order in which the above steps 12 and 15 are executed. After executing step 11, steps 12 to 14 may be executed, or steps 15 to 17 may be executed. This embodiment of the present application does not limit this.
[0177] It should be noted that the specific implementation process of the above steps 11 to 17 can refer to the relevant description of the above embodiment. To avoid repetition, the embodiment of the present application will not be repeated here.
[0178] It should be noted that the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or any two or more can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application do not impose any restrictions on this.
[0179] The image display method provided in the embodiment of the present application can be executed by an image display device. In the embodiment of the present application, an image display device executing the image display method is taken as an example to illustrate the image display device provided in the embodiment of the present application.
[0180] Figure 8 2 is a schematic diagram of the structure of an image display device provided in an embodiment of the present application. The image display device includes an acquisition module 21 and a display module 22.
[0181] The acquisition module 21 is used to acquire a first image.
[0182] The display module 22 is used to display the first image acquired by the acquisition module 21 or the processed first image based on the first information.
[0183] The first information includes at least one of the following: motion parameters of the head mounted device, depth information of the first image and depth information of the second image, where the second image is any frame image before the first image.
[0184] Optionally, in an embodiment of the present application, the first information includes motion parameters of the head mounted device, and the motion parameters include a rotation direction and a rotation angle;
[0185] The display module 22 is specifically used for:
[0186] When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, displaying the processed first image; or,
[0187] When the rotation angle of the head mounted device in any rotation direction is greater than or equal to the first threshold, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
[0188] Optionally, in the embodiment of the present application, the display module 22 is specifically used for:
[0189] When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, the first image is processed using a target processing method, and scene recognition is performed on the first image to obtain scene information; based on display parameters corresponding to the scene information, the first image processed by the target processing method is displayed.
[0190] Optionally, in the embodiment of the present application, the first information includes depth information of the first image and depth information of the second image;
[0191] The display module 22 is specifically used for:
[0192] In a case where the depth information of the first image matches the depth information of the second image, displaying the processed first image; or,
[0193] When the depth information of the first image does not match the depth information of the second image, the first image is displayed, or the first image is displayed in a display manner of a previous frame of the first image.
[0194] Optionally, in the embodiment of the present application, combined with Figure 8 ,like Fig. 9 As shown, the device also includes a determination module 23;
[0195] The determination module 23 is used to determine the state of the head mounted device based on the first information, where the state is the state of the head mounted device during the acquisition process from the second image to the first image, and the state includes a stationary state or a moving state.
[0196] The display module 22 is specifically configured to display the first image or the processed first image based on the state of the head mounted device.
[0197] Optionally, in the embodiment of the present application, the display module 22 is specifically used for:
[0198] When the head mounted device is in a stationary state, displaying the processed first image; or,
[0199] When the head mounted device is in a motion state, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
[0200] Optionally, in the embodiment of the present application, the acquisition module 21 is also used to start acquiring images.
[0201] The display module 22 is further used to display each frame of image acquired by the acquisition module 21 until the Nth frame of image is acquired and displayed; wherein the first image is the N+1th frame of image acquired by the acquisition module 21 .
[0202] An embodiment of the present application provides an image display device, which, after acquiring a first image, determines whether to process the first image based on motion parameters of the image display device, the first image, and depth information of any frame image before the first image, and then displays the first image or the processed first image. Compared with the related art that processes each frame image before displaying it, this reduces the process from motion to image display while ensuring the image quality of some images, thereby reducing the delay in image display, improving the efficiency of image display, and avoiding dizziness in users.
[0203] The image display device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device, an augmented reality / virtual reality device, a robot, a wearable device, a super mobile personal computer, a netbook or a personal digital assistant, etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the above-mentioned wearable device can be a head-mounted device, which is not specifically limited in the embodiment of the present application.
[0204] The image display device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0205] The image display device provided in the embodiment of the present application can implement each process implemented by each embodiment of the above-mentioned image display method, and will not be described again here to avoid repetition.
[0206] Alternatively, if Fig.10 As shown, an embodiment of the present application further provides an electronic device 900, including a processor 901 and a memory 902, wherein the memory 902 stores programs or instructions that can be executed on the processor 901, and when the program or instructions are executed by the processor 901, the various steps of the above-mentioned image display method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.
[0207] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0208] Fig.11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0209] The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010 and other components.
[0210] Those skilled in the art will appreciate that the electronic device 1000 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1010 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0211] Wherein, the processor 1010 is used to acquire a first image.
[0212] The display unit 1006 is configured to display the first image acquired by the processor 1010 or the processed first image based on the first information;
[0213] The first information includes at least one of the following: motion parameters of the head mounted device, depth information of the first image and depth information of the second image, where the second image is any frame image before the first image.
[0214] Optionally, in an embodiment of the present application, the first information includes motion parameters of the head mounted device, and the motion parameters include a rotation direction and a rotation angle.
[0215] The display unit 1006 is specifically used for:
[0216] When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, displaying the processed first image; or,
[0217] When the rotation angle of the head mounted device in any rotation direction is greater than or equal to the first threshold, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
[0218] Optionally, in the embodiment of the present application, the display unit 1006 is specifically used for:
[0219] When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, the first image is processed using a target processing method, and scene recognition is performed on the first image to obtain scene information; based on display parameters corresponding to the scene information, the first image processed by the target processing method is displayed.
[0220] Optionally, in an embodiment of the present application, the first information includes depth information of the first image and depth information of the second image.
[0221] The display unit 1006 is specifically used for:
[0222] In a case where the depth information of the first image matches the depth information of the second image, displaying the processed first image; or,
[0223] When the depth information of the first image does not match the depth information of the second image, the first image is displayed, or the first image is displayed in a display manner of a previous frame of the first image.
[0224] Optionally, in the embodiment of the present application, the processor 1010 is further configured to:
[0225] Based on the first information, a state of the head mounted device is determined, where the state is the state of the head mounted device during the acquisition process from the second image to the first image, and the state includes a stationary state or a moving state.
[0226] The display unit 1006 is specifically used for:
[0227] Based on the state of the head mounted device, the first image or the processed first image is displayed.
[0228] Optionally, in the embodiment of the present application, the display unit 1006 is specifically used for:
[0229] When the head mounted device is in a stationary state, displaying the processed first image; or,
[0230] When the head mounted device is in a motion state, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
[0231] Optionally, in an embodiment of the present application, the processor 1010 is also used to start acquiring images.
[0232] The display unit 1006 is further used to display each acquired frame of image until the Nth frame of image is acquired and displayed; wherein the first image is the acquired N+1th frame of image.
[0233] An embodiment of the present application provides an electronic device. After acquiring a first image, the electronic device determines whether to process the first image based on motion parameters of the electronic device, the first image, and depth information of any frame image before the first image, and then displays the first image or the processed first image. Compared with the related art that processes each frame image before displaying it, this reduces the process from motion to image display while ensuring the image quality of some images, thereby reducing the delay in image display, improving the efficiency of image display, and avoiding dizziness in users.
[0234] It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1007 includes a touch panel 10071 and at least one of other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0235] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0236] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1010.
[0237] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image display method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0238] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0239] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image display method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0240] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0241] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image display method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0242] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0243] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0244] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image display method, applied to a head mounted device, It is characterized in that The method comprises: acquiring a first image; Based on the first information, display the first image or the processed first image; The first information includes at least one of the following: motion parameters of the head mounted device, depth information of the first image and depth information of a second image, wherein the second image is any frame image before the first image.
2. The method according to claim 1, It is characterized in that The first information includes motion parameters of the head mounted device, wherein the motion parameters include a rotation direction and a rotation angle; The displaying the first image or the processed first image based on the first information includes: When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, displaying the processed first image; or, When the rotation angle of the head mounted device in any rotation direction is greater than or equal to the first threshold, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
3. The method according to claim 2, It is characterized in that The method of displaying the processed first image when the rotation angle of the head mounted device in each rotation direction is less than a first threshold value includes: When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, processing the first image in a target processing manner, and performing scene recognition on the first image to obtain scene information; Based on the display parameters corresponding to the scene information, the first image processed by the target processing method is displayed.
4. The method according to claim 1, It is characterized in that The first information includes depth information of the first image and depth information of the second image; The displaying the first image or the processed first image based on the first information includes: In a case where the depth information of the first image matches the depth information of the second image, displaying the processed first image; or, When the depth information of the first image does not match the depth information of the second image, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
5. The method according to any one of claims 1 to 4, It is characterized in that The displaying the first image or the processed first image based on the first information includes: Based on the first information, determine a state of the head mounted device, where the state is a state of the head mounted device during a process of acquiring from the second image to the first image, and the state includes a stationary state or a moving state; Based on the state of the head mounted device, the first image or the processed first image is displayed.
6. The method according to claim 5, It is characterized in that The displaying the first image or the processed first image based on the state of the head mounted device includes: When the head mounted device is in a stationary state, displaying the processed first image; or, When the head mounted device is in a motion state, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
7. The method according to any one of claims 1 to 4, It is characterized in that Before acquiring the first image, the method further includes: Start acquiring images and display each acquired frame of images until the Nth frame of images is acquired and displayed, where N is an integer greater than 1; The first image is the acquired N+1th frame image.
8. An image display device, It is characterized in that The device comprises: an acquisition module and a display module; The acquisition module is used to acquire a first image; The display module is used to display the first image acquired by the acquisition module or the first image after processing based on the first information; The first information includes at least one of the following: motion parameters of a head mounted device, depth information of the first image, and depth information of a second image, wherein the second image is any frame image before the first image.
9. The device according to claim 8, It is characterized in that The first information includes motion parameters of the head mounted device, wherein the motion parameters include a rotation direction and a rotation angle; The display module is specifically used for: When the rotation angle of the head mounted device in each rotation direction is less than a first threshold, displaying the processed first image; or, When the rotation angle of the head mounted device in any rotation direction is greater than or equal to the first threshold, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
10. The device according to claim 8, It is characterized in that The first information includes depth information of the first image and depth information of the second image; The display module is specifically used for: In a case where the depth information of the first image matches the depth information of the second image, displaying the processed first image; or, When the depth information of the first image does not match the depth information of the second image, the first image is displayed, or the first image is displayed in a display manner of a previous frame image of the first image.
11. An electronic device, It is characterized in that The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image display method according to any one of claims 1 to 7 are implemented.
12. A readable storage medium, It is characterized in that The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the image display method according to any one of claims 1 to 7 are implemented.
