Frame rate adjustment method, system, device and storage medium for augmented reality device
By dynamically adjusting the refresh frame rate of augmented reality devices and updating the display content according to changes in scene information, the problem of high power consumption of existing devices is solved and more efficient energy efficiency performance is achieved.
Patent Information
- Application Number
- CN202110816562.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-20
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-07-20
AI Technical Summary
Existing augmented reality devices need to constantly refresh content on the display side, resulting in increased power consumption and unable to effectively reduce system power consumption.
By obtaining scene information of augmented reality device, including external environment images, virtual images and pose parameters, the refresh frame rate is dynamically adjusted according to the change information when the scene information changes to update the refresh frequency of the displayed content.
It realizes dynamic update of the refresh frame rate of augmented reality equipment, reduces system power consumption and improves the energy efficiency performance of the equipment.
Smart Images

Figure CN113485544B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image display technology, and in particular to a frame rate adjustment method, system, device and storage medium for an augmented reality device. Background Art
[0002] AR (Augmented Reality) technology refers to the technology of using computers to generate a virtual environment with realistic vision, hearing, force, touch and movement. By wearing AR devices on their heads, users can combine the virtual environment with the real environment, thereby enabling direct and natural interaction between users and the environment.
[0003] At present, when the display screen of the existing AR device presents display content to the user, it is necessary to continuously refresh the displayed content at a high frame rate, which leads to an increase in the power consumption of the AR device. Summary of the invention
[0004] The embodiments of the present application aim to solve the technical problem of high power consumption of existing AR devices by providing a frame rate adjustment method, system, device and storage medium for an augmented reality device.
[0005] The embodiment of the present application provides a frame rate adjustment method for an augmented reality device, the frame rate adjustment method for the augmented reality device comprising:
[0006] Acquire scene information of an augmented reality device, wherein the scene information includes at least one of an external environment image, a virtual image, and a posture parameter of the augmented reality device;
[0007] When the scene information changes, obtaining a target refresh frame rate of the augmented reality device according to change information of the scene information;
[0008] The refresh frame rate of the augmented reality device is updated according to the target refresh frame rate to display the target content according to the target refresh frame rate.
[0009] In one embodiment, determining that the scene information has changed includes at least one of the following:
[0010] The first change information between the current frame external environment image and the previous frame external environment image is not within the first preset range;
[0011] The second change information between the current frame virtual image and the previous frame virtual image is not within the second preset range;
[0012] The third change information between the posture parameter of the augmented reality device at the current moment and the posture parameter at the previous moment is not within the third preset range.
[0013] In one embodiment, when the scene information changes, the step of acquiring the target refresh frame rate of the augmented reality device according to the change information of the scene information includes:
[0014] Obtaining first change information between the current frame external environment image and the previous frame external environment image, second change information between the current frame virtual image and the previous frame virtual image, third change information between the posture parameters at the current moment and the posture parameters at the previous moment, and a preset refresh frame rate;
[0015] The target refresh frame rate is determined according to the first change information, the second change information, the third change information and the preset refresh frame rate.
[0016] In one embodiment, the step of obtaining first change information between a current frame external environment image and a previous frame external environment image includes:
[0017] Determine the row pixel mean of each row and the column pixel mean of each column in the current frame external environment image and the previous frame external environment image;
[0018] The first change information is obtained based on the mean difference between the row pixel means of each row in the current frame external environment image and the previous frame external environment image, and the mean difference between the column pixel means of each column in the current frame external environment image and the previous frame external environment image.
[0019] In one embodiment, the step of obtaining second change information between a current frame virtual image and a previous frame virtual image includes:
[0020] Determine the row pixel mean of each row and the column pixel mean of each column in the current frame virtual image and the previous frame virtual image;
[0021] The second change information is obtained according to the mean difference between the row pixel mean values of each row in the current frame virtual image and the previous frame virtual image, and the mean difference between the column pixel mean values of each column in the current frame virtual image and the previous frame virtual image.
[0022] In one embodiment, the posture parameter includes coordinate information detected by a gyroscope, and the step of obtaining third change information between the posture parameter at the current moment and the posture parameter at the previous moment includes:
[0023] Get the coordinate difference between the current moment's coordinate information and the previous moment's coordinate information;
[0024] The third change information is acquired according to the coordinate difference.
[0025] In one embodiment, the step of determining the target refresh frame rate according to the first change information, the second change information, the third change information and the preset refresh frame rate includes:
[0026] Obtaining a first preset weight value corresponding to the external environment image, a second preset weight value corresponding to the virtual image, and a third preset weight value corresponding to the posture parameter;
[0027] Determine a sum of products of a product of the first change information and the first preset weight value, a product of the second change information and the second preset weight value, and a product of the third change information and the third preset weight value;
[0028] The target refresh frame rate is obtained according to the product sum, the amount of the scene information and the preset refresh frame rate.
[0029] In addition, to achieve the above-mentioned purpose, the present invention also provides a frame rate adjustment system for an augmented reality device, comprising:
[0030] A first acquisition module, configured to acquire scene information of an augmented reality device, wherein the scene information includes at least one of an external environment image, a virtual image, and a posture parameter of the augmented reality device;
[0031] A second acquisition module, configured to acquire a target refresh frame rate of the augmented reality device according to change information of the scene information when the scene information changes;
[0032] A display module is used to update the refresh frame rate of the augmented reality device according to the target refresh frame rate so as to display the target content according to the target refresh frame rate.
[0033] In addition, to achieve the above-mentioned purpose, the present invention also provides an augmented reality device including: a memory, a processor, and a frame rate adjustment program stored in the memory and executable on the processor, wherein the frame rate adjustment program implements the steps of the above-mentioned frame rate adjustment method when executed by the processor.
[0034] In addition, to achieve the above-mentioned purpose, the present invention further provides a storage medium on which a frame rate adjustment program is stored, and when the frame rate adjustment program is executed by a processor, the steps of the above-mentioned frame rate adjustment method are implemented.
[0035] The technical solutions of a frame rate adjustment method, system, device and storage medium of an augmented reality device provided in the embodiments of the present application have at least the following technical effects or advantages:
[0036] By adopting a technical solution of obtaining the target refresh frame rate of the augmented reality device according to the change information of the scene information when the scene information changes, updating the refresh frame rate of the augmented reality device according to the target refresh frame rate, and displaying the target content according to the target refresh frame rate, the problem of high power consumption of existing AR devices is solved, and the dynamic update of the refresh frame rate of the augmented reality device is realized, which is conducive to reducing the system power consumption of the augmented reality device. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 A schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;
[0038] Figure 2 A schematic diagram of a flow chart of a first embodiment of a method for adjusting a frame rate of an augmented reality device according to the present invention;
[0039] Figure 3 A schematic diagram of a flow chart of a second embodiment of a method for adjusting a frame rate of an augmented reality device according to the present invention;
[0040] Figure 4 A schematic diagram of a flow chart of a third embodiment of a method for adjusting a frame rate of an augmented reality device according to the present invention;
[0041] Figure 5 A schematic flow chart of a fourth embodiment of a method for adjusting the frame rate of an augmented reality device according to the present invention;
[0042] Figure 6 A schematic diagram of a flow chart of a fifth embodiment of a method for adjusting a frame rate of an augmented reality device according to the present invention;
[0043] Figure 7 This is a functional module diagram of the frame rate adjustment system of the augmented reality device of the present invention. DETAILED DESCRIPTION
[0044] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0045] like Figure 1 As shown, Figure 1 It is a schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention.
[0046] It should be noted that Figure 1 This is a structural diagram of the hardware operating environment of the augmented reality device.
[0047] like Figure 1As shown, the augmented reality device may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0048] Those skilled in the art will understand that Figure 1 The structure of the augmented reality device shown in the figure does not constitute a limitation on the augmented reality device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0049] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating network communication module, a user interface module and a frame rate adjustment program. Among them, the operating system is a program that manages and controls the hardware and software resources of the augmented reality device, the frame rate adjustment program and the operation of other software or programs.
[0050] exist Figure 1 In the augmented reality device shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used for the background server and communicates data with the background server; the processor 1001 can be used to call the frame rate adjustment program stored in the memory 1005.
[0051] In this embodiment, the augmented reality device includes: a memory 1005, a processor 1001, and a frame rate adjustment program stored in the memory 1005 and executable on the processor, wherein:
[0052] When the processor 1001 calls the frame rate adjustment program stored in the memory 1005, the following operations are performed:
[0053] Acquire scene information of an augmented reality device, wherein the scene information includes at least one of an external environment image, a virtual image, and a posture parameter of the augmented reality device;
[0054] When the scene information changes, obtaining a target refresh frame rate of the augmented reality device according to change information of the scene information;
[0055] The refresh frame rate of the augmented reality device is updated according to the target refresh frame rate to display the target content according to the target refresh frame rate.
[0056] Wherein, determining that the scene information has changed includes at least one of the following:
[0057] The first change information between the current frame external environment image and the previous frame external environment image is not within the first preset range;
[0058] The second change information between the current frame virtual image and the previous frame virtual image is not within the second preset range;
[0059] The third change information between the posture parameter of the augmented reality device at the current moment and the posture parameter at the previous moment is not within the third preset range.
[0060] When the processor 1001 calls the frame rate adjustment program stored in the memory 1005, it also performs the following operations:
[0061] Obtaining first change information between the current frame external environment image and the previous frame external environment image, second change information between the current frame virtual image and the previous frame virtual image, third change information between the posture parameters at the current moment and the posture parameters at the previous moment, and a preset refresh frame rate;
[0062] The target refresh frame rate is determined according to the first change information, the second change information, the third change information and the preset refresh frame rate.
[0063] When the processor 1001 calls the frame rate adjustment program stored in the memory 1005, it also performs the following operations:
[0064] Determine the row pixel mean of each row and the column pixel mean of each column in the current frame external environment image and the previous frame external environment image;
[0065] The first change information is obtained based on the mean difference between the row pixel means of each row in the current frame external environment image and the previous frame external environment image, and the mean difference between the column pixel means of each column in the current frame external environment image and the previous frame external environment image.
[0066] When the processor 1001 calls the frame rate adjustment program stored in the memory 1005, it also performs the following operations:
[0067] Determine the row pixel mean of each row and the column pixel mean of each column in the current frame virtual image and the previous frame virtual image;
[0068] The second change information is obtained according to the mean difference between the row pixel mean values of each row in the current frame virtual image and the previous frame virtual image, and the mean difference between the column pixel mean values of each column in the current frame virtual image and the previous frame virtual image.
[0069] The posture parameters include coordinate information detected by the gyroscope. When the processor 1001 calls the frame rate adjustment program stored in the memory 1005, the following operations are also performed:
[0070] Get the coordinate difference between the current moment's coordinate information and the previous moment's coordinate information;
[0071] The third change information is acquired according to the coordinate difference.
[0072] When the processor 1001 calls the frame rate adjustment program stored in the memory 1005, it also performs the following operations:
[0073] Obtaining a first preset weight value corresponding to the external environment image, a second preset weight value corresponding to the virtual image, and a third preset weight value corresponding to the posture parameter;
[0074] Determine a sum of products of a product of the first change information and the first preset weight value, a product of the second change information and the second preset weight value, and a product of the third change information and the third preset weight value;
[0075] The target refresh frame rate is obtained according to the product sum, the amount of the scene information and the preset refresh frame rate.
[0076] An embodiment of the present invention provides an embodiment of a frame rate adjustment method for an augmented reality device. It should be noted that although the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different from that here. The frame rate adjustment method for an augmented reality device is applied to the display processing of the augmented reality device.
[0077] like Figure 2 As shown, in the first embodiment of the present application, the frame rate adjustment method of the augmented reality device of the present application includes the following steps:
[0078] Step S210: Acquire scene information of the augmented reality device.
[0079] Augmented reality devices are called Augmented Reality devices, or AR devices for short, such as AR glasses. When a user wears an AR device on his head, the AR device can present the image content of the real world to the user in the user's field of view. In addition, the content of the real world can also be superimposed with virtual objects, and the superimposed virtual objects can "interact" with the objects in the real world. For example, through the AR device, you see a tree in the real world. After superimposing virtual objects, a bird appears on the tree. In this embodiment, the scene information includes at least one of an external environment image, a virtual image, and a posture parameter of the augmented reality device. Among them, the external environment image refers to the image content of the real world directly seen by the user through the AR device collected by the AR device, wherein the external environment image can be collected by a camera set on the AR device; the virtual image refers to the virtual content presented by the AR device; the posture parameter includes the coordinate information detected by the gyroscope, wherein the gyroscope is set inside the AR device, and the coordinate information detected by the gyroscope is the coordinate information of the AR device, which refers to the three-dimensional space coordinates of the AR device. According to the detected real-time posture parameters, it can be determined whether the AR device is in a stationary state or a moving state. The state of the AR device depends on the user. When the user's head is relatively still, the AR device is in a stationary state. When the user's head moves relatively, the AR device is in a moving state.
[0080] Step S220: When the scene information changes, the target refresh frame rate of the augmented reality device is obtained according to the change information of the scene information.
[0081] In this implementation, when the AR device displays the target content for the user, it detects in real time whether the scene information has changed. When the scene information is detected to have changed, the change information of the scene information is obtained, and then the target refresh frame rate of the display screen of the AR device is calculated based on the change information, so as to further control the display screen to display the target content according to the target refresh frame rate. The greater the frequency of scene information changes, the greater the target refresh frame rate obtained, and vice versa, the smaller the frequency of scene information changes, the smaller the target refresh frame rate obtained.
[0082] Among them, the change information of scene information may include the change of the target content to be displayed at the current moment in the AR device display relative to the target content that has been displayed at the previous moment, or the change of the external environment at the current moment seen by the user through the AR device relative to the external environment at the previous moment, or the change of the AR device posture at the current moment relative to the AR device posture at the previous moment due to the movement of the user's head relative to the outside world.
[0083] Specifically, when the change of scene information can be determined by the current frame external environment image corresponding to the current moment and the previous frame external environment image corresponding to the previous moment, or by the current frame virtual image corresponding to the current moment and the previous frame virtual image corresponding to the previous moment, or by the posture parameters at the current moment and the posture parameters at the previous moment, it is judged whether the scene information has changed based on the difference between the two.
[0084] Furthermore, the method of determining whether the scene information has changed includes at least one of the following:
[0085] Method 1: Real-time comparison of the external environment image of the current frame with the external environment image of the previous frame to determine the first change information between the external environment image of the current frame and the external environment image of the previous frame. When the first change information between the external environment image of the current frame and the external environment image of the previous frame is not within the first preset range, it is determined that there is a difference between the external environment image of the current frame and the external environment image of the previous frame, that is, it is determined that the scene information has changed. By presetting the first preset range, when it is determined that the first change information of the external environment image is not within the first preset range, it is determined that the scene information has changed, avoiding the slight difference between the external environment image of the current frame and the external environment image of the previous frame being mistakenly recognized as a change in scene information. Among them, the first preset range is pre-set, assuming that the first preset range is [0,0.1]. If the first change information is within [0,0.1], that is, the first change information is within the first preset range, it is determined that the scene information has not changed. If the first change information is within (0.1,7.5], that is, the first change information is not within the first preset range, it is determined that the scene information has changed.
[0086] Method 2: Real-time comparison of the current frame virtual image and the previous frame virtual image to determine the second change information of the current frame virtual image and the previous frame virtual image. When the second change information of the current frame virtual image and the previous frame virtual image is not within the second preset range, it is determined that there is a difference between the current frame virtual image and the previous frame virtual image, that is, it is determined that the scene information has changed. By presetting the second preset range, when it is determined that the second change information of the virtual image is not within the second preset range, it is determined that the scene information has changed, so as to avoid misidentification of the scene information change due to the slight difference between the current frame virtual image and the previous frame virtual image. For example, when an object in the virtual image moves greatly, the second change information of the current frame virtual image and the previous frame virtual image is not within the second preset range, and the scene information is determined to have changed. When an object in the virtual image moves slightly, the second change information of the current frame virtual image and the previous frame virtual image is within the second preset range. At this time, even if the current frame virtual image changes relative to the previous frame virtual image, it will not be determined that the scene information has changed, that is, it is determined that the scene information has not changed. Among them, the second preset range is pre-set. Assuming that the second preset range is [0, 0.1], if the second change information is within [0, 0.1], that is, the second change information is within the second preset range, it is determined that the scene information has not changed. If the second change information is within (0.1, 7.5], that is, the second change information is not within the second preset range, it is determined that the scene information has changed.
[0087] Method 3: Compare the posture parameters of the AR device at the current moment with the posture parameters at the previous moment in real time to determine the third change information of the posture parameters at the current moment and the posture parameters at the previous moment. When the third change information of the posture parameters at the current moment and the posture parameters at the previous moment is not within the third preset range, it is determined that the scene information has changed. Among them, the third preset range is pre-set. Assume that the third preset range is [0,1]. If the third change information is within [0,1], that is, the third change information is within the third preset range, it is determined that the scene information has not changed. If the third change information is within (1,15], that is, the third change information is not within the third preset range, it is determined that the scene information has changed.
[0088] Step S230: updating the refresh frame rate of the augmented reality device according to the target refresh frame rate to display the target content according to the target refresh frame rate.
[0089] In this embodiment, after obtaining the target refresh frame rate, the current refresh frame rate of the AR device is obtained, and then the current refresh frame rate is updated with the target refresh frame rate, thereby controlling the display screen to display the target content according to the updated current refresh frame rate. Wherein, updating the current refresh frame rate with the target refresh frame rate means replacing the current refresh frame rate with the target refresh frame rate. For example, if the target refresh frame rate is 100FPS and the current refresh frame rate is 80FPS, then after updating the current refresh frame rate with the target refresh frame rate, the current refresh frame rate becomes 100FPS, thereby controlling the display screen to display the target content according to 100FPS.
[0090] It is worth noting that when it is determined that the scene information has not changed, the refresh frame rate of the AR device will not be adjusted. For example, if the user wears the AR device and moves slightly, causing the external environment of the scene to change slightly, the current refresh frame rate does not need to be adjusted, that is, the target content is still displayed according to the current refresh frame rate.
[0091] According to the above technical solution, this embodiment adopts a technical means of acquiring the target refresh frame rate of the augmented reality device according to the change information of the scene information when the scene information changes, updating the refresh frame rate of the augmented reality device according to the target refresh frame rate, and displaying the target content according to the target refresh frame rate. This realizes the dynamic update of the refresh frame rate of the augmented reality device, which is beneficial to reducing the system power consumption of the augmented reality device.
[0092] like Figure 3 As shown, in the second embodiment of the present application, based on the first embodiment, step S220 includes the following steps:
[0093] Step S221: Obtain first change information between the current frame external environment image and the previous frame external environment image, second change information between the current frame virtual image and the previous frame virtual image, third change information between the current moment posture parameters and the previous moment posture parameters, and a preset refresh frame rate.
[0094] In this embodiment, the first change information refers to the degree of change of the external environment image of the current frame relative to the external environment image of the previous frame, which can be determined based on the pixel difference between the external environment image of the current frame and the external environment image of the previous frame; similarly, the second change information refers to the degree of change of the virtual image of the current frame relative to the virtual image of the previous frame, which can be determined based on the pixel difference between the virtual image of the current frame and the virtual image of the previous frame; the third change information can be determined based on the parameter difference between the posture parameters at the current moment and the posture parameters at the previous moment; the preset refresh frame rate refers to the maximum refresh frame rate that the display screen of the AR device can accept.
[0095] Specifically, when it is determined that the scene information has changed, the current frame external environment image and the previous frame external environment image, the current frame virtual image and the previous frame virtual image, the posture parameters at the current moment and the posture parameters at the previous moment, and the preset refresh frame rate are obtained respectively. Among them, the first change information is obtained by comparing the pixel points of the current frame external environment image with the pixel points of the previous frame external environment image, the second change information is obtained by comparing the pixel points of the current frame virtual image with the pixel points of the previous frame virtual image, and the third change information is obtained by calculating the parameter difference between the posture parameters at the current moment and the posture parameters at the previous moment.
[0096] Step S222: determining the target refresh frame rate according to the first change information, the second change information, the third change information and the preset refresh frame rate.
[0097] In this embodiment, step S222 specifically includes the following steps:
[0098] Obtaining a first preset weight value corresponding to the external environment image, a second preset weight value corresponding to the virtual image, and a third preset weight value corresponding to the posture parameter;
[0099] Determine the sum of the products of the first change information and the first preset weight value, the second change information and the second preset weight value, and the third change information and the third preset weight value;
[0100] The target refresh frame rate is obtained according to the sum of the products, the amount of scene information and the preset refresh frame rate.
[0101] Specifically, the first preset weight value, the second preset weight value, and the third preset weight value are all preset based on experience. The first preset weight value corresponds to the external environment image, the second preset weight value corresponds to the virtual image, and the third preset weight value corresponds to the posture parameter. After obtaining the first preset weight value, the second preset weight value, and the third preset weight value, the preset target refresh frame rate calculation formula is used to calculate the target refresh frame rate of the AR device when the scene information changes. The target refresh frame rate calculation formula is:
[0102]
[0103] Wherein, F represents the target refresh frame rate, S represents the amount of scene information, K represents the preset refresh frame rate, C1 represents the first change information, C2 represents the second change information, C3 represents the third change information, W1 represents the first preset weight value, W2 represents the second preset weight value, and W3 represents the third preset weight value. The amount of scene information S is determined according to the information contained in the scene information. The scene information described in this application includes the external environment image, the virtual image, and the posture parameters of the augmented reality device, so S=3. The preset refresh frame rate is a preset value, for example, K=120FPS, where, The value range of is [0.5,1]. When S=3, and When the result is 1, the target refresh frame rate F is 120FPS, when S = 3, and When the result is 0.5, the target refresh frame rate F is 60FPS. According to experience, W1 can be set to 0.4, W2 can be set to 0.4, and W3 can be set to 0.2.
[0104] According to the above technical solution, this embodiment improves the accuracy of calculating the target refresh frame rate.
[0105] like Figure 4 As shown, in the third embodiment of the present application, based on the first embodiment, obtaining first change information between the current frame external environment image and the previous frame external environment image includes the following steps:
[0106] Step S2211: Determine the row pixel mean of each row and the column pixel mean of each column in the current frame external environment image and the previous frame external environment image.
[0107] In this embodiment, the row pixel mean refers to the average value of each row of pixels in the external environment image, and the column pixel mean refers to the average value of each column of pixels in the external environment image. Before determining the first change information, the row pixel mean of each row in the current frame external environment image and the row pixel mean of each row in the previous frame external environment image are calculated based on the row pixel mean calculation formula. The row pixel mean calculation formula is:
[0108]
[0109] Among them, H represents the mean value of row pixels, n represents the number of pixels in each row, that is, the number of pixel columns in each frame of the external environment image, and pix(i) represents the pixel value of the i-th pixel in each row.
[0110] Then, based on the column pixel mean calculation formula, the column pixel mean of each column in the current frame external environment image and the column pixel mean of each column in the previous frame external environment image are calculated respectively. The column pixel mean calculation formula is:
[0111]
[0112] Among them, L represents the mean value of column pixels, m represents the number of pixels in each column, that is, the number of pixel rows in each frame of the external environment image, and pix(j) represents the pixel value of the jth pixel in each column.
[0113] Step S2212: Obtain the first change information based on the mean difference between the row pixel means of each row in the current frame external environment image and the previous frame external environment image, and the mean difference between the column pixel means of each column in the current frame external environment image and the previous frame external environment image.
[0114] Specifically, the first change information is calculated using a first change information calculation formula, and the first change information calculation formula is:
[0115]
[0116] Wherein, in order to distinguish, the row pixel mean of each row in the external environment image of the current frame is represented as H1, the row pixel mean of each row in the external environment image of the previous frame is represented as H2, the column pixel mean of each column in the external environment image of the current frame is represented as L1, and the column pixel mean of each column in the external environment image of the previous frame is represented as L2, C1 represents the first change information, m represents the number of rows of pixels in the external environment image, n represents the number of columns of pixels in the external environment image, H1(p) represents the row pixel mean of the pth row in the external environment image of the current frame, H2(p) represents The mean of the row pixels of the p-th row in the previous frame of the external environment image, L1(q) represents the mean of the column pixels of the q-th column in the current frame of the external environment image, L2(q) represents the mean of the column pixels of the q-th column in the previous frame of the external environment image, H1(p)-H2(p) represents the mean difference between the mean of the row pixels of the p-th row in the current frame of the external environment image and the mean of the row pixels of the p-th row in the previous frame of the external environment image, and L1(q)-L2(q) represents the mean difference between the mean of the column pixels of the q-th column in the current frame of the external environment image and the mean of the column pixels of the q-th column in the previous frame of the external environment image.
[0117] According to the above technical solution, this embodiment improves the accuracy of obtaining the first change information between the current frame external environment image and the previous frame external environment image.
[0118] like Figure 5 As shown, in the fourth embodiment of the present application, based on the first embodiment, obtaining the second change information between the current frame virtual image and the previous frame virtual image includes the following steps:
[0119] Step S2221: Determine the row pixel mean of each row and the column pixel mean of each column in the current frame virtual image and the previous frame virtual image.
[0120] In this embodiment, the row pixel mean refers to the average value of each row of pixels in the virtual image, and the column pixel mean refers to the average value of each column of pixels in the virtual image. Before determining the second change information, the row pixel mean of each row in the current frame virtual image and the row pixel mean of each row in the previous frame virtual image are calculated based on the above row pixel mean calculation formula, and then the column pixel mean of each column in the current frame virtual image and the column pixel mean of each column in the previous frame virtual image are calculated based on the above column pixel mean calculation formula.
[0121] Step S2222: Obtain the second change information according to the mean difference of the row pixel mean values of each row in the current frame virtual image and the previous frame virtual image, and the mean difference of the column pixel mean values of each column in the current frame virtual image and the previous frame virtual image.
[0122] Specifically, the second change information is calculated using a second change information calculation formula. The second change information calculation formula is similar to the first change information calculation formula. The second change information calculation formula is:
[0123]
[0124] In order to distinguish, the row pixel mean of each row in the current frame virtual image is represented as H'1, the row pixel mean of each row in the previous frame virtual image is represented as H'2, the column pixel mean of each column in the current frame virtual image is represented as L'1, the column pixel mean of each column in the previous frame virtual image is represented as L'2, C2 represents the second change information, m' represents the number of rows of pixels in the virtual image, n' represents the number of columns of pixels in the virtual image, H'1(p') represents the row pixel mean of the p'th row in the current frame virtual image, H'2(p') represents the column pixel mean of the p'th row in the previous frame virtual image The row pixel mean of the p'th row, L'1(q') represents the column pixel mean of the q'th column in the current frame virtual image, L'2(q') represents the column pixel mean of the q'th column in the previous frame virtual image, H'1(p')-H'2(p') represents the mean difference between the row pixel mean of the p'th row in the current frame external environment image and the row pixel mean of the p'th row in the previous frame external environment image, L'1(q')-L'2(q') represents the mean difference between the column pixel mean of the q'th column in the current frame external environment image and the column pixel mean of the q'th column in the previous frame external environment image.
[0125] According to the above technical solution, this embodiment improves the accuracy of obtaining the second change information between the current frame virtual image and the previous frame virtual image.
[0126] like Figure 6As shown, in the fifth embodiment of the present application, based on the first embodiment, the posture parameter includes coordinate information detected by the gyroscope, and obtaining the third change information between the posture parameter at the current moment and the posture parameter at the previous moment includes the following steps:
[0127] Step S2231: Obtain the coordinate difference between the coordinate information at the current moment and the coordinate information at the previous moment.
[0128] Step S2232: Obtain the third change information according to the coordinate difference.
[0129] In this embodiment, the posture parameters include coordinate information detected by a gyroscope, wherein the gyroscope is disposed inside the AR device, and the coordinate information detected by the gyroscope is the coordinate information of the AR device, and the coordinate information refers to the three-dimensional space coordinates of the AR device. When the scene information changes, the three-dimensional space coordinates of the current moment and the three-dimensional space coordinates of the previous moment are obtained, and then the coordinate difference between the three-dimensional space coordinates of the current moment and the three-dimensional space coordinates of the previous moment is calculated, and then the third change information is obtained based on the coordinate difference. In this embodiment, the coordinate difference is used as the third change information. In other embodiments, the coordinate change rate can also be used as the third change information. Specifically, the coordinate difference calculation formula between the three-dimensional space coordinates of the current moment and the three-dimensional space coordinates of the previous moment is as follows:
[0130]
[0131] Among them, C3 represents the third change information, S represents the amount of scene information, in this embodiment, S=3, D1 (x, y, z) represents the three-dimensional space coordinates at the current moment, and D2 (x', y', z') represents the three-dimensional space coordinates at the previous moment.
[0132] Furthermore, when the scene information changes, multiple sets of three-dimensional space coordinates can be obtained at the current moment, and correspondingly, multiple sets of three-dimensional space coordinates are also obtained at the previous moment. Then, based on the above-mentioned coordinate difference calculation formula, the average three-dimensional space coordinates of the multiple sets of three-dimensional space coordinates at the current moment, and the average three-dimensional space coordinates of the multiple sets of three-dimensional space coordinates at the previous moment, the third change information is obtained.
[0133] According to the above technical solution, this embodiment is conducive to improving the accuracy of the third change information.
[0134] like Figure 7 As shown, the present application also provides a frame rate adjustment system for an augmented reality device, comprising:
[0135] A first acquisition module 310, configured to acquire scene information of an augmented reality device, wherein the scene information includes at least one of an external environment image, a virtual image, and a posture parameter of the augmented reality device;
[0136] A second acquisition module 320 is used to acquire a target refresh frame rate of the augmented reality device according to change information of the scene information when the scene information changes;
[0137] The display module 330 is used to update the refresh frame rate of the augmented reality device according to the target refresh frame rate, so as to display the target content according to the target refresh frame rate.
[0138] Further, determining that the scene information has changed includes at least one of the following:
[0139] The first change information between the current frame external environment image and the previous frame external environment image is not within the first preset range;
[0140] The second change information between the current frame virtual image and the previous frame virtual image is not within the second preset range;
[0141] The third change information between the posture parameter of the augmented reality device at the current moment and the posture parameter at the previous moment is not within the third preset range.
[0142] Furthermore, the second acquisition module 320 includes:
[0143] An information acquisition unit, used to acquire first change information between a current frame external environment image and a previous frame external environment image, second change information between a current frame virtual image and a previous frame virtual image, third change information between a posture parameter at a current moment and a posture parameter at a previous moment, and a preset refresh frame rate;
[0144] The frame rate calculation unit is used to determine the target refresh frame rate according to the first change information, the second change information, the third change information and the preset refresh frame rate.
[0145] Furthermore, the information acquisition unit includes:
[0146] A pixel calculation unit, used to determine the row pixel mean of each row and the column pixel mean of each column in the current frame external environment image and the previous frame external environment image;
[0147] A change information calculation unit is used to obtain the first change information based on the mean difference of the row pixel means in each row between the current frame external environment image and the previous frame external environment image, and the mean difference of the column pixel means in each column between the current frame external environment image and the previous frame external environment image.
[0148] Furthermore, the pixel calculation unit is also used to determine the row pixel point average of each row and the column pixel point average of each column in the current frame virtual image and the previous frame virtual image;
[0149] The change information calculation unit is also used to obtain the second change information based on the mean difference between the row pixel mean values of each row in the current frame virtual image and the previous frame virtual image, and the mean difference between the column pixel mean values of each column in the current frame virtual image and the previous frame virtual image.
[0150] Furthermore, the posture parameter includes coordinate information detected by the gyroscope, and the pixel calculation unit is further used to obtain the coordinate difference between the coordinate information at the current moment and the coordinate information at the previous moment;
[0151] The change information calculation unit is further used to obtain the third change information according to the coordinate difference.
[0152] The specific implementation of the frame rate adjustment system of the present invention is basically the same as the above-mentioned embodiments of the frame rate adjustment method, and will not be described in detail here.
[0153] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0154] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0157] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.
[0158] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0159] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A frame rate adjustment method for an augmented reality device, characterized in that: The frame rate adjustment method comprises: Acquire scene information of an augmented reality device, wherein the scene information includes at least one of an external environment image, a virtual image, and a posture parameter of the augmented reality device; When the scene information changes, obtaining a target refresh frame rate of the augmented reality device according to change information of the scene information; Determining that the scene information has changed includes at least one of the following: The first change information between the current frame external environment image and the previous frame external environment image is not within the first preset range; The second change information between the current frame virtual image and the previous frame virtual image is not within the second preset range; The third change information between the posture parameter of the augmented reality device at the current moment and the posture parameter at the previous moment is not within the third preset range; The step of acquiring the target refresh frame rate of the augmented reality device according to the change information of the scene information when the scene information changes includes: Obtaining first change information between the current frame external environment image and the previous frame external environment image, second change information between the current frame virtual image and the previous frame virtual image, third change information between the posture parameters at the current moment and the posture parameters at the previous moment, and a preset refresh frame rate; Obtaining a first preset weight value corresponding to the external environment image, a second preset weight value corresponding to the virtual image, and a third preset weight value corresponding to the posture parameter; Determine a sum of products of a product of the first change information and the first preset weight value, a product of the second change information and the second preset weight value, and a product of the third change information and the third preset weight value; Obtaining the target refresh frame rate according to the product sum, the amount of the scene information, and the preset refresh frame rate; The refresh frame rate of the augmented reality device is updated according to the target refresh frame rate to display the target content according to the target refresh frame rate.
2. The method according to claim 1, characterized in that The step of obtaining first change information between the current frame external environment image and the previous frame external environment image includes: Determine the row pixel mean of each row and the column pixel mean of each column in the current frame external environment image and the previous frame external environment image; The first change information is obtained based on the mean difference between the row pixel means of each row in the current frame external environment image and the previous frame external environment image, and the mean difference between the column pixel means of each column in the current frame external environment image and the previous frame external environment image.
3. The method according to claim 1, characterized in that The step of obtaining second change information between the current frame virtual image and the previous frame virtual image includes: Determine the row pixel mean of each row and the column pixel mean of each column in the current frame virtual image and the previous frame virtual image; The second change information is obtained according to the mean difference between the row pixel mean values of each row in the current frame virtual image and the previous frame virtual image, and the mean difference between the column pixel mean values of each column in the current frame virtual image and the previous frame virtual image.
4. The method according to claim 1, characterized in that The posture parameter includes coordinate information detected by the gyroscope, and the step of obtaining third change information between the posture parameter at the current moment and the posture parameter at the previous moment includes: Get the coordinate difference between the current moment's coordinate information and the previous moment's coordinate information; The third change information is acquired according to the coordinate difference.
5. A frame rate adjustment system for an augmented reality device, characterized in that: include: A first acquisition module, configured to acquire scene information of an augmented reality device, wherein the scene information includes at least one of an external environment image, a virtual image, and a posture parameter of the augmented reality device; A second acquisition module, configured to acquire a target refresh frame rate of the augmented reality device according to change information of the scene information when the scene information changes; Determining that the scene information has changed includes at least one of the following: first change information between the current frame external environment image and the previous frame external environment image is not within a first preset range; second change information between the current frame virtual image and the previous frame virtual image is not within a second preset range; third change information between the posture parameter of the augmented reality device at the current moment and the posture parameter of the previous moment is not within a third preset range; The step of obtaining the target refresh frame rate of the augmented reality device according to the change information of the scene information when the scene information changes includes: obtaining first change information between the external environment image of the current frame and the external environment image of the previous frame, second change information between the virtual image of the current frame and the virtual image of the previous frame, third change information between the posture parameters at the current moment and the posture parameters at the previous moment, and a preset refresh frame rate; obtaining a first preset weight value corresponding to the external environment image, a second preset weight value corresponding to the virtual image, and a third preset weight value corresponding to the posture parameter; determining the sum of products of the product of the first change information and the first preset weight value, the product of the second change information and the second preset weight value, and the product of the third change information and the third preset weight value; obtaining the target refresh frame rate according to the sum of products, the amount of the scene information, and the preset refresh frame rate; A display module is used to update the refresh frame rate of the augmented reality device according to the target refresh frame rate so as to display the target content according to the target refresh frame rate.
6. An augmented reality device, characterized in that: include: A memory, a processor, and a frame rate adjustment program stored in the memory and executable on the processor, wherein the frame rate adjustment program, when executed by the processor, implements the steps of the frame rate adjustment method according to any one of claims 1 to 4.
7. A storage medium, characterized in that: A frame rate adjustment program is stored thereon, and when the frame rate adjustment program is executed by a processor, the steps of the frame rate adjustment method described in any one of claims 1 to 4 are implemented.
