Indication information display method and device, electronic equipment and storage medium

Abstract

The invention discloses an indication information display method and device, electronic equipment and a storage medium. The method comprises the following steps: determining depth information of a target area watched by a user; based on the depth information, determining a target size of the indication information, the target size being inversely proportional to the depth information of the target area; and displaying the indication information according to the target size at the position associated with the target area. According to the technical scheme, the target size of the indication information is inversely proportional to the depth information of the target area, so that the determined target size can be matched with the depth information of the watched target area, the indication information is displayed according to the target size, flexible display of the indication information is achieved, a user can perceive the watching depth change, and the user experience is improved. The method is closer to real watching characteristics of human eyes, the display effect is enhanced, and the experience and reality of the user are greatly improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of eye tracking technology, and in particular to an indication information display method, device, electronic device and storage medium. Background Art

[0002] Eye tracking, also known as gaze tracking, is a technique for estimating the eye's gaze and / or fixation point by measuring eye movement. Eye tracking technology is widely used in experimental psychology, applied psychology, engineering psychology, cognitive neuroscience and other fields.

[0003] Eye tracking technology is applied to electronic devices, which can be controlled through human-computer interaction. When controlling electronic devices through human-computer interaction, indication information will appear on the display interface where the user's eyes are looking, and the indication information moves with the movement of the gaze point.

[0004] However, when people look at the three-dimensional scene displayed by the electronic device or look at the real world, the display mode of the indication information is constant, thereby reducing the display effect of the electronic device. Summary of the invention

[0005] The present invention provides an indication information display method, device, electronic equipment and storage medium, so as to flexibly display indication information.

[0006] In a first aspect, an embodiment of the present invention provides an indication information display method, which is applied to an electronic device with an eye tracking function, comprising:

[0007] Determine the depth information of the target area where the user is looking;

[0008] Based on the depth information, determining a target size of the indication information, wherein the target size is inversely proportional to the depth information of the target area;

[0009] The indication information is displayed at a position associated with the target area according to the target size.

[0010] In a second aspect, an embodiment of the present invention provides an indication information display device, which is integrated in an electronic device with an eye tracking function, comprising:

[0011] A first determination module is used to determine the depth information of the target area that the user is gazing at;

[0012] A second determining module, configured to determine a target size of the indication information based on the depth information, wherein the target size is inversely proportional to the depth information of the target area;

[0013] The display module is used to display the indication information according to the target size at a position associated with the target area.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the indication information display method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the indication information display method as described in the first aspect when executed.

[0019] The embodiments of the present invention provide a method, device, electronic device and storage medium for displaying indication information. The method comprises: determining the depth information of a target area that a user is gazing at; based on the depth information, determining the target size of the indication information, the target size being inversely proportional to the depth information of the target area; and displaying the indication information at the target size at a position associated with the target area. In the above technical solution, the target size of the indication information is inversely proportional to the depth information of the target area, so that the determined target size can match the depth information of the target area being gazed at, and displaying the indication information at the target size realizes the flexible display of the indication information, so that the user can perceive the change in gaze depth, which is closer to the real gaze characteristics of the human eye, enhances the display effect, and greatly improves the user's experience and sense of reality.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart of a method for displaying indication information provided in Embodiment 1 of the present invention;

[0023] Figure 2A schematic diagram of a display screen provided in Embodiment 1 of the present invention;

[0024] Figure 3 A schematic diagram of an object with different depth information provided by the first embodiment of the present invention;

[0025] Figure 4 A flowchart of another method for displaying indication information provided in Embodiment 2 of the present invention;

[0026] Figure 5 A flowchart of another method for displaying indication information provided in Embodiment 3 of the present invention;

[0027] Figure 6 A schematic diagram of the structure of an indication information display device provided in Embodiment 4 of the present invention;

[0028] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0029] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. In addition, the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict. It is also necessary to explain that, for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all structures.

[0030] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0031] It should be noted that the concepts such as "first" and "second" mentioned in the embodiments of the present invention are only used to distinguish different devices, modules, units or other objects, and are not used to limit the order or interdependence of the functions performed by these devices, modules, units or other objects.

[0032] Eye tracking technology is an important technology in contemporary psychology research. When a person's eyes look in different directions, there will be subtle changes in the eyes, which will produce extractable features. Computers can extract these features through image capture or scanning, thereby tracking eye changes in real time, predicting the user's state and needs, and responding to achieve the purpose of controlling the device with the eyes.

[0033] The optical recording method is currently widely used for eye tracking: using a camera or video camera to record the eye movement of the subject, that is, to obtain an eye image reflecting the eye movement, and to extract eye features from the obtained eye image to establish a line of sight / fixation point estimation model. Among them, eye features may include: pupil position, pupil shape, iris position, iris shape, eyelid position, eye corner position, light spot (also called Purkinje spot) position, etc. Among them, the line of sight can be understood as a three-dimensional vector, and the fixation point can be understood as the two-dimensional coordinate of the above three-dimensional vector projected on a certain plane.

[0034] Among optical recording methods, eye tracking methods include pupil-corneal reflection methods. Other methods may also include methods based on eye images, such as inferring eye movements based on contact / non-contact sensors (such as electrodes, capacitive sensors).

[0035] Among them, the pupil-corneal reflection method is summarized as follows:

[0036] 1. The working principle can be simply summarized as follows:

[0037] Acquiring an eye image;

[0038] Estimating gaze / fixation from eye images.

[0039] 2. Hardware requirements:

[0040] (1) Light source: generally an infrared light source, because infrared light does not affect the vision of the eyes; and can be multiple infrared light sources arranged in a predetermined manner, such as a herringbone shape, a straight line shape, etc.;

[0041] (2) Image acquisition equipment: such as infrared camera equipment, infrared image sensor, camera or video camera, etc.

[0042] 3. The specific implementation is:

[0043] (1) Eye image acquisition:

[0044] The light source is directed toward the eye, and the image acquisition device takes a picture of the eye, and the reflection point of the light source on the cornea is photographed accordingly, i.e., the light spot (also called Purkinje spot), thereby obtaining an eye image with the light spot.

[0045] (2) Line of sight / fixation point estimation:

[0046] As the eyeball rotates, the relative position relationship between the pupil center and the light spot changes accordingly, and the correspondingly collected eye images with light spots reflect such position change relationship;

[0047] The line of sight / fixation point estimation is performed according to the position change relationship.

[0048] When eye tracking technology is applied, a cursor, prompt content and other indication information will appear at the position where the user's eyes are looking on the display interface of the screen, and the indication information can move and change as the gaze point moves. Head-mounted display devices, electronic devices, etc. can display three-dimensional scenes, and can display a cursor following the user's gaze point or display prompt information based on the object being looked at. In existing products, this cursor display method displays a cursor of the same size regardless of whether the user is looking at a far or near object. In this display method, the depth information of the displayed content and the object being looked at does not match, which may reduce the expressiveness of the head-mounted display device. In order to solve this technical problem, the present invention provides an indication information display method to achieve flexible display of indication information.

[0049] Embodiment 1

[0050] Figure 1 A flowchart of an indication information display method provided for Embodiment 1 of the present invention, this embodiment can be applied to the situation where indication information is displayed more flexibly. Specifically, the indication information display method can be executed by an indication information display device, and the indication information display device can be implemented by software and / or hardware, and integrated in an electronic device with an eye tracking function. Furthermore, electronic devices include but are not limited to: electronic devices such as desktop computers, laptop computers, smart phones, wearable devices, and servers. Among them, wearable devices include but are not limited to: augmented reality technology (Augmented Reality, AR), virtual reality (Virtual Reality, VR), mixed reality (Mixed Reality, MR) and analytical glasses. Analytical glasses can be considered as glasses with analytical functions, such as glasses with eye tracking analysis functions.

[0051] like Figure 1 As shown, the method specifically comprises the following steps:

[0052] S110: Determine depth information of a target area that the user is gazing at.

[0053] In this embodiment, the user may be looking at the screen displayed on the electronic device, or may be looking at the real-world screen viewed by the user through an AR, MR or other device. The target area may be understood as the area where the user's eyes are looking. The depth information may be understood as information characterizing the depth of the target area, such as the distance of the target area from the user. More specifically, the depth information may be the distance between the target area and the human eye in a preset display screen, or may be the distance between the target area and the human eye in the real world.

[0054] In this operation, the target area that the user is gazing at can be determined by eye tracking technology, and then the depth information of the target area can be determined. When determining the target area of ​​gazing, the user's gaze point can be determined first, and then the target area where the gaze point is located can be determined.

[0055] In this embodiment, if the screen the user is looking at is a screen displayed by an electronic device, the displayed screen may be divided into a plurality of depth ranges according to depth, and the depth information of the target area may be determined by determining the depth range in which the target area is located.

[0056] The depth range after the display screen is divided may be transmitted to the electronic device together with the display screen, or may be obtained by dividing the display screen when determining the depth information.

[0057] If the user is looking at a picture in the real world, the target object can be determined based on the user's gaze point, and then the distance between the target object and the user can be determined based on the position of the target object. The distance is the depth information of the target area the user is looking at; or the user's line of sight direction is used to intersect with the picture taken by the scene camera to determine the target area of ​​gaze. The electronic device has a depth information acquisition function or a deep learning algorithm to acquire its depth information, wherein the line of sight direction can be the line of sight direction of binocular fusion or the line of sight direction of the dominant eye; the depth information can also be calculated by calculating the user's binocular parallax.

[0058] S120. Determine a target size of the indication information based on the depth information, where the target size is inversely proportional to the depth information of the target area.

[0059] In this embodiment, the indication information can be understood as information used to indicate the target area. The indication information can move as the target area changes, and the indication information can be presented in a variety of ways, such as a cursor, text-type indication information, image-type indication information, graphic-type indication information, etc. The target size can be understood as the size of the indication information, such as the size presented on the screen displayed by the electronic device; or the size presented on the screen in the real world.

[0060] Among them, the target size can be determined by an electronic device, and the target size is inversely proportional to the depth information of the target area, that is, the larger the depth information is, the farther away from the user, the smaller the target size of the indication information is, and the smaller the depth information is, the closer to the user, the larger the target size of the indication information is. For example, when a user looks at the screen of an electronic device, the indication information is a circular cursor, and the cursor moves in real time with the user's gaze point. When the user looks at object A, the depth information of object A is 1 meter, and the diameter of the circular cursor is 5 mm; when the user looks at object B, the depth information of object B is 5 meters, and the diameter of the circular cursor becomes 3 mm.

[0061] This operation does not limit the means of determining the target size, as long as the determined target size is inversely proportional to the depth information.

[0062] In one embodiment, different depth information may correspond to different candidate sizes. This operation may first determine the candidate size corresponding to the depth information, and then determine the target size based on the candidate size, such as directly determining the corresponding candidate size as the target size, or analyzing the candidate size corresponding to the depth information and determining the target size based on the analysis result.

[0063] The candidate size can be understood as a size that is inversely proportional to the depth information, that is, regardless of the situation where the size of the indication information is too small and cannot be seen clearly due to the depth information being too deep, as long as it is inversely proportional to the depth information, it is a candidate size. In one embodiment, this operation can first analyze the depth information, and determine the target size based on the result of the depth information analysis.

[0064] The display rules of the indication information can be set arbitrarily. For example, the indication information can be displayed smoothly according to the depth information of the target area. For example, as the depth information changes continuously, the target size of the indication information also shows a trend of continuous change. It can also be divided into multiple depth ranges, and the target size of the indication information remains unchanged within the same depth range, and different target sizes correspond to different depth ranges.

[0065] This operation determines the target size of the indication information that is inversely proportional to the depth information based on the depth information, solving the technical problem in the prior art that the cursor or prompt information at the gaze point does not change in size according to the change in gaze depth, which does not conform to the general cognition of the human eye when observing objects. By determining the target size for the indication information, so that the indication information is displayed at the target size, it conforms to the general cognition of the user when observing objects (for example, when the human eye generally looks at an object, it is larger in size when it is close, and when the object becomes farther away, the object should gradually become smaller), it helps the user to be immersed in the interaction, thereby improving the experience.

[0066] S130: Display the indication information at a position associated with the target area according to the target size.

[0067] The position associated with the target area can be understood as the location of the indication information, which can be the location of the gaze point, the position within the target area, or the position near the target area, such as a position around the target area (up, down, left, right, upper right, lower right, upper left, lower left, etc.).

[0068] The indication information presents different sizes as the user's gaze depth changes, which realizes the flexible display of the indication information. The user can perceive the change of gaze depth, which is closer to the real gaze characteristics of the human eye and enhances the display effect. Especially in virtual reality scenes, it will not cause dizziness, nausea, and discomfort to the user due to the unreality of the virtual display screen, which can greatly enhance the product experience and sense of reality.

[0069] For example, Figure 2 A schematic diagram of a display screen provided in Embodiment 1 of the present invention. Figure 3 This is a schematic diagram of an object with different depth information provided by the first embodiment of the present invention. Figure 2 As shown in FIG. 1 , a dog, a car and a mountain are displayed on the display screen. Although the display screen is a plane, the specific objects displayed are in different depth planes, that is, the target areas to which different objects belong have different depth information. Figure 3 As shown, the puppy is closest to the human eye, followed by the car, and the farthest is the mountain. The indication information is presented in the form of a five-pointed star. The size of the indication information (such as a five-pointed star) displayed when the user looks at the puppy is larger than the size of the indication information displayed when the user looks at the car and the mountain.

[0070] A method for displaying indication information provided in a first embodiment of the present invention includes: determining the depth information of a target area that a user is gazing at; determining a target size of the indication information based on the depth information, wherein the target size is inversely proportional to the depth information of the target area; and displaying the indication information at the target size at a position associated with the target area. In the above technical solution, the target size of the indication information is inversely proportional to the depth information of the target area, so that the determined target size can match the depth information of the target area being gazed at, and displaying the indication information at the target size realizes flexible display of the indication information, so that the user can perceive the change in gaze depth, which is closer to the real gaze characteristics of the human eye, enhances the display effect, and greatly improves the user's experience and sense of reality.

[0071] In one embodiment, the depth information is the distance between the target area and the user, the indication information is a cursor, and / or the indication information is prompt information associated with the target area.

[0072] The cursor can be any shape, such as an arrow, a circle, a five-pointed star, a cross, or a cartoon character. The prompt information can be understood as information that appears according to pre-set rules when the user looks at an object. The prompt information can display the information of the object, or the next operation that the user needs to perform, or other interactive information. When the user looks at the object, the prompt information will appear. The size of the prompt information and the text and other contents inside it can vary according to the depth information of the target area to which the object belongs. The smaller the depth information, the larger the prompt information, and the larger the depth information, the smaller the prompt information.

[0073] In an embodiment, the depth information of the indication information is the same as the depth information of the target area.

[0074] In a three-dimensional world, the target area has its own depth plane, and the indication information can also have its own depth plane. When the depth information of the indication information and the target area is the same, it can be understood that the two are located in the same depth plane. In the display interface, the depth information of the indication information is set to the depth information of the target area, so that the indication information also has different depth information, which can provide users with a more realistic visual effect and is more in line with the viewing habits of the human eye.

[0075] Embodiment 2

[0076] Figure 4 This is a flowchart of another method for displaying indication information provided by Embodiment 2 of the present invention. This embodiment is a refinement of the above embodiment, and specifically describes determining the target size of the indication information based on the depth information. It should be noted that the technical details not described in detail in this embodiment can be referred to any of the above embodiments.

[0077] Specifically, Figure 4 As shown, the method specifically comprises the following steps:

[0078] S210: Determine depth information of a target area that the user is gazing at.

[0079] S220: Determine a candidate size corresponding to the depth information, where the candidate size is a size of the indication information determined based on the depth information of the target area.

[0080] Specifically, the electronic device may determine a candidate size that is inversely proportional to the depth information, so as to facilitate analysis based on the candidate size and determine the target size based on the analysis result.

[0081] In one embodiment, this step may search for a candidate size corresponding to the depth information of the target area based on the correspondence between the depth information and the size. The depth information and the candidate size may be in a one-to-one correspondence, and the greater the depth indicated by the depth information, the smaller the candidate size corresponding to the depth information. The smaller the depth indicated by the depth information, the larger the candidate size corresponding to the depth information. For example, candidate size = K / depth information, where K is a coefficient, and the size can be set according to actual conditions.

[0082] In another embodiment, this step may first determine the depth range to which the depth information belongs. Different depth ranges correspond to different candidate sizes. In this embodiment, the candidate size corresponding to the depth information may be determined based on the candidate size of the depth range to which the depth information belongs.

[0083] When determining the candidate size corresponding to the depth information based on the candidate size of the depth range to which the depth information belongs, the candidate size of the depth range to which the depth information belongs may be directly determined as the candidate size corresponding to the depth information, and the candidate size may be determined as the target size.

[0084] In another embodiment, in order to further refine the target size so that the size of the indication information is not too small, the candidate sizes may be analyzed, specifically:

[0085] S230, determining whether the candidate size corresponding to the depth information is greater than or equal to the set size, if so, executing S240; if not, executing S250.

[0086] In this embodiment, the set size can be understood as a size pre-set for the indication information when analyzing the size of the indication information, for example, when the size of the candidate size is used as the basis for judgment, a minimum value is set for the size of the indication information to avoid the size of the indication information being too small.

[0087] Specifically, determine whether the candidate size corresponding to the depth information is greater than or equal to the set size. If so, determine the candidate size as the target size; if not, determine the set size as the target size. Candidate sizes that are greater than or equal to the set size can be considered as sizes that can be clearly seen by the user. Candidate sizes that are smaller than the set size can be considered as sizes that cannot be clearly seen by the user. Determining the candidate size as the target size can ensure that the determined target size meets the visual characteristics. Determining the set size as the target size can ensure that the determined target size is not displayed clearly due to being too small.

[0088] S240: Determine the candidate size as the target size.

[0089] S250: Determine the set size as the target size.

[0090] S260: Display the indication information at a position associated with the target area according to the target size.

[0091] In this embodiment, the electronic device can determine a candidate size corresponding to the depth information. If the candidate size is greater than or equal to the set size, the candidate size is determined as the target size. That is to say, when the determined candidate size of the indication information is large and does not affect the clarity of the user's viewing, the candidate size is determined as the target size of the indication information, and the indication information is displayed at the position associated with the target area. If the candidate size is smaller than the set size, the set size is determined as the target size of the indication information. That is to say, when the determined candidate size of the indication information is small and has affected the user's viewing experience, the indication information is not further reduced according to the candidate size, but the set size is directly determined as the target size, and the indication information is displayed at the position associated with the target area. This ensures that the indication information will not be infinitely reduced when the depth information is too deep, thereby achieving flexible display of the indication information and improving the viewing effect.

[0092] A method for displaying indication information provided in the second embodiment of the present invention is refined on the basis of the above embodiment, by determining the depth information of the target area that the user is gazing at and the candidate size corresponding to the depth information, if the candidate size corresponding to the depth information is greater than or equal to the set size, the candidate size is determined as the target size; if the candidate size corresponding to the depth information is less than the set size, the set size is determined as the target size, and then the indication information is displayed at the position associated with the target area. This method uses the candidate size as a judgment basis to determine the target size, thereby avoiding the problem that the indication information size is too small due to the excessive depth information of the target area, which affects the user's viewing experience. By analyzing the candidate size corresponding to the depth information, the target size is flexibly determined, and then the indication information is flexibly displayed, which greatly improves the user's experience and sense of reality.

[0093] Optionally, determining the candidate size corresponding to the depth information includes:

[0094] At least two depth ranges are divided according to the depth; each of the depth ranges does not overlap, each of the depth ranges corresponds to a candidate size, and the candidate size is inversely proportional to the depth range;

[0095] A candidate size corresponding to a depth range in which the depth information is located is determined as the candidate size corresponding to the depth information.

[0096] In one embodiment, the depth range may be an interval representing the depth. The depth range may be preset according to actual conditions or user requirements, such as a depth range of greater than 1 meter and less than or equal to 4 meters, a depth range of greater than 4 meters and less than or equal to 8 meters, and a depth range of greater than 8 meters and less than or equal to 12 meters. The candidate size corresponding to the depth range is also inversely proportional to the depth range, which conforms to the visual characteristic that objects are larger when near and smaller when far.

[0097] For example, when dividing the depth range, the division can be based on the objects displayed on the screen (such as the screen displayed on the electronic device, the real world screen viewed through AR, MR and other devices); or the depth of the screen can be evenly divided to obtain multiple depth ranges. Figure 2 The displayed picture shown includes three objects at different depths, so the picture can be divided into three depth ranges according to the depths. Different displayed pictures can be divided into different depth ranges.

[0098] When the depth range is divided based on the objects in the picture, different depth ranges correspond to different candidate sizes. The specific values ​​of the candidate sizes corresponding to the depth ranges are not limited here, so that the candidate sizes corresponding to adjacent depth ranges can be as smooth as possible without causing sudden changes in the candidate sizes. For example, the candidate size corresponding to the depth range greater than 1 meter and less than or equal to 4 meters is 3 mm, the candidate size corresponding to the depth range greater than 4 meters and less than or equal to 8 meters is 2 mm, and the candidate size corresponding to the depth range greater than 8 meters and less than or equal to 12 meters is 1 mm.

[0099] After determining the depth range included in the display picture, this embodiment may determine in which depth range the depth information is located, and then determine the candidate size corresponding to the depth range in which the depth information is located as the candidate size of the depth information.

[0100] In this embodiment, the corresponding candidate size is determined by determining the depth range of the depth information, which improves the efficiency of determining the candidate size, thereby helping to improve the efficiency of determining the target size.

[0101] Embodiment 3

[0102] Figure 5 This is a flowchart of another method for displaying indication information provided in Embodiment 3 of the present invention. This embodiment is a refinement of the above embodiment, and specifically describes determining the target size of the indication information based on the depth information. It should be noted that the technical details not described in detail in this embodiment can be referred to any of the above embodiments.

[0103] Specifically, Figure 5 As shown, the method specifically comprises the following steps:

[0104] S310: Determine depth information of a target area that the user is gazing at.

[0105] S320, determining whether the value of the depth information is greater than or equal to a set depth, if so, executing S330; if not, executing S340.

[0106] In this embodiment, the set depth can be considered as a pre-set depth. The set depth can be a critical value of the depth. After the depth information is greater than the set depth, the target size can be kept constant or displayed in other ways to avoid the target size being too small. The set depth can be used to analyze the depth information to avoid the depth information being too large, which results in the target size of the indication information being too small.

[0107] Specifically, determine whether the value of the depth information is greater than or equal to the set depth. If so, determine the default size as the target size of the indication information; if not, determine the candidate size corresponding to the depth information, and determine the candidate size as the target size.

[0108] S330: Determine a default size as a target size of the indication information.

[0109] In this embodiment, the default size can be considered as a size pre-set for the indication information when analyzing the depth information. For example, when the depth information is used as the basis for judgment, the minimum value set for the size of the indication information is used to avoid the size of the indication information being too small. Determining the default size as the target size of the indication information avoids the depth information taking a value that is too large, thereby causing the target size of the indication information to be too small.

[0110] S340: Determine a candidate size corresponding to the depth information, and determine the candidate size as the target size.

[0111] The candidate size is the size of the indication information determined based on the depth information of the target area. The means for determining the candidate size can refer to the above embodiment.

[0112] When the depth information is less than the set depth, the candidate size is determined as the target size, so that the determined target size satisfies the visual characteristics.

[0113] S350: Display the indication information at a position associated with the target area according to the target size.

[0114] A method for displaying indication information provided in the third embodiment of the present invention is refined on the basis of the above embodiment, by determining the depth information of the target area that the user is gazing at, and then judging whether the value of the depth information is greater than or equal to the set depth. If so, the default size is determined as the target size of the indication information; if not, the candidate size corresponding to the depth information is determined, and the candidate size is determined as the target size, and finally the indication information is displayed at the position associated with the target area according to the target size. In the above technical solution, the depth information is used as the judgment basis to determine the target size, and then the indication information is displayed, which avoids the display effect of the indication information being affected by the target size being too small due to the depth information being too large, and further realizes the flexible display of the indication information.

[0115] Optionally, the appearance of the indication information displayed in the default size is different from the appearance of the indication information displayed in the candidate size.

[0116] Exemplarily, when the depth information is greater than or equal to the set depth, the appearance of the indication information can be changed, for example, by enlarging the indication information with special effects such as magnifying bubbles. That is to say, due to the difference in depth information, the appearance of the indication information displayed at the default size may be different from the appearance of the indication information displayed at the candidate size. This enriches the display method of the indication information and makes the indication information easier to view by changing the appearance.

[0117] When the value of the depth information is greater than the value of the set depth, as the depth increases, the indication information may be displayed in different default sizes or the same default size, as long as the indication information can be clearly seen by the user.

[0118] Optionally, the default size is larger than the candidate size corresponding to the depth information.

[0119] When the depth information is greater than or equal to the set depth, the candidate size corresponding to the depth information will be smaller. In order to avoid the problem of unclear displayed indication information, the indication information can be displayed in a default size larger than the candidate size corresponding to the depth information, so that the indication information can be viewed in a larger size, improving the viewing experience.

[0120] Taking AR products as an example, if the user is using AR glasses, what the user sees is the real scene superimposed on the virtual scene. AR glasses have the function of perceiving the depth information of objects in the real scene. When the user looks at an object or target area in the real scene, AR glasses calculate the distance of the object or target area from the user, thereby obtaining the depth information of the object or target area. Depending on the depth information, the size of the indication information that appears on the display screen is different. Virtual information and objects in the real scene are integrated, which can give users a better user experience.

[0121] For example, if the user is looking at a vase that is 2 meters away from the user, the prompt message that appears on the AR glasses will be "The price of the vase is 500 yuan"; when the user is looking at a piano in the distance that is 10 meters away from the user, the prompt message that appears at this time will be "The price of the piano is 30,000 yuan", which will be smaller than the prompt message of the vase; when the user is looking at a building in the distance, because the building is too far away from the user, the prompt message "Oriental Pearl Tower" can be the same size as the price of the piano, or displayed in the form of an enlarged bubble, which is larger than the price of the piano.

[0122] Take virtual reality products as an example. The screen of VR products displays virtual reality images, and each scene can display a three-dimensional scene, so that the wearer, that is, the user, has an immersive feeling. After calculating the wearer's gaze point, VR with eye tracking function can display a cursor on the display screen in real time, and the cursor moves as the gaze point changes.

[0123] When the cursor is displayed, the size of the cursor is displayed according to the depth information of the target area. The smaller the depth of the target area, that is, the closer it is to the user, the larger the cursor is, and the larger the depth, that is, the farther it is from the user, the smaller the cursor is.

[0124] The specific display rules can be set arbitrarily. For example, the display cursor or prompt information can be displayed smoothly according to the depth information of the target area. A threshold can also be set so that the prompt information will not be infinitely reduced when the depth of the object is too large, so that the user cannot see the prompt information clearly. A minimum value can be set, and the cursor will no longer be reduced when it reaches the minimum value (that is, if the value of the depth information is greater than or equal to the set depth, the default size is determined as the target size of the indication information). Several display standards can also be set. According to the depth range, when the depth information of the target area being looked at falls into a certain depth range, the candidate size corresponding to the depth range is displayed.

[0125] Embodiment 4

[0126] Figure 6 This is a schematic diagram of the structure of an indication information display device provided in Embodiment 4 of the present invention. The device can execute the indication information display method provided in any of the above embodiments of the present invention. The indication information display provided in this embodiment includes:

[0127] A first determination module 410 is used to determine the depth information of the target area that the user is gazing at;

[0128] A second determination module 420, configured to determine a target size of the indication information based on the depth information, wherein the target size is inversely proportional to the depth information of the target area;

[0129] The display module 430 is configured to display the indication information according to the target size at a position associated with the target area.

[0130] Optionally, based on the above embodiment, the depth information is the distance between the target area and the user, the indication information is a cursor, and / or the indication information is prompt information associated with the target area.

[0131] Optionally, the depth information of the indication information is the same as the depth information of the target area.

[0132] Optionally, based on the above embodiment, the second determining module 420 includes:

[0133] A first determining unit, configured to determine a candidate size corresponding to the depth information, wherein the candidate size is a size of the indication information determined based on the depth information of the target area;

[0134] The second determining unit is configured to determine the candidate size as the target size if the candidate size corresponding to the depth information is greater than or equal to a set size; otherwise, determine the set size as the target size.

[0135] Optionally, based on the above embodiment, the first determining unit includes:

[0136] A depth range division subunit is used to divide at least two depth ranges according to the depth; each of the depth ranges does not overlap, each of the depth ranges corresponds to a candidate size, and the candidate size is inversely proportional to the depth range;

[0137] The first determining subunit is configured to determine a candidate size corresponding to a depth range in which the depth information is located as the candidate size corresponding to the depth information.

[0138] Optionally, based on the above embodiment, the second determining module 420 includes:

[0139] The third determination unit is used to determine a default size as the target size of the indication information if the value of the depth information is greater than or equal to the set depth; otherwise, determine a candidate size corresponding to the depth information, and determine the candidate size as the target size, wherein the candidate size is the size of the indication information determined based on the depth information of the target area.

[0140] Optionally, based on the above embodiment, the appearance of the indication information displayed in the default size is different from the appearance of the indication information displayed in the candidate size.

[0141] Optionally, based on the above embodiment, the default size is larger than the candidate size corresponding to the depth information.

[0142] The indication information display device provided in the fourth embodiment of the present invention can be used to execute the indication information display method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0143] Embodiment 5

[0144] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement any of the above-mentioned embodiments of the present invention is shown. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, user equipment, wearable devices (such as AR, VR, MR helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0145] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0146] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, wireless networks.

[0147] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described in any of the above embodiments, such as the indication information display method.

[0148] In some embodiments, the indication information display method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the indication information display method in any other appropriate manner (e.g., by means of firmware).

[0149] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0151] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0152] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device 10 having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device 10. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0153] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0154] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0155] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0156] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for displaying indication information, characterized in that: Applied to an electronic device with an eye tracking function, the method comprises: Determine the depth information of the target area where the user is looking; Based on the depth information, determining a target size of the indication information, wherein the target size is inversely proportional to the depth information of the target area; The indication information is displayed at a position associated with the target area according to the target size.

2. The method according to claim 1, characterized in that The depth information is the distance between the target area and the user, the indication information is a cursor, and / or the indication information is prompt information associated with the target area.

3. The method according to claim 1, characterized in that The depth information of the indication information is the same as the depth information of the target area.

4. The method according to claim 1, 2 or 3, characterized in that: The determining, based on the depth information, a target size of the indication information includes: Determine a candidate size corresponding to the depth information, where the candidate size is the size of the indication information determined based on the depth information of the target area; If the candidate size corresponding to the depth information is greater than or equal to the set size, the candidate size is determined as the target size; otherwise, the set size is determined as the target size.

5. The method according to claim 4, characterized in that The determining the candidate size corresponding to the depth information includes: At least two depth ranges are divided according to the depth; each of the depth ranges does not overlap, each of the depth ranges corresponds to a candidate size, and the candidate size is inversely proportional to the depth range; A candidate size corresponding to a depth range in which the depth information is located is determined as the candidate size corresponding to the depth information.

6. The method according to claim 1, 2 or 3, characterized in that: The determining, based on the depth information, a target size of the indication information includes: If the value of the depth information is greater than or equal to the set depth, the default size is determined as the target size of the indication information; otherwise, a candidate size corresponding to the depth information is determined, and the candidate size is determined as the target size, where the candidate size is the size of the indication information determined based on the depth information of the target area.

7. The method according to claim 6, characterized in that The appearance of the indication information displayed in the default size is different from the appearance of the indication information displayed in the candidate size.

8. The method according to claim 6, characterized in that The default size is larger than the candidate size corresponding to the depth information.

9. An indication information display device, characterized in that: An electronic device having an eye tracking function is integrated into the device, the device comprising: A first determination module is used to determine the depth information of the target area that the user is gazing at; A second determining module, configured to determine a target size of the indication information based on the depth information, wherein the target size is inversely proportional to the depth information of the target area; The display module is used to display the indication information according to the target size at a position associated with the target area.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the indication information display method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement any method of claims 1-8 when executed.

Images