Image display method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the field of intelligent equipment information processing, and discloses an image display method and device, electronic equipment and a storage medium, the image display method comprises the following steps: detecting a sight falling point track formed by projecting the sight of a user on the surface of a refrigerator; on the basis of the sight falling point track, a stored article, corresponding to a current sight falling point in the sight falling point track, in the refrigerator is selected; and forming a three-dimensional image of the stored article, and displaying the three-dimensional image through a display screen on the refrigerator. According to the technical scheme, the problems that in the prior art, the object image definition is low, the object recognition difficulty is large, the recognition display accuracy is low, and the user experience of refrigerator storage object management is poor due to a refrigerator storage object photographing mode are solved, the object image definition can be improved, the object recognition difficulty is reduced, and the user experience is improved. The identification display accuracy is improved, and the refrigerator storage article management experience of the user is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent device information processing, and in particular to an image display method, device, electronic device and storage medium. Background Art

[0002] With the continuous advancement and development of science and technology, intelligence has become an inevitable trend in the development of home appliances. With the popularization of smart refrigerators and the emergence of large-screen refrigerators, refrigerators are no longer just cabinets for refrigerating food, but have been given more abundant functions. However, as far as the refrigerator itself is concerned, its most important function is still to store and manage food ingredients. Therefore, the intelligent management of refrigerator storage items is an important component of refrigerator intelligence.

[0003] The larger the user's refrigerator is, the more food it contains, and the more difficult it is for the user to remember the storage status of the items in the refrigerator. The existing technology achieves this by installing a camera in the refrigerator to take pictures of the items. However, this camera photo taking method has the problem of low photo clarity, difficulty in object recognition, and low recognition display accuracy, which brings a bad experience to users in managing refrigerator storage items. Summary of the invention

[0004] The purpose of the present invention is to at least provide an image display method, device, electronic device and storage medium, which can at least solve the problem that the existing method of photographing items stored in the refrigerator results in low image clarity of the item, great difficulty in item recognition, low recognition and display accuracy, and poor user experience in managing items stored in the refrigerator. At least the effect of improving the image clarity of the item, reducing the difficulty of item recognition, improving the recognition and display accuracy, and improving the user experience in managing items stored in the refrigerator can be achieved.

[0005] To solve the above technical problem, at least one embodiment of the present application provides a method for displaying an image, comprising:

[0006] Detect the trajectory of the user's sight points projected on the refrigerator surface;

[0007] Based on the sight point trajectory, selecting the storage items inside the refrigerator corresponding to the current sight point in the sight point trajectory;

[0008] A three-dimensional image of the stored items is formed, and the three-dimensional image is displayed on a display screen on the refrigerator.

[0009] At least one embodiment of the present application further provides an image display device, including:

[0010] A detection module, used to detect the trajectory of the user's sight points projected on the refrigerator surface;

[0011] A selection module, configured to select, based on the sight point trajectory, a storage item inside the refrigerator corresponding to a current sight point in the sight point trajectory;

[0012] The display module is used to form a three-dimensional image of the stored items and display the three-dimensional image through a display screen on the refrigerator.

[0013] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned image display method.

[0014] At least one embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned image display method when executed by a processor.

[0015] The embodiment of the present application provides a method for displaying an image, which detects the sight point trajectory formed by the user's sight projected on the refrigerator surface; based on the sight point trajectory, selects the storage items inside the refrigerator corresponding to the current sight point in the sight point trajectory; forms a three-dimensional image of the storage items, and displays the three-dimensional image through the display screen on the refrigerator. It can improve the clarity of the item image, reduce the difficulty of item recognition, improve the recognition display accuracy, and enhance the user's refrigerator storage item management experience.

[0016] In some optional embodiments, the step of detecting a sight point trajectory formed by the user's sight projected on the surface of the refrigerator includes:

[0017] Detecting the position of the user's eyes relative to the surface of the refrigerator;

[0018] Use infrared equipment to collect the user's line of sight angle projected on the refrigerator surface;

[0019] According to the position point of the user's eye and the sight angle, the sight point where the user's sight is projected on the refrigerator surface is calculated, and a sight point trajectory is formed.

[0020] Among them, by accurately knowing the landing point and trajectory of the user's eye line of sight on the surface of the refrigerator, the refrigerator's intelligent system can provide users with personalized interactive content based on the landing point trajectory of the user's eyes, making the user's interaction with the refrigerator more in line with their own needs and improving the convenience and fun of use.

[0021] In some optional embodiments, detecting the position of the user's eyes relative to the surface of the refrigerator includes:

[0022] An X-axis is established along the width direction of the front of the refrigerator, a Y-axis is established along the height direction of the front of the refrigerator, and a Z-axis is established along a direction perpendicular to the front of the refrigerator to form a spatial coordinate system for describing the spatial position relationship between the user's eyes and the surface of the refrigerator;

[0023] The coordinate point of the user's eye in the spatial coordinate system is used as the position point of the user's eye relative to the surface of the refrigerator.

[0024] Among them, by establishing a spatial coordinate system, the position of the user's eyes relative to the surface of the refrigerator can be accurately and quickly determined, thereby determining the storage items inside the refrigerator corresponding to the user's line of sight, and then displaying a three-dimensional image of the storage items to improve the user's viewing experience.

[0025] In some optional embodiments, the step of using an infrared device to collect the sight angle of the user's eyes projected on the refrigerator surface includes:

[0026] Using an infrared transmitter to emit infrared rays to the user's eyes, and using the infrared camera to collect infrared rays reflected by the cornea of ​​the user's eyes;

[0027] The sight angle of the user's eye projected onto the refrigerator surface is calculated based on the angle formed between the infrared ray emitted to the user's eye and the infrared ray reflected by the cornea of ​​the user's eye.

[0028] Among them, it can accurately determine the direction of the user's line of sight, so that the refrigerator can make an intelligent response based on the area the user is paying attention to, without the need for the user to manually operate the query, providing a more convenient and efficient interactive experience.

[0029] In some optional embodiments, selecting, based on the sight point trajectory, the stored items inside the refrigerator corresponding to the current sight point in the sight point trajectory includes:

[0030] Determine a sight area on the refrigerator surface that includes the current sight point according to the current sight point in the sight point trajectory and the direction of the current sight point in the sight point trajectory;

[0031] A target area inside the refrigerator corresponding to the visual area is determined, and the stored items in the target area are selected.

[0032] Among them, users can quickly obtain the storage items inside the refrigerator corresponding to the target area without manual search or recall, which greatly saves time and energy and makes the operation more convenient and efficient; at the same time, this precise correspondence and selection function makes the interaction between the user and the refrigerator more natural and intuitive, so that the refrigerator actively provides the corresponding three-dimensional image of the storage items according to the user's focus, enhancing the interactive experience between the user and the refrigerator, and improving the user's favorability and dependence on the product.

[0033] In some optional embodiments, determining the target area inside the refrigerator corresponding to the sight area includes:

[0034] For the internal area of ​​the refrigerator, the space area enclosed by the extension line of the edge of the visual area passing through the position point of the user's eyes is used as the target area.

[0035] Among them, by determining the target area inside the refrigerator, only the stored items in the target area are displayed, and the stored items in the user's visual blind spot are not displayed. This is more in line with the actual viewing effect of the user's eyes, thereby improving the user's visual authenticity.

[0036] In some optional embodiments, forming the stereoscopic image of the stored items and displaying the stereoscopic image on a display screen on the refrigerator includes:

[0037] Controlling the binocular camera group to collect a stereo image of the stored items;

[0038] The stereoscopic image is displayed through a display screen on the refrigerator.

[0039] Among them, by displaying a three-dimensional image of the items stored in the refrigerator on the display screen on the refrigerator, it is possible to improve the clarity of the image of the item while improving the user's visual perception and viewing experience, reduce the difficulty of identifying the item, improve the accuracy of identification and display, and enhance the user's experience in managing the items stored in the refrigerator. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0041] Figure 1 It is a flowchart of an image display method provided by an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of the principle of confirming the position of the user's eyes relative to the surface of the refrigerator provided by an embodiment of the present application;

[0043] Figure 3is a schematic diagram of the cornea and pupil of a user's eye provided by an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of infrared reflections from the cornea and retina of a user's eye provided by an embodiment of the present application;

[0045] Figure 5 It is a schematic diagram of the directions of different sight point trajectories and their corresponding target areas provided by the embodiments of the present application;

[0046] Figure 6 is a schematic diagram of a user's visual blind spot provided by an embodiment of the present application;

[0047] Figure 7 is a structural schematic diagram of an image display system provided in an embodiment of the present application;

[0048] Figure 8 It is a module schematic diagram of an image display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0050] In order to solve the technical problem that the above-mentioned prior art method of photographing items stored in the refrigerator results in low clarity of the item image, great difficulty in item identification, low identification and display accuracy, and poor user experience of refrigerator storage item management, the present invention proposes a method for displaying an image. The implementation details of a method for displaying an image of this embodiment are specifically described below. The following content is only the implementation details provided for ease of understanding and is not necessary for implementing this solution.

[0051] Embodiment 1:

[0052] The image display method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. The specific process can be as follows: Figure 1 As shown, including:

[0053] Step 101, detecting a sight point trajectory formed by the user's sight projected on the surface of the refrigerator;

[0054] Step 102, based on the sight point trajectory, selecting the storage items inside the refrigerator corresponding to the current sight point in the sight point trajectory;

[0055] Step 103: forming a three-dimensional image of the stored items, and displaying the three-dimensional image on a display screen on the refrigerator.

[0056] In this embodiment, the sight point trajectory formed by the user's sight projected on the refrigerator surface is detected; based on the sight point trajectory, the stored items inside the refrigerator corresponding to the current sight point in the sight point trajectory are selected; a three-dimensional image of the stored items is formed, and the three-dimensional image is displayed on the display screen on the refrigerator. It can improve the clarity of the item image, reduce the difficulty of item recognition, and improve the recognition display accuracy, providing users with a simple, direct, and accurate intelligent display of refrigerator storage items, thereby improving the user's refrigerator storage item management experience and user perception.

[0057] Embodiment 2:

[0058] The image display method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. This embodiment specifically describes the implementation details of steps 101 to 103 in the above-mentioned embodiment 1. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0059] Step 101, detecting a trajectory of a user's sight point projected onto a refrigerator surface.

[0060] Here, it should be noted that before step 101, the method further includes: when the refrigerator is in a normal working state, when it is detected that a user appears in the monitoring range, step 101 is started. The monitoring range may be 180 degrees from the front of the refrigerator.

[0061] Specifically, in some embodiments, step 101 may include:

[0062] Step 1011, detecting the position of the user's eyes relative to the surface of the refrigerator;

[0063] Step 1012, using an infrared device to collect the sight angle of the user's eyes projected on the refrigerator surface;

[0064] Step 1013, based on the position of the user's eyes and the sight angle, calculate the sight point where the user's sight is projected on the refrigerator surface, and form a sight point trajectory.

[0065] Furthermore, in some embodiments, step 1011 may include:

[0066] Step 10111, establishing an X-axis along the width direction of the front of the refrigerator, establishing a Y-axis along the height direction of the front of the refrigerator, and establishing a Z-axis along the direction perpendicular to the front of the refrigerator, to form a spatial coordinate system for describing the spatial position relationship between the user's eyes and the surface of the refrigerator;

[0067] Step 10112, taking the coordinate point of the user's eye in the spatial coordinate system as the position point of the user's eye relative to the surface of the refrigerator.

[0068] Furthermore, in some embodiments, step 1012 may include:

[0069] Step 10121, using an infrared transmitter to emit infrared rays to the user's eyes, and using an infrared camera to collect infrared rays reflected by the cornea of ​​the user's eyes;

[0070] Step 10122, calculate the sight angle of the user's eyes projected onto the surface of the refrigerator based on the angle formed between the infrared rays emitted to the user's eyes and the infrared rays reflected by the cornea of ​​the user's eyes.

[0071] like Figure 2 As shown, in a specific implementation, in an optional embodiment, the process of determining the sight angle of the user's eyes projected on the refrigerator surface may include:

[0072] The connecting line between the first infrared emitter and the second infrared emitter arranged in the width extension direction of the refrigerator surface is set as the X-axis, and the connecting line between the second infrared emitter and the third infrared emitter arranged in the height extension direction of the refrigerator surface is set as the Y-axis. The X-axis is set perpendicular to the extension direction of the refrigerator surface to form a spatial coordinate system for characterizing the spatial position relationship between the user's eyes and the refrigerator surface, and the intersection of the X-axis and the Y-axis is the origin 0 of the spatial coordinate system.

[0073] The coordinate point of the first infrared emitter in the space coordinate system on the surface of the refrigerator is (a, 0), the coordinate point of the second infrared emitter in the space coordinate system on the surface of the refrigerator is (-a, 0), the coordinate point of the third infrared emitter in the space coordinate system on the surface of the refrigerator is (b, 0), and the coordinate point of the third infrared emitter in the space coordinate system on the surface of the refrigerator is (-b, 0);

[0074] The distances from the first, second, third and fourth infrared emitters to the cornea of ​​the user's eye are measured respectively by infrared ranging, so as to calculate the coordinate point (c, d) of the user's cornea relative to the refrigerator surface in the spatial coordinate system;

[0075] The infrared ray is emitted to the user's eyes by the second infrared emitter, and the infrared ray is reflected to point a by the cornea of ​​the user's eyes. According to the angle ∠a formed between the infrared ray emitted to the user's eyes and the infrared ray reflected by the cornea of ​​the user's eyes, The sight angle θ of the user's eyes projected on the refrigerator surface is calculated.

[0076] It should be noted that if Figure 3 and Figure 4 As shown, due to the physiological structure and physical properties of the user's eyeball, under the premise that the relative positions of the light source and the head remain unchanged, the light spot formed by the corneal reflection of the user's eye will not move, and the direction of the light reflected on the retina indicates the direction of the pupil. Since the infrared light emitted by the infrared transmitter enters the pupil of the user's eye, the infrared light reflected by the retina is emitted from the pupil, and the angle between the cornea of ​​the user's eye and the light reflected by the pupil is collected by the infrared receiving camera, the direction of the user's eye movement can be calculated according to the angle between the cornea of ​​the user's eye and the light reflected by the pupil, thereby realizing the line of sight tracking of the user's eye. In another optional implementation, the connecting line between the cornea of ​​the user's eye and the origin of the spatial coordinate system is used as the reference line of sight, and the actual line of sight angle of the user's eye relative to the surface of the refrigerator can also be obtained by translation according to the reference line of sight.

[0077] Among them, the position of the user's eyes relative to the refrigerator plane can be obtained by taking pictures with a camera and using an infrared transmitter to emit infrared rays toward the user's eyes, and the infrared rays reflected back from the cornea of ​​the user's eyes can be collected by a camera. According to the positional relationship of the infrared transmitters at different positions and the angles of the emitted and received infrared rays, the current user's line of sight angle can be obtained, thereby completing the initialization calibration of the user's line of sight monitoring.

[0078] The step 102 is to select, based on the sight point trajectory, the storage items inside the refrigerator corresponding to the current sight point in the sight point trajectory.

[0079] Specifically, in some embodiments, step 102 may include:

[0080] Step 1021, determining a sight area on the refrigerator surface including the current sight point according to the current sight point in the sight point trajectory and the direction of the current sight point in the sight point trajectory;

[0081] Here, the user's human eyes project different sight point trajectories, and the sight areas presented to the user are different. For example, the storage items seen by the user's human eyes are as follows: Figure 5 When the point in Figure 5 As shown by the dotted line in , when the trajectory of the user's eye projection is from top to bottom, the sight area presented to the user may be the area range shown in the dotted circle, such as Figure 5 As shown by the dotted line in , when the trajectory of the sight point projected by the user's eyes is from bottom to top, the sight area presented to the user's eyes may be the area range shown in the dotted circle.

[0082] Step 1022, determine the target area inside the refrigerator corresponding to the visual area, and select the stored items in the target area.

[0083] Further, in some embodiments, in step 1022, determining the target area inside the refrigerator corresponding to the sight area may include:

[0084] Step 10221, for the internal area of ​​the refrigerator, the space area enclosed by the extension line of the edge of the visual area passing through the position point of the user's eyes is used as the target area.

[0085] It should be noted that, in the present embodiment, the target area is the space area enclosed by the extension line of the edge of the visual area passing through the position point of the user's human eye, that is, the target area is the visible area of ​​the user's human eye, and only the stereoscopic image of the stored items in the internal area of ​​the refrigerator corresponding to the visible area of ​​the user's human eye is displayed. However, due to the limitation of the visible range of the user's human eye, there is a visual blind spot of the user's human eye. In some optional embodiments, the stereoscopic image of the stored items in the entire area of ​​the refrigerator can be displayed. In this case, in order to make the display effect of the stereoscopic image of the stored items closer to the real visual effect of the user's human eye, the stereoscopic image of the stored items in the internal area of ​​the refrigerator corresponding to the visual blind spot of the user's human eye can be hidden. This can be achieved in the following way:

[0086] like Figure 6 As shown, if the coordinate point of the user's cornea in the spatial coordinate system of the refrigerator surface is (x, y), the horizontal distance from the user's cornea to the first side edge of the refrigerator surface is xx, the horizontal distance from the user's cornea to the second side edge of the refrigerator surface is x--x, and the vertical distance is y, the angle θ1 between the user's cornea and the first side edge of the refrigerator and the angle θ2 between the user's cornea and the second side edge of the refrigerator can be calculated. When θ1<θ2<90°, the area inside the refrigerator based on the angle (180°-θ1) of the first side edge of the refrigerator surface is determined as the visual blind spot of the user's eye; when θ2<θ1<90°, the area inside the refrigerator based on the angle (180°-θ2) of the second side edge of the refrigerator surface is determined as the visual blind spot of the user's eye.

[0087] The stereoscopic image of the stored items within the area within the angle (180°-θ1) of the first side edge of the refrigerator surface or the area within the angle (180°-θ2) of the second side edge of the refrigerator surface is hidden and then displayed on the refrigerator display screen.

[0088] Among them, the three-dimensional image of the stored items inside the refrigerator at the corresponding angle can be hidden according to the visual blind spot formed by the angle between the user's line of sight and the edges of the two sides of the refrigerator. When the visual blind spot of the user's human eye changes, the hidden image also changes accordingly, so that the user's perception of the image when looking at it achieves the same visual effect as looking at the real scene, which is closer to the actual perception of the user's human eye.

[0089] The step 103 forms a three-dimensional image of the stored items and displays the three-dimensional image on a display screen on the refrigerator.

[0090] Specifically, in some embodiments, step 103 may include:

[0091] Step 1031, controlling the binocular camera group to collect a stereo image of the stored items;

[0092] Step 1032: Display the stereoscopic image via the display screen on the refrigerator.

[0093] Among them, the content displayed on the display screen of the refrigerator is a three-dimensional image of the stored items collected by the internal collection unit.

[0094] Embodiment three:

[0095] The third embodiment of the present application provides an image display system. The structure of the image display system described in the third embodiment is described in detail below. The following content is only for the convenience of understanding the visual details provided and is not necessary for the implementation of this solution. Figure 7 As shown, an image display system may include: an acquisition module, a core processing module, a display module and a power supply module;

[0096] The acquisition module may specifically include: an internal acquisition unit and an external acquisition unit;

[0097] The internal acquisition unit is arranged on the inner wall of the refrigerator door, and is used to obtain image information of items stored inside the refrigerator. The internal acquisition unit may be composed of a binocular zoom camera group, and a stereoscopic image close to that obtained by human binocular eyes can be obtained through the binocular zoom camera group.

[0098] The external collection unit can be composed of an infrared transmitter group and an infrared camera, which can sense and collect the visual information of human eyes. The infrared camera can be set in the center of the top of the front of the refrigerator. Figure 7 As shown by the curve in the figure, the infrared emitter can be set at the center of the four sides of the front of the refrigerator, as shown in the figure. Figure 7 As shown in the position indicated by the dotted line in the figure; the infrared transmitter is used to monitor and collect the sight information of the user and the user's eyes within a certain distance and angle range of the display screen;

[0099] The core processing module is used to analyze and process the three-dimensional image information of the stored items provided by the internal acquisition unit and the human eye sight information provided by the external acquisition unit, thereby controlling the display information and the on / off status of the display screen;

[0100] The display module is composed of one or more liquid crystal screens to form a liquid crystal combination display panel. Each screen integrates an infrared receiver for receiving infrared rays reflected by the human eye and displaying the information of the stored items inside the refrigerator processed by the core processing module. The liquid crystal combination display panel can be set on the outside of the refrigerator door. Figure 6 As shown in the position indicated by the dotted line in the figure, in a specific implementation, the splicing structure of the display module depends on the refrigerator storage compartment structure and the refrigerator door structure, that is, each refrigerator storage compartment has one or more corresponding liquid crystal display panels;

[0101] The power supply module is used to supply power to the above functional modules.

[0102] It should be noted that the installation structure of the above-mentioned modules only provides an installation method. In specific implementation, the installation structure of each module is not limited by the installation method provided in this embodiment. It can be adaptively adjusted according to the actual refrigerator structure and the actual needs of collection and detection to make the collection and detection of the equipment more accurate and efficient.

[0103] In this embodiment, the image display system can improve the clarity of the object image, reduce the difficulty of object recognition, improve the recognition and display accuracy, and enhance the user's refrigerator storage item management experience.

[0104] Embodiment 4:

[0105] This embodiment provides an optimal implementation method of an image display method, and specifically describes the implementation details of the image display method in combination with the image display system as described in the above-mentioned embodiment three. The following content is only provided for the convenience of understanding the implementation details and is not necessary for implementing this solution.

[0106] In some optional embodiments, a method for displaying an image may include:

[0107] The power supply module supplies power to the refrigerator, so that all modules of the refrigerator start to work normally.

[0108] The external collection unit continuously monitors whether there is a user within the preset distance and preset angle range based on the refrigerator display screen. If so, the infrared transmitter group of the internal collection unit starts working and cooperates with the infrared camera of the external collection unit to initialize the user's line of sight based on the cornea-pupil reflex. Otherwise, no subsequent action is performed and the infrared camera of the external collection unit continues to collect information.

[0109] Specifically, when the user appears within 180° of the front of the refrigerator display, the infrared emitter group starts working, and cooperates with the infrared camera of the external collection unit to complete the user's line of sight initialization based on the cornea-pupil reflection. The infrared emitter emits infrared rays toward the user's eyes and uses the camera to collect the infrared rays reflected back by the cornea. Under the premise that the relative position of the infrared emitter and the user's head remains unchanged, the light spot formed by the corneal reflection of the user's eye will not move. According to the positional relationship of the infrared emitters at different positions and the angles of the emitted and received infrared rays, the current user's line of sight angle can be obtained, thereby completing the initialization calibration of the user's line of sight monitoring.

[0110] After completing the initialization calibration, the external acquisition unit collects the user's current line of sight information based on the cornea-pupil reflection and feeds it back to the core processing module. Due to the physiological structure and physical properties of the human eye, under the premise that the relative position of the infrared emitter and the user's head remains unchanged, the light spot formed by the corneal reflection of the user's eye will not move, and the direction of the infrared reflected on the retina indicates the direction of the pupil, that is, the infrared light enters the pupil of the user's eye, and the infrared light reflected by the retina is emitted from the pupil. According to the angle between the infrared light reflected by the cornea and the pupil, the direction of the user's eye movement can be calculated, thereby realizing human eye line tracking.

[0111] Specifically, the reflected infrared rays of the cornea and pupil are collected by an infrared camera. The spatial coordinates of the cornea and the reflection angle of the infrared rays obtained by initializing the sight angle of the user's eyes can be used to obtain the coordinates of the user's sight line on the surface of the refrigerator in the spatial coordinate system when the user's eyes are looking straight ahead. The reflected infrared rays of the pupil and its offset relative to the reflected light of the cornea can be used to obtain the offset of the user's current sight line relative to the straight ahead sight line, thereby obtaining the sight line coordinates, and then realizing the sight tracking of the user's eyes.

[0112] The internal acquisition unit collects stereo images of the storage compartments through the binocular camera group in each storage compartment and feeds them back to the core processing module; the core processing module analyzes the angle between the user's line of sight and the edges of the display screen based on the collected user's line of sight, and can obtain the user's visual blind spot for the storage compartment. According to the visual blind spot, the collected stereo images of the stored items in the storage compartment are hidden and transmitted to the refrigerator's display screen.

[0113] Specifically, if the coordinate point of the user's cornea in the spatial coordinate system of the refrigerator surface is (x, y), the horizontal distance from the user's cornea to the first side edge of the refrigerator surface is xx, the horizontal distance from the user's cornea to the second side edge of the refrigerator surface is x--x, and the vertical distance is y, the angle θ1 between the user's cornea and the first side edge of the refrigerator and the angle θ2 between the user's cornea and the second side edge of the refrigerator can be calculated. When θ1<θ2<90°, the inside of the refrigerator is based on the first side edge of the refrigerator surface. The area within the angle (180°-θ1) of the first side edge of the refrigerator surface is determined as the visual blind spot of the user's eyes; when θ2<θ1<90°, the area inside the refrigerator within the angle (180°-θ2) of the second side edge of the refrigerator surface is determined as the visual blind spot of the user's eyes; the stereoscopic image of the stored items within the area within the angle (180°-θ1) of the first side edge of the refrigerator surface or the area within the angle (180°-θ2) of the second side edge of the refrigerator surface is hidden and then displayed on the refrigerator display screen.

[0114] The display screen is a LCD combination display screen. When the user's line of sight is tracked within the display screen area, a three-dimensional image processed by the core processing unit will be displayed. This can simulate the effect of the user actually viewing the storage compartment and give the user a more realistic visual effect.

[0115] Each LCD screen of the display screen is an independent display, that is, when one or more refrigerator doors are opened, the corresponding real scene can still form a nearly complete image with the simulated three-dimensional images displayed by other LCD screens, thereby reducing the difference between the virtual and real changes of the image and achieving a visual effect as close to reality as possible, thereby improving the user's visual experience.

[0116] In this embodiment, the image display method can improve the clarity of object images, reduce the difficulty of object identification, and improve the accuracy of identification and display, providing users with a simple, direct, and accurate intelligent display of refrigerator storage items, thereby improving the user's refrigerator storage item management experience and user perception.

[0117] Embodiment five:

[0118] Another embodiment of the present application relates to an image display device 800. The implementation details of the image display device 800 of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details, and is not necessary for implementing this solution. The schematic diagram of the image display device 800 of this embodiment can be as follows Figure 8 As shown, it includes: a detection module 801, a selection module 802 and a display module 803.

[0119] The detection module 801 is used to detect the trajectory of the user's sight point projected on the refrigerator surface;

[0120] A selection module 802 is used to select, based on the sight point trajectory, the stored items inside the refrigerator corresponding to the current sight point in the sight point trajectory;

[0121] The display module 803 is used to form a three-dimensional image of the stored items and display the three-dimensional image through a display screen on the refrigerator.

[0122] In some optional embodiments, the step of detecting a sight point trajectory formed by the user's sight projected on the surface of the refrigerator includes:

[0123] Detecting the position of the user's eyes relative to the surface of the refrigerator;

[0124] Use infrared equipment to collect the user's line of sight angle projected on the refrigerator surface;

[0125] According to the position point of the user's eye and the sight angle, the sight point where the user's sight is projected on the refrigerator surface is calculated, and a sight point trajectory is formed.

[0126] In some optional embodiments, detecting the position of the user's eyes relative to the surface of the refrigerator includes:

[0127] An X-axis is established along the width direction of the front of the refrigerator, a Y-axis is established along the height direction of the front of the refrigerator, and a Z-axis is established along a direction perpendicular to the front of the refrigerator to form a spatial coordinate system for describing the spatial position relationship between the user's eyes and the surface of the refrigerator;

[0128] The coordinate point of the user's eye in the spatial coordinate system is used as the position point of the user's eye relative to the surface of the refrigerator.

[0129] In some optional embodiments, the step of using an infrared device to collect the sight angle of the user's eyes projected on the refrigerator surface includes:

[0130] Using an infrared transmitter to emit infrared rays to the user's eyes, and using the infrared camera to collect infrared rays reflected by the cornea of ​​the user's eyes;

[0131] The sight angle of the user's eye projected onto the refrigerator surface is calculated based on the angle formed between the infrared ray emitted to the user's eye and the infrared ray reflected by the cornea of ​​the user's eye.

[0132] In some optional embodiments, selecting, based on the sight point trajectory, the stored items inside the refrigerator corresponding to the current sight point in the sight point trajectory includes:

[0133] Determine a sight area on the refrigerator surface that includes the current sight point according to the current sight point in the sight point trajectory and the direction of the current sight point in the sight point trajectory;

[0134] A target area inside the refrigerator corresponding to the visual area is determined, and the stored items in the target area are selected.

[0135] In some optional embodiments, determining the target area inside the refrigerator corresponding to the sight area includes:

[0136] For the internal area of ​​the refrigerator, the space area enclosed by the extension line of the edge of the visual area passing through the position point of the user's eyes is used as the target area.

[0137] In some optional embodiments, forming the stereoscopic image of the stored items and displaying the stereoscopic image on a display screen on the refrigerator includes:

[0138] Controlling the binocular camera group to collect a stereo image of the stored items;

[0139] The stereoscopic image is displayed through a display screen on the refrigerator.

[0140] In this embodiment, the image processing device can detect the sight point trajectory formed by the user's sight projected on the refrigerator surface; based on the sight point trajectory, select the storage items inside the refrigerator corresponding to the current sight point in the sight point trajectory; form a three-dimensional image of the storage items, and display the three-dimensional image through the display screen on the refrigerator. It can improve the clarity of the item image, reduce the difficulty of item recognition, improve the recognition display accuracy, and provide users with a simple, direct, and accurate intelligent display of refrigerator storage items, thereby improving the user's refrigerator storage item management experience and user perception.

[0141] It should be noted that the image display device described in this embodiment is an image display device corresponding to the image display method in the above-mentioned embodiment. All implementation methods in the embodiment of the above-mentioned image display method are applicable to the embodiment of the image display device and can achieve the same effect.

[0142] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0143] Embodiment six:

[0144] Another embodiment of the present application relates to an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image display method in the above-mentioned embodiments.

[0145] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0146] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0147] Embodiment seven:

[0148] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0149] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0150] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for displaying an image, characterized in that: include: Detect the trajectory of the user's sight points projected on the refrigerator surface; Based on the sight point trajectory, selecting the storage items inside the refrigerator corresponding to the current sight point in the sight point trajectory; A three-dimensional image of the stored items is formed, and the three-dimensional image is displayed on a display screen on the refrigerator.

2. The image display method according to claim 1, characterized in that: The detecting of the sight point trajectory formed by the user's sight projected on the refrigerator surface includes: Detecting the position of the user's eyes relative to the surface of the refrigerator; Use infrared equipment to collect the user's line of sight angle projected on the refrigerator surface; According to the position point of the user's eye and the sight angle, the sight point where the user's sight is projected on the refrigerator surface is calculated, and a sight point trajectory is formed.

3. The image display method according to claim 2, characterized in that: The detecting the position of the user's eyes relative to the surface of the refrigerator includes: An X-axis is established along the width direction of the front of the refrigerator, a Y-axis is established along the height direction of the front of the refrigerator, and a Z-axis is established along a direction perpendicular to the front of the refrigerator to form a spatial coordinate system for describing the spatial position relationship between the user's eyes and the surface of the refrigerator; The coordinate point of the user's eye in the spatial coordinate system is used as the position point of the user's eye relative to the surface of the refrigerator.

4. The image display method according to claim 3, characterized in that: The method of using an infrared device to collect the sight angle of the user's eyes projected on the refrigerator surface includes: Using an infrared transmitter to emit infrared rays to the user's eyes, and using the infrared camera to collect infrared rays reflected by the cornea of ​​the user's eyes; The sight angle of the user's eye projected onto the refrigerator surface is calculated based on the angle formed between the infrared ray emitted to the user's eye and the infrared ray reflected by the cornea of ​​the user's eye.

5. The image display method according to claim 1, characterized in that: The selecting, based on the sight point trajectory, the storage item inside the refrigerator corresponding to the current sight point in the sight point trajectory includes: Determine a sight area on the refrigerator surface that includes the current sight point according to the current sight point in the sight point trajectory and the direction of the current sight point in the sight point trajectory; A target area inside the refrigerator corresponding to the visual area is determined, and the stored items in the target area are selected.

6. The image display method according to claim 5, characterized in that: The determining of the target area inside the refrigerator corresponding to the sight area includes: For the internal area of ​​the refrigerator, the space area enclosed by the extension line of the edge of the visual area passing through the position point of the user's eyes is used as the target area.

7. The image display method according to claim 1, characterized in that: The forming of the three-dimensional image of the stored items and displaying the three-dimensional image through a display screen on the refrigerator comprises: Controlling the binocular camera group to collect a stereo image of the stored items; The stereoscopic image is displayed through a display screen on the refrigerator.

8. An image display device, characterized in that: include: A detection module, used to detect the trajectory of the user's sight points projected on the refrigerator surface; A selection module, configured to select, based on the sight point trajectory, a storage item inside the refrigerator corresponding to a current sight point in the sight point trajectory; The display module is used to form a three-dimensional image of the stored items and display the three-dimensional image through a display screen on the refrigerator.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image display method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image display method according to any one of claims 1 to 7 is implemented.

Images