Image display method and device, electronic equipment and storage medium

Abstract

The embodiment of the application relates to the field of intelligent device information processing, and discloses an image display method and device, electronic equipment and a storage medium, wherein the image display method comprises the following steps: detecting a visual line drop point track formed by a visual line of a user projected on a refrigerator surface; selecting, based on the visual line drop point track, a storage article in the refrigerator corresponding to a current visual line drop point in the visual line drop point track; forming a stereoscopic image of the storage article; and displaying the stereoscopic image on a display screen on the refrigerator. The technical scheme disclosed in the application solves the problem of the low image definition of the storage article of the refrigerator, the high difficulty of article recognition, the low recognition and display accuracy, and the poor user experience of the refrigerator storage article management in the prior art, and can improve the image definition of the storage article, reduce the difficulty of article recognition, improve the recognition and display accuracy, and improve the user experience of the refrigerator storage article management.

Description

Technical Field

[0001] This invention relates to the field of information processing for intelligent devices, and particularly to an image display method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous progress and development of technology, intelligentization has become an inevitable trend in the development of home appliances. With the popularization of smart refrigerators and the emergence of large-screen refrigerators, refrigerators are no longer just cabinets for refrigerating food, but have been endowed with richer functions. However, the most important function of a refrigerator is still to store and manage food. Therefore, the intelligent management of stored items in a refrigerator is an important component of refrigerator intelligence.

[0003] The larger a user's refrigerator, the more food it holds, making it increasingly difficult for the user to remember the contents. Current technology addresses this by installing a camera in the refrigerator to take photos of the items. However, this method suffers from low image clarity, difficulty in item recognition, and low accuracy in displaying information, resulting in a poor user experience in managing their refrigerator's contents. Summary of the Invention

[0004] The purpose of this invention is to provide at least one image display method, device, electronic device, and storage medium, which can at least solve the problems of low image clarity, high difficulty in object recognition, low recognition and display accuracy, and poor user experience in refrigerator storage management caused by the existing methods of photographing refrigerator stored items. The invention can at least improve the image clarity, reduce the difficulty of object recognition, improve the recognition and display accuracy, and improve the user's refrigerator storage management experience.

[0005] To address the aforementioned technical problems, at least one embodiment of this application provides a method for displaying an image, comprising:

[0006] Detects the trajectory of the user's gaze as it falls on the surface of the refrigerator.

[0007] Based on the trajectory of the line of sight, select the stored items inside the refrigerator that correspond to the current line of sight in the trajectory of the line of sight;

[0008] A three-dimensional image of the stored items is formed and displayed on the display screen on the refrigerator.

[0009] At least one embodiment of this application also provides an image display device, comprising:

[0010] The detection module is used to detect the trajectory of the user's gaze as it falls on the surface of the refrigerator.

[0011] The selection module is used to select the stored items inside the refrigerator that correspond to the current line of sight in the line of sight trajectory, based on the line of sight trajectory.

[0012] The display module is used to form a three-dimensional image of the stored items and display the three-dimensional image on the display screen on the refrigerator.

[0013] At least one embodiment of this application also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described image display method.

[0014] At least one embodiment of this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image display method.

[0015] This application provides an image display method that detects the trajectory of a user's gaze projected onto the surface of a refrigerator; based on the trajectory, selects stored items inside the refrigerator corresponding to the current point of gaze on the trajectory; forms a 3D image of the stored items; and displays the 3D image on a display screen on the refrigerator. This method improves image clarity, reduces the difficulty of item recognition, increases recognition accuracy, and enhances the user's refrigerator storage management experience.

[0016] In some optional embodiments, detecting the trajectory of the user's gaze projected onto the refrigerator surface includes:

[0017] Detect the position of the user's eye relative to the surface of the refrigerator;

[0018] Infrared equipment is used to collect the angle of the user's line of sight projected onto the surface of the refrigerator;

[0019] Based on the location of the user's eye and the angle of the line of sight, the point where the user's line of sight is projected onto the surface of the refrigerator is calculated, and the trajectory of the line of sight is formed.

[0020] In particular, by accurately knowing the location and trajectory of the user's eye on the refrigerator surface, the refrigerator's intelligent system can provide personalized interactive content based on the user's eye's trajectory, making the interaction between the user and the refrigerator more in line with their own needs and improving the convenience and fun of use.

[0021] In some optional embodiments, detecting the position of the user's eye relative to the surface of the refrigerator includes:

[0022] An X-axis is established along the width direction of the front of the refrigerator, a Y-axis is established along the height direction of the front of the refrigerator, and a Z-axis is established along the direction perpendicular to the front of the refrigerator, forming a spatial coordinate system for describing the spatial positional relationship between the user's eye and the surface of the refrigerator;

[0023] The coordinate point in the spatial coordinate system of the user's eye is taken as the position point of the user's eye relative to the surface of the refrigerator.

[0024] By establishing a spatial coordinate system, the position of the user's eye relative to the refrigerator surface can be accurately and quickly determined, thereby identifying the stored items inside the refrigerator corresponding to the user's line of sight. This allows for the display of a three-dimensional image of the stored items, improving the user's viewing experience.

[0025] In some optional embodiments, the use of an infrared device to collect the angle of the user's eye projection onto the refrigerator surface includes:

[0026] An infrared emitter emits infrared light into the user's eye, and an infrared camera captures the infrared light reflected from the cornea of ​​the user's eye.

[0027] The viewing angle of the user's eye projected onto the refrigerator surface is calculated based on the angle formed between the infrared light emitted towards the user's eye and the infrared light reflected by the user's cornea.

[0028] Among its features, it can accurately determine the user's line of sight, allowing the refrigerator to respond intelligently based on the area the user is focusing on, eliminating the need for manual operation and providing a more convenient and efficient interactive experience.

[0029] In some optional embodiments, selecting the stored items inside the refrigerator corresponding to the current line of sight trajectory based on the line of sight trajectory includes:

[0030] Based on the current point of view in the trajectory of the line of sight, and the direction of the current point of view in the trajectory of the line of sight, the line of sight area containing the current point of view on the surface of the refrigerator is determined.

[0031] Determine the target area inside the refrigerator corresponding to the line of sight area, and select the stored items in the target area.

[0032] Users can quickly find the items stored inside the refrigerator corresponding to the target area without having to manually search or recall, greatly saving time and effort and making the operation more convenient and efficient. At the same time, this precise matching and selection function makes the interaction between users and the refrigerator more natural and intuitive. The refrigerator can proactively provide a 3D image of the corresponding stored items based on the user's focus, enhancing the interactive experience between users and the refrigerator and increasing users' favorability and dependence on the product.

[0033] In some optional embodiments, determining the target area inside the refrigerator corresponding to the line-of-sight area includes:

[0034] For the interior area of ​​the refrigerator, the target area is defined as the spatial area enclosed by the extension line of the edge of the line of sight passing through the user's eye position.

[0035] By defining a target area inside the refrigerator, only the stored items within that target area are displayed, while items outside the user's visual blind spot are not. This better matches the actual visual perception of the user, thereby enhancing the visual realism.

[0036] In some optional embodiments, forming a stereoscopic image of the stored items and displaying the stereoscopic image on a display screen on the refrigerator includes:

[0037] Control the binocular camera array to acquire stereoscopic images of the stored items;

[0038] The stereoscopic image is displayed on the screen on the refrigerator.

[0039] Among these features, displaying 3D images of the items stored inside the refrigerator on a screen on the refrigerator's display panel can improve the clarity of the item images, enhance the user's visual experience, reduce the difficulty of item recognition, improve the accuracy of recognition and display, and improve the user's experience in managing the items stored in the refrigerator. Attached Figure Description

[0040] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0041] Figure 1 This is a schematic flowchart illustrating an image display method provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram illustrating the principle of confirming the position of a user's eye relative to the surface of the refrigerator, provided by an embodiment of this application.

[0043] Figure 3This is a schematic diagram of the cornea and pupil of a user's eye provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the cornea and retina of a user's eye reflecting infrared light, provided in an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the different line-of-sight trajectories and their corresponding target areas provided in the embodiments of this application;

[0046] Figure 6 This is a schematic diagram of the visual blind spot of the human eye provided in an embodiment of this application;

[0047] Figure 7 This is a schematic diagram of the structure of an image display system provided in an embodiment of this application;

[0048] Figure 8 This is a schematic diagram of a module of an image display device provided in an embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0050] To address the technical problems of low image clarity, difficulty in item recognition, low display accuracy, and poor user experience in refrigerator storage management caused by existing methods of photographing stored items, this invention proposes an image display method. The implementation details of this image display method are described below. These details are provided for ease of understanding and are not essential for implementing this solution.

[0051] Example 1:

[0052] The image display method of this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes:

[0053] Step 101: Detect the trajectory of the user's gaze as it falls on the surface of the refrigerator.

[0054] Step 102: Based on the line of sight trajectory, select the stored items inside the refrigerator that correspond to the current line of sight trajectory.

[0055] Step 103: Create a 3D image of the stored items and display the 3D image on the display screen on the refrigerator.

[0056] In this embodiment, the system detects the trajectory of the user's gaze projected onto the refrigerator surface; based on this trajectory, it selects the stored items inside the refrigerator corresponding to the current point of gaze within the trajectory; it then forms a 3D image of the stored items and displays this image on the refrigerator's display screen. This improves the clarity of the item image, reduces the difficulty of item recognition, and increases the accuracy of recognition and display, providing users with a simple, direct, and accurate intelligent display of refrigerator stored items, thereby enhancing the user's refrigerator storage management experience and overall user experience.

[0057] Example 2:

[0058] The image display method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. This embodiment provides a detailed description of the implementation details of steps 101 to 103 in the above embodiment one. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0059] Step 101: Detect the trajectory of the user's gaze as it falls on the surface of the refrigerator.

[0060] It should be noted that, prior to step 101, the process includes: when the refrigerator is operating normally, step 101 is initiated when a user is detected within the monitoring range. The monitoring range can be 180 degrees from the front of the refrigerator.

[0061] Specifically, in some embodiments, step 101 may include:

[0062] Step 1011: Detect the position of the user's eye relative to the surface of the refrigerator;

[0063] Step 1012: Use an infrared device to collect the angle of the user's eye projection onto the surface of the refrigerator;

[0064] Step 1013: Based on the user's eye position and viewing angle, calculate the point where the user's line of sight is projected onto the refrigerator surface and form the trajectory of the viewing point.

[0065] Furthermore, in some embodiments, step 1011 may include:

[0066] Step 10111: Establish an X-axis along the width direction of the front of the refrigerator, an Y-axis along the height direction of the front of the refrigerator, and a Z-axis along the direction perpendicular to the front of the refrigerator to form a spatial coordinate system for describing the spatial positional relationship between the user's eyes and the surface of the refrigerator.

[0067] Step 10112: Take the coordinates of the user's eye in the spatial coordinate system as the position of the user's eye relative to the surface of the refrigerator.

[0068] Furthermore, in some embodiments, step 1012 may include:

[0069] Step 10121: Use an infrared emitter to emit infrared rays into the user's eyes, and use an infrared camera to collect the infrared rays reflected by the cornea of ​​the user's eyes.

[0070] Step 10122: Calculate the viewing angle of the user's eye projected onto the refrigerator surface based on the angle formed between the infrared light emitted towards the user's eye and the infrared light reflected by the user's cornea.

[0071] like Figure 2 As shown, in a specific implementation, in one optional embodiment, the process of determining the viewing angle of the user's eye projected onto the refrigerator surface may include:

[0072] The connecting line between the first and second infrared emitters, which are set in the width extension direction of the refrigerator surface, is set as the X-axis. The connecting line between the second and third infrared emitters, which are set in the height extension direction of the refrigerator surface, is set as the Y-axis. The X-axis is set perpendicular to the extension direction of the refrigerator surface to form a spatial coordinate system for characterizing the spatial positional relationship between the user's eye and the refrigerator surface. The intersection of the X-axis and the Y-axis is the origin 0 of the spatial coordinate system.

[0073] The coordinates of the first infrared transmitter on the surface of the refrigerator are (a, 0), the coordinates of the second infrared transmitter on the surface of the refrigerator are (-a, 0), the coordinates of the third infrared transmitter on the surface of the refrigerator are (b, 0), and the coordinates of the third infrared transmitter on the surface of the refrigerator are (-b, 0).

[0074] Infrared ranging was used to measure the distances from the first, second, third, and fourth infrared emitters to the user's cornea, and the coordinates (c, d) of the user's cornea relative to the refrigerator surface in the spatial coordinate system were calculated.

[0075] Infrared rays are emitted towards the user's eye via a second infrared emitter. These rays are reflected by the user's cornea to point a. The angle ∠a formed between the emitted infrared ray and the reflected infrared ray is calculated based on... The angle θ of the user's eye projection onto the surface of the refrigerator is calculated.

[0076] It should be noted that, as Figure 3 and Figure 4 As shown, due to the physiological structure and physical properties of the user's eyeball, the light spot formed by the reflection of the cornea of ​​the user's eye will not move, provided that the relative position of the light source and the head remains unchanged. The direction of the light reflected on the retina indicates the orientation of the pupil. Since the infrared rays emitted by the infrared emitter enter the user's eye through the pupil and the infrared rays reflected by the retina exit the pupil, and the angle between the reflected light from the user's cornea and the pupil is collected by the infrared receiving camera, the direction of the user's eye movement can be calculated based on the angle between the reflected light from the user's cornea and the pupil, thereby realizing the tracking of the user's gaze. In another optional implementation, the line connecting the user's cornea to the origin of the spatial coordinate system is used as the reference line of sight. The angle of the user's actual line of sight relative to the surface of the refrigerator can also be obtained by translating the reference line of sight.

[0077] Specifically, the position of the user's eye relative to the refrigerator plane can be obtained by taking pictures with a camera and emitting infrared light towards the user's eye with an infrared emitter. The infrared light reflected back from the user's cornea is also collected by the camera. Based on the positional relationship of the infrared emitters at different locations and the angles of the emitted and received infrared light, the current viewing angle of the user can be obtained, thereby completing the initial calibration of the user's viewing angle monitoring.

[0078] In step 102, based on the trajectory of the line of sight, the stored items inside the refrigerator corresponding to the current line of sight trajectory are selected.

[0079] Specifically, in some embodiments, step 102 may include:

[0080] Step 1021: Based on the current line of sight trajectory and the direction of the current line of sight trajectory, determine the line of sight area on the refrigerator surface that includes the current line of sight.

[0081] Here, the trajectory of the user's gaze, projected in different directions, results in different visual areas for the user. For example, the stored items appear as different to the user's eyes. Figure 5 When the point is shown in the middle, as Figure 5 As shown by the dotted line, when the trajectory of the user's gaze is from top to bottom, the area of ​​vision presented to the user may be the region shown within the dotted line, such as... Figure 5 As shown by the dotted line, when the trajectory of the user's gaze is from bottom to top, the area of ​​vision presented to the user may be the region shown within the dotted circle.

[0082] Step 1022: Determine the target area inside the refrigerator corresponding to the line of sight area, and select the stored items in the target area.

[0083] Furthermore, in some embodiments, step 1022, determining the target area inside the refrigerator corresponding to the line-of-sight area, may include:

[0084] Step 10221: For the internal area of ​​the refrigerator, the target area is the space enclosed by the extension line of the edge of the line of sight passing through the user's eye position.

[0085] It should be noted that in this embodiment, the target area is the spatial area enclosed by the extension line of the edge of the user's eye's position point passing through the line of sight area. That is, the target area is the user's visible area. Only the stereoscopic image of the stored items in the refrigerator's interior area corresponding to the user's visible area is displayed. However, since the user's visible range is limited, there are blind spots. In some optional embodiments, stereoscopic images of the stored items in the entire refrigerator area can be displayed. In this case, in order to make the display effect of the stereoscopic image of the stored items closer to the user's real visual effect, the stereoscopic image of the stored items in the refrigerator's interior area corresponding to the user's blind spot can be hidden. Specifically, this can be achieved in the following way:

[0086] like Figure 6 As shown, if the coordinates of the user's cornea on the refrigerator surface are (x, y), the horizontal distance from the user's cornea to the first edge of the refrigerator surface is xx, the horizontal distance from the user's cornea to the second edge of the refrigerator surface is x-x, and the vertical distance is y, then the angle θ1 between the user's cornea and the first edge of the refrigerator, and the angle θ2 between the user's cornea and the second edge of the refrigerator can be calculated. When θ1 < θ2 < 90°, the area inside the refrigerator based on the angle (180° - θ1) of the first edge of the refrigerator surface is defined as the user's visual blind spot; when θ2 < θ1 < 90°, the area inside the refrigerator based on the angle (180° - θ2) of the second edge of the refrigerator surface is defined as the user's visual blind spot.

[0087] The 3D image of the stored items within the area within the angle (180°-θ1) of the first side edge of the refrigerator surface or the area within the angle (180°-θ2) of the second side edge of the refrigerator surface is hidden before being displayed on the refrigerator's screen.

[0088] Specifically, the system can hide the 3D images of the stored items inside the refrigerator at the corresponding angles formed by the user's line of sight and the angles of the refrigerator's sides. When the user's visual blind spot changes, the hidden image also changes, so that the user's visual experience when viewing the image is the same as viewing a real scene, thus more closely resembling the user's actual visual experience.

[0089] In step 103, a three-dimensional image of the stored items is formed and displayed on the display screen on the refrigerator.

[0090] Specifically, in some embodiments, step 103 may include:

[0091] Step 1031: Control the binocular camera group to acquire stereoscopic images of the stored items;

[0092] Step 1032: Display the stereoscopic image on the display screen on the refrigerator.

[0093] The display screen on the refrigerator shows 3D images of the stored items captured by the internal acquisition unit.

[0094] Example 3:

[0095] This application provides an image display system according to its third embodiment. The structure of the image display system described in this third embodiment will be specifically described below. The following details are for ease of understanding and are not essential for implementing this solution. Figure 7 As shown, an image display system may include: an acquisition module, a core processing module, a display module, and a power supply module;

[0096] Specifically, the acquisition module may include: an internal acquisition unit and an external acquisition unit;

[0097] The internal acquisition unit is located on the inner wall of the refrigerator door and is used to acquire image information of the items stored inside the refrigerator. It can be composed of a binocular zoom camera group, which can acquire stereo images that are close to those acquired by human binoculars.

[0098] The external acquisition unit can consist of an infrared transmitter group and an infrared camera, which can sense and collect information about human eye gaze. The infrared camera can be installed in the center of the top front of the refrigerator, as shown in the following figure. Figure 7 As indicated by the curve in the image, the infrared emitters can be positioned centered on the top, bottom, left, and right sides of the refrigerator's front, as shown in the image. Figure 7 The position indicated by the dotted line is shown; the infrared emitter is used to monitor and collect information on the user's gaze and the user's eyes within a certain distance and angle range of the display screen;

[0099] The core processing module is used to analyze and process the 3D image information of the stored items provided by the internal acquisition unit, as well as the line-of-sight information of the human eyes provided by the external acquisition unit; thereby controlling the display information and the brightness of the display screen.

[0100] The display module consists of one or more LCD screens, forming a combined LCD display panel. Each screen integrates an infrared receiver to receive infrared light reflected from the human eye and to display information about the stored items inside the refrigerator, processed by the core processing module. The combined LCD display panel can be installed on the outside of the refrigerator door, specifically as follows: Figure 6 As indicated by the dotted line, in actual implementation, the splicing structure of the display module depends on the refrigerator storage compartment structure and the refrigerator door structure, that is, each refrigerator storage compartment has one or more corresponding LCD display panels.

[0101] The power supply module is used to supply power to the above-mentioned functional modules.

[0102] It should be noted that the installation structure of each module described above is only one installation method. In actual implementation, the installation structure of each module is not limited to the installation method provided in this embodiment. It can be adaptively adjusted according to the actual refrigerator structure and the actual needs of data collection and detection, so as to make the data collection and detection of the device more accurate and efficient.

[0103] In this embodiment, the image display system can improve the clarity of the item image, reduce the difficulty of item recognition, improve the recognition and display accuracy, and enhance the user's experience in managing items stored in the refrigerator.

[0104] Example 4:

[0105] This embodiment provides an optimal implementation of an image display method, and specifically describes the implementation details of the image display method in conjunction with the image display system described in Embodiment 3 above. The following implementation details are provided only for ease of understanding and are not necessary for implementing this solution.

[0106] In some alternative embodiments, an image display method may include:

[0107] The power supply module supplies power to the refrigerator, enabling all modules to start working normally when the refrigerator is powered on.

[0108] The external acquisition unit continuously monitors whether there is a user within a preset distance and preset angle range based on the refrigerator display screen. If so, the infrared transmitter group of the internal acquisition unit starts working and works with the infrared camera of the external acquisition unit to initialize the user's line of sight based on corneal-pupil reflection. Otherwise, no further actions are taken, and the infrared camera of the external acquisition unit continues to collect information.

[0109] Specifically, when a user appears within a 180° range in front of the refrigerator display screen, the infrared transmitter group starts working. In conjunction with the infrared camera of the external acquisition unit, it completes the user's gaze initialization based on corneal-pupil reflection. The infrared transmitter emits infrared rays towards the user's eyes and the camera collects the infrared rays reflected back from the cornea. Under the premise that the relative positions of the infrared transmitter and the user's head remain unchanged, the light spot formed by the corneal reflection of the user's eyes will not move. Based on the positional relationship of the infrared transmitters at different locations and the angles of the emitted and received infrared rays, the current user's gaze angle can be obtained, thus completing the initial calibration of user gaze monitoring.

[0110] After initial calibration, the external acquisition unit collects the user's current gaze information based on corneal-pupil reflection and feeds it back to the core processing module. Due to the physiological structure and physical properties of the human eyeball, the light spot formed by the corneal reflection of the user's eye will not move, provided that the relative position of the infrared emitter and the user's head remains unchanged. The direction of the infrared light reflected on the retina indicates the orientation of the pupil. That is, infrared light enters from the user's pupil and infrared light reflected from the retina exits from the pupil. Based on the angle between the infrared light reflected by the cornea and the pupil, the direction of the user's eye movement can be calculated, thereby realizing eye gaze tracking.

[0111] Specifically, by collecting reflected infrared light from the cornea and pupil using an infrared camera, and using the spatial coordinates of the cornea and the reflected angle of the infrared light obtained by initializing the user's eye gaze angle, the coordinates of the user's line of sight on the refrigerator surface can be obtained. By using the reflected infrared light from the pupil and its offset relative to the reflected light from the cornea, the offset of the user's current line of sight relative to the eye gaze can be obtained, thus obtaining the line of sight coordinates, and thereby realizing the tracking of the user's eye gaze.

[0112] The internal acquisition unit captures stereoscopic images of each storage compartment using a binocular camera array and feeds them back to the core processing module. The core processing module analyzes the angle between the user's line of sight and the two edges of the display screen based on the acquired user's line of sight, thereby obtaining the user's visual blind spot of the storage compartment. Based on the visual blind spot, the stereoscopic images of the items stored in the compartment are hidden and then transmitted to the refrigerator's display screen.

[0113] Specifically, if the coordinates of the user's cornea on the refrigerator surface are (x, y), the horizontal distance from the user's cornea to the first edge of the refrigerator surface is xx, the horizontal distance from the user's cornea to the second edge of the refrigerator surface is x-x, and the vertical distance is y, then the angle θ1 between the user's cornea and the first edge of the refrigerator, and the angle θ2 between the user's cornea and the second edge of the refrigerator can be calculated. When θ1 < θ2 < 90°, the interior of the refrigerator is positioned based on the first edge of the refrigerator surface. The area within the angle (180°-θ1) of the first edge of the refrigerator surface is defined as the user's visual blind spot. When θ2<θ1<90°, the area within the refrigerator interior based on the angle (180°-θ2) of the second edge of the refrigerator surface is defined as the user's visual blind spot. The stereoscopic images of stored items within the area within the angle (180°-θ1) of the first edge of the refrigerator surface or the area within the angle (180°-θ2) of the second edge of the refrigerator surface are hidden before being displayed on the refrigerator's screen.

[0114] The display screen is a combination LCD display. When the user's gaze falls on the display area, it will display a stereoscopic image processed by the core processing unit, which can simulate the effect of the user actually viewing the storage compartment and give the user a more realistic visual effect.

[0115] Each LCD panel of the display is displayed independently. This means that when one or more refrigerator doors are opened, the corresponding real scene can still form a nearly complete image with the simulated 3D images displayed on other LCD panels. This reduces the difference between the real and virtual images and achieves a visual effect that is as close to real as possible, thereby improving the user's visual experience.

[0116] In this embodiment, the image display method can improve the clarity of the item image, reduce the difficulty of item recognition, and improve the accuracy of recognition and display. It provides users with a simple, direct, and accurate intelligent display of refrigerator storage items, thereby improving the user's refrigerator storage item management experience and user perception.

[0117] Example 5:

[0118] Another embodiment of this application relates to an image display device 800. The implementation details of the image display device 800 of this embodiment are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution. The schematic diagram of the image display device 800 of this embodiment can be seen as follows: Figure 8 As shown, it includes: a detection module 801, a selection module 802, and a display module 803.

[0119] Detection module 801 is used to detect the trajectory of the user's line of sight projected onto the surface of the refrigerator;

[0120] The selection module 802 is used to select the stored items inside the refrigerator that correspond to the current line of sight in the line of sight trajectory based on the line of sight trajectory.

[0121] Display module 803 is used to form a three-dimensional image of the stored items and display the three-dimensional image on the display screen on the refrigerator.

[0122] In some optional embodiments, detecting the trajectory of the user's gaze projected onto the refrigerator surface includes:

[0123] Detect the position of the user's eye relative to the surface of the refrigerator;

[0124] Infrared equipment is used to collect the angle of the user's line of sight projected onto the surface of the refrigerator;

[0125] Based on the location of the user's eye and the angle of the line of sight, the point where the user's line of sight is projected onto the surface of the refrigerator is calculated, and the trajectory of the line of sight is formed.

[0126] In some optional embodiments, detecting the position of the user's eye relative to the surface of the refrigerator includes:

[0127] An X-axis is established along the width direction of the front of the refrigerator, a Y-axis is established along the height direction of the front of the refrigerator, and a Z-axis is established along the direction perpendicular to the front of the refrigerator, forming a spatial coordinate system for describing the spatial positional relationship between the user's eye and the surface of the refrigerator;

[0128] The coordinates of the user's eye in the spatial coordinate system are taken as the position of the user's eye relative to the surface of the refrigerator.

[0129] In some optional embodiments, the use of an infrared device to collect the angle of the user's eye projection onto the refrigerator surface includes:

[0130] An infrared emitter emits infrared light into the user's eye, and an infrared camera captures the infrared light reflected from the cornea of ​​the user's eye.

[0131] The viewing angle of the user's eye projected onto the refrigerator surface is calculated based on the angle formed between the infrared light emitted towards the user's eye and the infrared light reflected by the user's cornea.

[0132] In some optional embodiments, selecting the stored items inside the refrigerator corresponding to the current line of sight trajectory based on the line of sight trajectory includes:

[0133] Based on the current point of view in the trajectory of the line of sight, and the direction of the current point of view in the trajectory of the line of sight, the line of sight area containing the current point of view on the surface of the refrigerator is determined.

[0134] Determine the target area inside the refrigerator corresponding to the line of sight area, and select the stored items in the target area.

[0135] In some optional embodiments, determining the target area inside the refrigerator corresponding to the line-of-sight area includes:

[0136] For the interior area of ​​the refrigerator, the target area is defined as the spatial area enclosed by the extension line of the edge of the line of sight passing through the user's eye position.

[0137] In some optional embodiments, forming a stereoscopic image of the stored items and displaying the stereoscopic image on a display screen on the refrigerator includes:

[0138] Control the binocular camera array to acquire stereoscopic images of the stored items;

[0139] The stereoscopic image is displayed on the screen on the refrigerator.

[0140] In this embodiment, the image processing device can detect the trajectory of the user's gaze projected onto the refrigerator surface; based on the trajectory, it selects the stored items inside the refrigerator corresponding to the current point of gaze on the trajectory; it forms a 3D image of the stored items and displays the 3D image on the refrigerator's display screen. This improves the clarity of the item image, reduces the difficulty of item recognition, and increases the accuracy of recognition and display, providing users with a simple, direct, and accurate intelligent display of refrigerator stored items, thereby enhancing the user's refrigerator storage management experience and overall user experience.

[0141] It should be noted that the image display device described in this embodiment is the same as the image display method described in the above embodiments. All implementations of the image display method described above are applicable to the embodiment of this image display device and can achieve the same effect.

[0142] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0143] Example 6:

[0144] Another embodiment of this application relates to an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image display methods of the above embodiments.

[0145] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0146] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0147] Example 7:

[0148] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0149] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A display method of an image, characterized by, The method comprises: detecting a visual line drop point trajectory formed by a visual line of a user projected on a surface of a refrigerator; selecting, based on the visual line drop point trajectory, a storage item in the refrigerator corresponding to a current visual line drop point in the visual line drop point trajectory; forming a stereoscopic image of the storage item and displaying the stereoscopic image on a display screen on the refrigerator; The detection of the visual line drop point trajectory formed by the visual line of the user projected on the surface of the refrigerator comprises: detecting a position point of an eye of the user relative to the surface of the refrigerator; using an infrared device to collect an angle of the visual line of the eye of the user projected on the surface of the refrigerator; calculating, according to the position point of the eye of the user and the angle of the visual line, a visual line drop point of the eye of the user projected on the surface of the refrigerator and forming a visual line drop point trajectory; The detection of the position point of the eye of the user relative to the surface of the refrigerator comprises: establishing an X-axis along a width direction of a front surface of the refrigerator, a Y-axis along a height direction of the front surface of the refrigerator and a Z-axis along a direction perpendicular to the front surface of the refrigerator to form a space coordinate system for describing a spatial position relationship between the eye of the user and the surface of the refrigerator; taking a coordinate point of the eye of the user in the space coordinate system as the position point of the eye of the user relative to the surface of the refrigerator; The collection of the angle of the visual line of the eye of the user projected on the surface of the refrigerator by the infrared device comprises: using an infrared emitter to emit infrared rays to the eye of the user and using an infrared camera to collect infrared rays reflected by a cornea of the eye of the user; calculating the angle of the visual line of the eye of the user projected on the surface of the refrigerator according to an included angle formed between the infrared rays emitted to the eye of the user and the infrared rays reflected by the cornea of the eye of the user.

2. The display method of an image according to claim 1, characterized by, The selection of the storage item in the refrigerator corresponding to the current visual line drop point in the visual line drop point trajectory based on the visual line drop point trajectory comprises: determining a visual line area on the surface of the refrigerator containing the current visual line drop point according to the current visual line drop point in the visual line drop point trajectory and a direction of the current visual line drop point in the visual line drop point trajectory; determining a target area in the refrigerator corresponding to the visual line area and selecting the storage item in the target area.

3. The display method of an image according to claim 2, characterized by, The determination of the target area in the refrigerator corresponding to the visual line area comprises: for the area in the refrigerator, taking a space area surrounded by an extension line of an edge of the visual line area and passing through the position point of the eye of the user as the target area.

4. The display method of an image according to claim 1, characterized by, The formation of the stereoscopic image of the storage item and the display of the stereoscopic image on the display screen on the refrigerator comprises: controlling a binocular camera group to collect the stereoscopic image of the storage item; displaying the stereoscopic image on the display screen on the refrigerator.

5. An image display device for implementing the image display method of any one of claims 1 to 4, characterized by, The method comprises: a detection module configured to detect a visual line drop point trajectory formed by a visual line of a user projected on a surface of a refrigerator; a selection module configured to select, based on the visual line drop point trajectory, a storage item in the refrigerator corresponding to a current visual line drop point in the visual line drop point trajectory; a display module configured to form a stereoscopic image of the storage item and display the stereoscopic image on a display screen on the refrigerator.

6. An electronic device, comprising: The method comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of displaying an image according to any one of claims 1 to 4.

7. A computer readable storage medium storing a computer program, wherein the computer program comprises program instructions configured to cause a processor to perform the method according to any one of claims 1 to 6. The computer program is executed by the processor to implement the method of displaying an image according to any one of claims 1 to 4.

Images