Audio and video progress synchronization method, display device and electronic device

Abstract

The present application discloses a method for synchronizing audio and video progress, a display device, and an electronic device. The present application provides a display device, including: a display; a user interface; a communicator for connecting the display device to a server; a first controller connected to the display, the communicator, and the user interface, respectively, for performing: receiving a first operation of a user, and playing audio and video according to the received first operation; when the user leaves, sending the playback record of the audio and video to a server, and when the server receives an acquisition request sent by an electronic device corresponding to the display device in response to a trigger operation of the user, the server sends the playback record to the electronic device, so that the electronic device continues to play the audio and video according to the playback record. In this way, the user can watch the audio and video with the playback progress synchronized with the display device on the electronic device, thereby improving the user's experience.

Description

[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on September 7, 2020, with application number 202010930923.2, and invention name “A method for synchronizing audio and video progress, a display device and a refrigeration device”, the Chinese patent application filed with the China Patent Office on September 7, 2020, with application number 202010929852.4, and invention name “A method for synchronizing audio and video progress, a display device and a refrigeration device”, and the Chinese patent application filed with the China Patent Office on September 30, 2020, with application number 202011057568.9, and invention name “Information recommendation method, food storage device and server”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of audio and video data processing, and in particular to an audio and video progress synchronization method, a display device, and an electronic device. Background Art

[0003] With the popularization of AIOT (Artificial Intelligence & Internet of Things), smart home appliances are becoming more and more popular, and users can watch audio and video on different smart home appliances. However, when users watch the same audio and video on different smart home appliances, they find that the audio and video playback progress on different smart home appliances is not synchronized, which affects the user experience. In some embodiments, the smart home appliances include display devices and electronic devices. For example, the electronic devices can be refrigeration equipment, coffee tables, fitting mirrors and other devices, and can also be computers, mobile phones, tablet computers and other devices. Summary of the invention

[0004] In a first aspect, the present application provides a display device, comprising: a display; a user interface; a communicator for connecting the display device to a server; and a first controller connected to the display, the communicator and the user interface, respectively, for executing:

[0005] Receive a first operation from a user, and play audio and video according to the first operation received; when the user leaves, send the playback record of the audio and video to a server, and when the server receives an acquisition request sent by an electronic device corresponding to the display device in response to the user's trigger operation, send the playback record to the electronic device, so that the electronic device continues to play the audio and video according to the playback record, wherein the playback record includes an audio and video identifier and a playback progress.

[0006] In a second aspect, the present application provides a display device, comprising: a display; a user interface; a communicator for connecting the display device to a server; and a first controller connected to the display, the communicator and the user interface, respectively, for executing:

[0007] Receive a trigger operation from a user; respond to an acquisition request sent to a server in response to the trigger operation, so that the server sends the playback record of the audio and video sent by the electronic device to the server when the user leaves to the display device, wherein the playback record includes an audio and video identifier and a playback progress; continue to play the audio and video at the playback progress according to the audio and video identifier.

[0008] In a third aspect, the present application provides an electronic device, comprising: a display screen for displaying an interactive interface; a second controller for executing: receiving a trigger operation; in response to the trigger operation, obtaining a playback record of audio and video on a display device corresponding to the electronic device; obtaining audio and video according to the audio and video identifier of the playback record, and playing the audio and video from the acquired playback progress, wherein the playback record includes the audio and video identifier and the playback progress.

[0009] In a fourth aspect, the present application provides an audio and video progress synchronization method, which is applied to a display device and is characterized by comprising:

[0010] Receive a first operation from a user, and play audio and video according to the first operation received; send the playback record of the audio and video to a server, and when the server receives an acquisition request sent by an electronic device corresponding to the display device in response to the user's trigger operation, the server sends the playback record to the electronic device, so that the electronic device continues to play the audio and video according to the playback record, wherein the playback record includes an audio and video identifier and a playback progress.

[0011] In a fifth aspect, the present application provides an audio and video progress synchronization method, which is applied to an electronic device and is characterized by comprising:

[0012] Receive a trigger operation; in response to the trigger operation, obtain a playback record corresponding to the user account; obtain audio and video according to the audio and video identifier of the playback record, and play the audio and video from the obtained playback progress, wherein the playback record includes the audio and video identifier and the playback progress. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a communication scenario between a display device and an electronic device in some embodiments of the present application;

[0014] Figure 2 A schematic diagram of another communication scenario between a display device and an electronic device in some embodiments of the present application;

[0015] Figure 3 A schematic diagram of a user interface of a display device in some embodiments of the present application;

[0016] Figure 4 A schematic diagram of another user interface of a display device in some embodiments of the present application;

[0017] Figure 5 A schematic diagram of a user interface of a health screen in some embodiments of the present application;

[0018] Figure 6 A schematic diagram of another user interface of a display device in some embodiments of the present application;

[0019] Figure 7 A signaling diagram of an audio and video progress synchronization method in some embodiments of the present application;

[0020] Figure 8 A method for synchronizing audio and video progress in some embodiments of the present application;

[0021] Fig. 9 This is a schematic structural diagram of a refrigeration device in some embodiments of the present application;

[0022] Fig.10 A flowchart of another information recommendation method in some embodiments of the present application;

[0023] Fig.11 A display interface diagram of a display screen of a refrigeration device in some embodiments of the present application;

[0024] Fig.12 This is a display interface diagram of a display screen of another refrigeration device in some embodiments of the present application;

[0025] Fig.13 This is a display interface diagram of a display screen of another refrigeration device in some embodiments of the present application;

[0026] Fig.14 This is a display interface diagram of a display screen of another refrigeration device in some embodiments of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0028] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.

[0029] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to orders other than those given in the diagrams or descriptions of some embodiments of this application.

[0030] The term "remote control" as used in this application may refer to a component of a display device that can wirelessly control the display device, usually within a short range. It is generally connected to the display device using infrared and / or radio frequency (RF) signals and / or Bluetooth, and may also include functional modules such as WiFi, wireless USB, Bluetooth, motion sensors, etc. For example: a handheld touch remote control replaces most of the physical built-in hard keys in a general remote control device with a user interface in a touch screen.

[0031] The term "gesture" used in this application refers to a user's behavior of expressing an intended idea, action, purpose, or result through a change in hand shape or hand movement.

[0032] Figure 1 FIG. 2 is a schematic diagram showing an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through the mobile terminal 300 and the control apparatus 100 .

[0033] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., to control the display device 200 wirelessly or in a wired manner. The user may input user commands through buttons on the remote controller, voice input, control panel input, etc. to control the display device 200. For example, the user may input corresponding control commands through the volume plus and minus keys, channel control keys, up / down / left / right movement keys, voice input keys, menu keys, power on / off keys, etc. on the remote controller to realize the function of controlling the display device 200.

[0034] In some embodiments, a mobile terminal, a tablet computer, a computer, a laptop computer, and other electronic devices may also be used to control the display device 200. For example, an application running on the electronic device is used to control the display device 200. The application may be configured to provide the user with various controls in an intuitive user interface (UI) on a screen associated with the electronic device.

[0035] Taking the electronic device as a terminal as an example, in some embodiments, the mobile terminal 300 can install software applications with the display device 200, realize connection and communication through the network communication protocol, and realize the purpose of one-to-one control operation and data communication. For example, it is possible to establish a control instruction protocol with the mobile terminal 300 and synchronize the remote control keyboard to the mobile terminal 300, and realize the function of controlling the display device 200 by controlling the user interface on the mobile terminal 300. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to realize the synchronous display function. The mobile terminal 300 can also communicate with the server.

[0036] like Figure 1 It is also shown that the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. For example, the display device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 400 may be a cluster or multiple clusters, and may include one or more types of servers. Other network service content such as audio and video on demand and advertising services is provided through the server 400.

[0037] The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size, and resolution are not limited. It is understood by those skilled in the art that the display device 200 may be modified in performance and configuration as required.

[0038] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0039] The electronic device can also be a refrigeration device with audio and video output functions.

[0040] Figure 24 is another scenario diagram provided by some embodiments of the present application. It includes a server 400 and multiple refrigeration devices 10 that are in communication connection, and the refrigeration device 10 can be a refrigerator. In some embodiments, the multiple refrigeration devices 10 can all be logged in with a target account. In some embodiments of the present application, the multiple refrigeration devices 10 include two refrigeration devices, such as a first refrigeration device and a second refrigeration device. In some embodiments, the information recommendation system can include one server 400 or multiple servers 400, Figure 2 Taking the example that the information recommendation system includes only one server 400, the number of refrigeration devices with target accounts logged in in the information recommendation system may also be three, four or even more, which is not limited in some embodiments of the present application. The server may also be connected to refrigeration devices with other accounts logged in, and the number of refrigeration devices with other accounts logged in may also be any number, which is not limited in some embodiments of the present application.

[0041] Smart home appliances are becoming more and more popular, and users can watch audio and video on different smart home appliances. However, when users watch the same audio and video on different smart home appliances, they find that the audio and video playback progress on different smart home appliances is not synchronized, which affects the user experience. In some embodiments, the smart home appliance includes a display device and an electronic device. The electronic device can be a refrigeration device, a coffee table, a dressing mirror, etc., or a computer, a mobile phone, a tablet computer, etc.

[0042] In some embodiments, audio and video refers to audio and / or video. Audio and video synchronization refers to audio synchronization and / or video synchronization.

[0043] In some embodiments of the present application, the method is carried out for the synchronization of audio and video between the display device and the electronic device, and the display device receives the first operation of the user and plays the audio and video. For example, the first operation can be completed by the user operating the control device to move the selector to select the control corresponding to the audio and video to be viewed on the interface of the display device, and clicking the confirmation key.

[0044] In some embodiments of the present application, in order to facilitate users to synchronously watch the audio and video played on the display device on the electronic device, the display device plays the audio and video according to the first operation, and sends the playback record of the audio and video to the server, and the server stores the playback record based on the user account. In some embodiments, the playback record includes the audio and video identification and the playback progress. In subsequent steps, the electronic device can obtain the playback record through the server, and then continue to play the audio and video according to the playback progress in the playback record.

[0045] In some embodiments of the present application, the audio and video identifier corresponds to the audio and video one-to-one. For example, the playback progress is the playback progress of the audio and video when the playback record is sent, which can be 11min28s, 5min12s, etc. In some embodiments of the present application, the audio and video playback records are stored in the dimension of user accounts. For example, the user account logged in on the display device is Xiao Ming, and the playback records of the user account Xiao Ming stored on the server may include ZHZ:11min28s, GTX:5min12s. For example, the user account logged in on the display device is Xiao Hong, and the playback records of the user account Xiao Hong stored on the server include MYZ:1h12min. In some embodiments, the audio and video identifiers can be represented by numbers, text and / or images.

[0046] In some embodiments of the present application, the display device can send audio and video playback records to a server in various scenarios. In some embodiments, a pause operation is received as input, and the audio and video playback records are sent to the server. In some embodiments, the display device also includes a human body detector, which determines whether the user is within the preset viewing range of the display device through the human body detector, and then determines whether to send the audio and video playback records to the server. In some embodiments, the audio and video playback records are sent to the server according to a preset period. In other embodiments, by determining whether other devices logged in to the same user account are playing audio and video, it is determined whether to send the playback record to the server. The following is a specific description of the method for synchronizing audio and video progress involved in sending audio and video playback records in the above different scenarios.

[0047] In some embodiments, before sending the audio and video playback record to the server, the process further includes: receiving a pause operation input. Figure 3 As shown. In some embodiments, after receiving the pause operation input, the play operation input by the user is received, that is, after pausing the audio and video, the audio and video continue to be watched. Since there is a time difference between pausing the audio and video playback and continuing to watch the audio and video playback, if the audio and video are played by other electronic devices logged in to the same user account during the time period corresponding to this time difference, the playback progress is changed at this time. In order to avoid duplication of audio and video content watched by the user, therefore, in some embodiments of the present application, if the playback progress recorded by the server is updated, the audio and video is played according to the updated playback progress sent by the server. For example, Figure 3 As shown, the playing progress of the audio and video is 11min5s. When the audio and video are played by other electronic devices during the time period corresponding to the time difference, the display device plays the audio and video again. Figure 4 As shown, the playback starts from the updated playback progress of 28min5s.

[0048] If the playback progress recorded in the server is not updated, it means that the user does not play the audio or video through other electronic devices, so the audio or video is directly continued to be played according to the playback progress after receiving the pause operation.

[0049] In some embodiments, when the playback progress recorded by the server is updated, the audio and video are played according to the received updated playback progress; when the playback progress recorded by the server is not updated, the step of playing the audio and video according to the playback progress after receiving the pause operation includes: in response to the playback operation, sending a request for obtaining the playback record to the server; receiving the playback record fed back by the server in response to the request for obtaining the playback record; when the feedback playback progress is different from the playback progress at the time of pause, playing the audio and video according to the feedback playback progress; when the feedback playback progress is the same as the playback progress at the time of pause, continuing to play the audio and video from the playback progress at the time of pause.

[0050] For example, when the playback progress after the pause operation is 11min28s and the feedback playback progress is 12min5s, the audio and video will continue to play according to the feedback playback progress of 12min5s. When the playback progress after the pause operation is 11min28s and the feedback playback progress is 11min28s, the audio and video will continue to play according to the playback progress of 11min28s after the pause.

[0051] In some embodiments, the display device also includes a human body detector; before sending the audio and video playback record to the server, it also includes: obtaining a detection signal input by the detector; when the detection signal indicates that the user has left the preset viewing range, sending the audio and video playback record to the server; when the detection signal indicates that the user has not left the preset viewing range, not sending the audio and video playback record to the server.

[0052] In some embodiments, when the user leaves the preset viewing range, it means that the user is no longer viewing the audio and video through the display device in the preset viewing range. At this time, the user may need to view the audio and video through other devices, so the playback record of the audio and video is sent to the server. If the user is viewing the audio and video in the preset viewing range, the playback record of the audio and video is not sent to the server.

[0053] In some embodiments, in order to save resources, when the detection signal indicates that the user has left the preset viewing range, the playback of the audio and video is also paused when the playback record of the audio and video is sent to the server. Since the number of users who use the display device to watch the audio and video may be a single person or multiple people, in some embodiments, when multiple people watch the audio and video, the detection signal indicates that a user has left the preset viewing range, and the playback of the audio and video is paused at this time. In some embodiments, when multiple people watch the audio and video, the detection signal indicates that a user has left the preset viewing range, and the playback of the audio and video is not paused at this time, but the playback record of the audio and video is uploaded to the server so that the user who left can use other devices to watch the audio and video synchronously. In some embodiments, when multiple people watch the audio and video, the detection signal indicates that all users have left the preset viewing range, and the playback of the audio and video is paused. In some embodiments, when the number of users watching the audio and video is one, the playback of the audio and video is paused when the user leaves the preset viewing range.

[0054] In some embodiments, after pausing the playing of the audio and video, the detection signal indicates that the user who left has re-entered the preset viewing range, that is, after leaving the preset viewing range, the user re-enters the preset viewing range, and at this time the display device continues to play the audio and video. Before playing the audio and video, it is first determined whether the playing progress recorded by the server is updated;

[0055] When the playback progress recorded by the server is updated, the audio and video are played according to the received updated playback progress; when the playback progress recorded by the server is not updated, the audio and video are continued to be played according to the playback progress after the pause operation is received.

[0056] In some embodiments, in order to avoid affecting the normal playback of other devices that are playing the audio and video, the step of sending the playback record of the audio and video to the server includes: sending a playback status confirmation request to the server to determine whether other devices under the user account are playing the audio and video; when other devices under the user account are not playing the audio and video, sending the playback record of the audio and video to the server;

[0057] When other devices under the user account are playing the audio and video, the playback record of the audio and video is not sent to the server. In some embodiments, the server includes a business server and an IoT server; the playback record of the audio and video is stored in the business server, and whether other devices under the user account are playing the audio and video is confirmed by the IoT server. In some embodiments, after the device is turned on, a long link is maintained with the IoT server, and the playback status of the device is reported to the IoT server in real time.

[0058] Taking the electronic device as a refrigeration device as an example, in some embodiments of the present application, the refrigeration device includes a box body for placing food, a door body arranged on the box body, and a display screen arranged on the door body, and the display screen is used to display an interactive interface. When a user wants to watch the same audio and video as on the display device through the refrigeration device, at this time, the user can log in to the same user account as the display device on the refrigeration device, receive a trigger operation, and send a request to obtain the playback record to the server in response to the trigger operation. The server receives the acquisition request and sends the playback record to the refrigeration device. The refrigeration device obtains the audio and video according to the audio and video identifier in the playback record, and continues to play the audio and video according to the playback progress in the playback record. In some embodiments, if the playback record is not obtained, the display screen is controlled to display a specified interface. In some embodiments, the specified interface can be a health screen, and the health screen mainly displays functions related to health. For example, Figure 5 As shown, the health screen displays health management, ice control, food management, recipes, etc. In some embodiments of the present application, the content of the specified interface is not limited.

[0059] In some embodiments, the refrigeration device receives an operation to exit playing audio and video; in response to the operation to exit playing audio and video, the display screen is controlled to display a specified interface, and a playback record of the audio and video is sent to the server.

[0060] In some embodiments, the trigger operation includes an operation of starting a display screen, or an operation of switching the display screen from a first mode to a second mode, wherein the first mode is a food management mode and the second mode is a mode different from the food management mode; or, an operation of opening the door. In some embodiments, the trigger operation also includes re-powering on the refrigeration device or re-logging into a user account. The operation of starting the display screen may include operations such as opening the door, a person approaching the refrigeration device, and clicking the display screen.

[0061] In some embodiments, after the refrigeration device continues to play the audio and video according to the playback progress in the playback record, an operation of exiting the playback of the audio and video is received. In response to the operation of exiting the playback of the audio and video, the display screen is controlled to display a specified interface, and the playback record of the audio and video is sent to the server. In some embodiments of the present application, in order to ensure that the display device and the refrigeration device can synchronize the playback progress of other devices when playing audio and video, in addition to the display device needing to upload the playback record of the audio and video, the refrigeration device also needs to upload the playback record of the audio and video to the server.

[0062] In some embodiments, the step of obtaining the playback record corresponding to the user account in response to the trigger operation includes: sending a playback record acquisition request to the server according to the user account of the refrigeration device; and receiving the playback record fed back by the server.

[0063] In some embodiments, the step of playing the audio and video from the acquired playback progress includes:

[0064] Obtain the playback record corresponding to the user account from the server, and query the playback status of the display device. In some embodiments, the server includes a business server and an IoT server, the playback record of the audio and video is obtained from the business server, the playback status of the device is determined by the IoT server, and the display device maintains a long link with the IoT server after it is turned on, and the playback status of the display device is reported to the IoT server in real time.

[0065] In response to the state of the display device being in the paused playback state, the upper layer of the user interface of the display screen is controlled to display a synchronization prompt message to prompt the user to synchronously watch the audio and video played on the display device. When the state of the display device is in the paused playback state, the user has no need to watch the audio and video on the display device. At this time, a synchronization prompt message is displayed on the display screen. The synchronization prompt message is exemplarily as follows: Figure 6 As shown, Figure 6 The synchronization prompt message in the display has the words "Do you want to continue watching? It is detected that you are watching "QYN" on the TV. Do you want to watch it synchronously on the refrigerator?" In addition, the synchronization prompt message includes a viewing control and a cancel control. When the synchronization prompt message is displayed, the user interface of the display screen can also be seen.

[0066] In response to selecting the viewing control in the synchronization prompt message, the synchronization prompt message is not displayed in the upper layer of the user interface of the display screen, the corresponding audio and video are played according to the playback progress, and the playback record in the server is updated in real time. Exemplarily, the viewing control in the synchronization prompt message can be selected by moving the focus to the viewing control through the user control device and pressing the confirmation key on the control device. In some embodiments, the playback record of the refrigeration equipment is also uploaded to the server in real time.

[0067] In response to selecting the cancel control in the synchronization prompt message, the display screen is controlled to stop displaying the synchronization prompt message and only display the specified interface.

[0068] In response to the state of the display device being in a normal playing state, the display screen is controlled to display a specified interface. In some embodiments, when the display device is playing audio and video normally, the user may still want to watch the audio and video on the display device, so at this time, there is no need to control the display screen to play audio and video synchronously, and the specified interface can be displayed.

[0069] In view of the above description, in some embodiments, when the number of users watching audio and video is one, the user watches the audio and video on the display device, enters the kitchen to watch the audio and video using the refrigeration device, and then returns to the living room to continue watching the audio and video on the display device. In this scenario, the method for synchronizing the progress of audio and video is as follows: Figure 7 As shown, the method includes: S101, the display device receives a first operation of the user. S102, the audio and video are played according to the first operation. S103, the audio and video playback record is sent to the server. S104, the server stores the playback record. When the user leaves the living room and enters the kitchen, after the user performs a trigger operation, S105, the refrigeration device receives the trigger operation. S106, in response to the trigger operation, the playback record corresponding to the user account on the refrigeration device is obtained. S107, the playback record sent by the server is received. S108, the audio and video are obtained according to the audio and video identifier in the playback record, and the audio and video are played from the acquired playback progress. S109, the audio and video playback record is sent to the server. S110, the server stores the playback record. When the user leaves the kitchen and enters the living room again to watch the audio and video through the display device. The user performs a playback operation, S111, the display device receives the playback operation input by the user. S112, in response to the playback operation, and the playback record is updated, the audio and video is played according to the updated playback progress. S113, the audio and video playback record is sent to the server. S114, storing the playback record. Through the methods in some embodiments of the present application, when a user watches audio and video through different devices, it is possible to avoid repeated viewing of audio and video content, thereby improving the user experience.

[0070] In some embodiments, Figure 8 As shown, the present application provides an audio and video progress synchronization method, which is applied to a display device, including:

[0071] Receiving a first operation from a user, and playing audio and video according to the received first operation;

[0072] The playback record of the audio and video is sent to a server. When the server receives an acquisition request sent by the refrigeration device corresponding to the display device in response to a user's trigger operation, the server sends the playback record to the refrigeration device, so that the refrigeration device continues to play the audio and video according to the playback record, wherein the playback record includes the audio and video identification and the playback progress.

[0073] In some embodiments, an audio and video progress synchronization method is applied to a refrigeration device, comprising:

[0074] Receive trigger operation;

[0075] In response to the triggering operation, obtaining the playback record corresponding to the user account;

[0076] The audio and video are acquired according to the audio and video identifier of the playback record, and the audio and video are played from the acquired playback progress, wherein the playback record includes the audio and video identifier and the playback progress.

[0077] In some embodiments, the electronic device may be multiple terminals logged in with the same account, so it is impossible to directly determine on which terminal the user will watch the audio and video. Therefore, in some embodiments of the present application, it is necessary to determine on which terminal the user may continue to watch the audio and video content through the trigger operation received by the terminal.

[0078] In some embodiments, the electronic device that receives the user operation may not have the ability to play audio and video. In this case, the terminal that does not have the ability to play audio and video can report / broadcast the event of obtaining the user operation, so that the terminal with audio and video playback capability located in the same room will enter the step of prompting the user whether to continue watching when receiving the broadcast / or corresponding reporting time instruction.

[0079] In some embodiments, the display device can determine whether there are other users watching through the acquisition component when uploading the audio and video playback progress. If there are other users, the display device will not pause, and if there are no other users, it will pause.

[0080] In some embodiments, when other users are watching, the display device periodically uploads the progress of audio and video playback until receiving information from the television device to start continuing playback. This allows the display device and the electronic device to keep the viewing progress consistent.

[0081] In some embodiments, the display device and the electronic device may be interchangeable.

[0082] In some embodiments, when the television device is a refrigeration device, the refrigeration device may be as follows: Fig. 9 As shown, the refrigeration device 10 may include: a housing 101, a door 102, a display screen 103, a communication interface and a controller ( Fig. 9101 includes a storage room having an opening K; a door 102 is movably connected to the housing 101 for shielding the opening K; and a display screen 103 is located on the surface of the door 102 facing away from the opening K for displaying a graphical user interface (GUI). In some embodiments, the display screen 103 is a touch screen, and a user can operate on the touch screen so that the controller receives corresponding instructions. The communication interface is a component for communicating with an external device or an external server according to various communication protocol types. For example, the communication interface can be a Wifi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip, or other network communication protocol chips or near field communication protocol chips. The controller controls the operation of the refrigeration device 10 and responds to the user's operation through various software control programs in the refrigeration device. In some embodiments, application software can also be installed in the refrigeration device, and icons of various application software can be displayed on the display screen 103. The user can click on the icon of the application software, and then the controller can start the application software according to the operation and control the display screen 103 to display the interface corresponding to the application software. The communication interface and the controller may be located at any position inside or outside the housing 101 and the door 102, and some embodiments of the present application do not limit the location of the communication interface and the controller. Other communicative devices may also be controlled.

[0083] In some embodiments, the refrigeration device may also include multiple structures such as an image collector (such as a video camera or a camera), a door switch detector, a speaker, a microphone, and a memory, and the multiple structures can be connected to the controller in communication. Among them, at least one of the speaker and the display screen can be used as an information output component to output information for interaction with the user; at least one of the microphone and the display screen can be used as an information input component to receive information from the user's interaction with the refrigeration device. When the information output component includes a speaker, the information output by the information output component is the voice information played by the speaker; when the information output component includes a display screen, the information output by the information output component is the information displayed on the display screen (such as text or image information). When the information input component includes a microphone, the information input by the information input component is the voice information collected by the microphone; when the information input component includes a display screen, the information input by the information input component is the information (such as text or image information) that the user operates on the display screen to trigger the display screen to be displayed.

[0084] In some embodiments, the image collector can be used to collect external environment scenes to adaptively change display parameters; it can also be used to collect user attributes or interactive gestures with the user to realize the function of interacting with the user. In some embodiments, the refrigeration device may include a first image collector 104a and a second image collector 104b. The first image collector 104a can be used to collect images of food in the storage room, such as the first image collector 104a is located on the top of the housing 101; the second image collector 104b can be used to collect images of the environment outside the storage room or collect interactive gestures of the user, and the second image collector 104b is located outside the storage room, such as above the display screen 103. In some embodiments, the first image collector 104a can collect images in the storage room when the door of the refrigeration device is in an open state, and stop collecting images when the door of the refrigeration device is in a closed state. In some embodiments, the first image collector 104a can also be located in the storage room, and the first image collector 104a can collect images of food in the storage room when the door of the refrigeration device is in a closed state. In some embodiments, a light source may also be provided in the storage room to illuminate the storage room when the door of the refrigeration device is in a closed state, so that the first image collector 104a can collect images of the food in the storage room. The first image collector 104a may also be located at other locations, such as the top of the door 102, which is not limited in some embodiments of the present application. In some embodiments, the "top" of a certain structure described in some embodiments of the present application is the end of the structure away from the ground when the refrigeration device is placed on the ground for normal use.

[0085] In some embodiments, the memory can be arranged in the housing or the door, or it can be independent of the housing and the door. The memory can be used to store the images collected by the image collector, the recognition results obtained by the controller for recognizing the food image, and other information that needs to be stored. In some embodiments, the door switch detector can be used to detect whether the door is in an open state or a closed state. In some embodiments, the door switch detector can be arranged in a position in the housing that can contact the door, and when the door switch detector does not contact the door, it is determined that the door is in an open state, and when the door switch detector contacts the door, it is determined that the door is in a closed state. The controller can control the display screen to display the recognition results or other information of the food image, and the display screen serves as an information output component. The display screen can also be a touch display screen, and the user can interact with the refrigeration device through the touch display screen, such as the user can touch on the touch display screen to trigger the controller to generate corresponding instructions, and execute the generated instructions, and the display screen serves as an information input component.

[0086] In some embodiments, the speaker and the microphone can be arranged on the casing or on the door. In some embodiments, the speaker and the microphone can be arranged at the same position as the image collector. For example, the speaker, the microphone and the image collector can together constitute a perception module, which is arranged on the top of the casing, and the perception module can be connected to a driving component, which can be used to drive the perception module to extend or retract from the top of the casing. The controller can control the speaker to send out voice information to prompt the user to perform corresponding operations. The controller can control the microphone to collect sound information in the environment where the refrigeration equipment is located, so as to generate corresponding instructions.

[0087] In some embodiments, the controller can control the driving component to move the perception module according to the state of the door (open state or closed state) or the voice command issued by the user. In some embodiments, when the controller determines that the door is in an open state, the driving component can be controlled to push the perception module in the direction close to the door, and when the perception module is pushed to a predetermined position, the driving component is controlled to stop the pushing action. Then the image collector in the perception module is triggered to work. When the controller determines that the door is in a closed state, the image collector can be controlled to stop working, and the driving component can be controlled to retract the perception module in the direction away from the door, and when the perception module is retracted to the original position, the driving component is controlled to stop the retraction action.

[0088] Refrigeration equipment is generally used to store food. In some embodiments, the present application also provides a method for recommending information based on the food in the refrigeration equipment.

[0089] Some embodiments of the present application provide an information recommendation method, which can be used to Figure 2 Any of the plurality of refrigeration devices 10 shown, or Fig. 9 In the refrigerated storage device 10 shown, the method may include:

[0090] Step 401: after receiving the dish recommendation instruction, send a dish recommendation request to the server, the dish recommendation request carries the identifier of the refrigeration device and the target account logged in to the refrigeration device.

[0091] Step 402: Receive the dish recommendation information sent by the server, where the dish recommendation information is determined by the server based on the dish recommendation request, according to the ingredients stored in the refrigeration device and the reference dish corresponding to the target account.

[0092] Step 403: Display the recommended information of dishes.

[0093] In summary, in the information recommendation method provided in some embodiments of the present application, the dishes recommended by the refrigeration device to the user are determined based on the ingredients stored in the refrigeration device and the reference dishes corresponding to the target account logged in to the refrigeration device. If the reference dishes corresponding to the target account are different, the dishes recommended by the refrigeration device may also be different, thereby improving the flexibility of the refrigeration device in recommending dishes.

[0094] Another information recommendation method provided by some embodiments of the present application can be used for Figure 2 As shown in the server 400, the method may include:

[0095] Step 501: Receive a dish recommendation request sent by a refrigeration device, where the dish recommendation request carries an identifier of the refrigeration device and a target account logged in to the refrigeration device.

[0096] Step 502: Determine the ingredients stored in the refrigeration device and the reference dishes corresponding to the target account according to the dish recommendation request.

[0097] Step 503: Determine dish recommendation information based on the ingredients stored in the refrigeration device and the reference dishes corresponding to the target account.

[0098] Step 504: Send dish recommendation information to the refrigeration device.

[0099] In summary, in the information recommendation method provided by some embodiments of the present application, the server can determine the recommended information of the dish based on the ingredients stored in the refrigeration device and the reference dish corresponding to the target account logged in on the refrigeration device, and then send the recommended information of the dish to the refrigeration device so that the refrigeration device can display the recommended information of the dish. If the reference dish corresponding to the target account is different, the recommended information of the determined dish is also different, and then the refrigeration device recommends different dishes to the user based on the recommended information, thereby improving the flexibility of the refrigeration device in recommending dishes.

[0100] Fig.10 is a flowchart of another information recommendation method provided in some embodiments of the present application, which can be used for Figure 2 The information recommendation system shown. The following embodiments of the present application take the information recommendation system including a server, a first refrigeration device and a second refrigeration device, and the first refrigeration device and the second refrigeration device are both logged in with a target account as an example to explain the information recommendation method provided by some embodiments of the present application. Fig.10 As shown, the information recommendation method may include:

[0101] Step 601: A first refrigeration device identifies food stored therein.

[0102] In some embodiments, the refrigeration device can identify the food when the stored food changes, so as to determine the food currently stored in the refrigeration device. In some embodiments, the refrigeration device can identify the food stored therein when the door changes from an open state to a closed state. Alternatively, the refrigeration device can identify the food stored and taken out in real time when the user deposits and takes out food, and then determine the food currently stored in the refrigeration device in combination with the food originally stored in the refrigeration device.

[0103] In the first food identification method, the refrigeration equipment can use image recognition to identify the food.

[0104] In some embodiments, the controller of the refrigeration device can control the light source in the storage room to turn on when the door of the refrigeration device is determined to be changed from an open state to a closed state by the door switch detector, and control the image collector in the storage room to collect images of the food in the storage room. The controller can input the image collected by the image collector into the food recognition model, thereby obtaining a recognition result, and determine the type of food stored in the refrigeration device according to the recognition result. Alternatively, the controller of the refrigeration device can control the image collector to collect an image at the opening of the storage room when the door switch detector determines that the door of the refrigeration device is changed from a closed state to an open state, and then recognize the image to determine the user's access action and determine the type of food stored or taken out by the user. When it is determined that the user deposits food into the storage room, the originally stored food and the newly deposited food are jointly determined as the food currently stored in the storage room. When it is determined that the user takes out food from the storage room, the food other than the food to be taken out in the originally stored food is determined as the food currently stored in the storage room. In some embodiments, in addition to determining the type of food, other information about the food may also be determined through image recognition, such as the volume of the food and the storage location of the food in the storage room.

[0105] In the second food identification method, the refrigeration equipment can use Radio Frequency Identification (RFID) to identify the food.

[0106] In some embodiments, please refer to Fig. 9 The refrigeration device may further include an electronic label box 105, which may be used to place electronic labels (not shown in the figure), and each electronic label may correspond to a food. In some embodiments, Fig. 9 Taking the label box 105 installed on the surface of the door 102 close to the opening K as an example, in some embodiments, the label box can also be installed on the surface of the door 102 away from the opening K. Alternatively, the label box can also be located at any other position on the housing 101 or the door 102, which is not limited in some embodiments of the present application.

[0107] When a user places a certain food into the storage room of the refrigeration device, the user can take out the corresponding electronic label from the electronic label box and attach the electronic label to the food. The controller of the refrigeration device can determine the type of food stored in the storage room based on the food corresponding to the missing electronic label in the label box. When the user takes out the food from the storage room, the user can remove the electronic label on the food and put it into the label box. The controller of the refrigeration device can determine the type of food taken out from the storage room based on the food corresponding to the newly added electronic label in the label box. Furthermore, the controller can determine the food currently stored in the storage room in combination with the food originally stored in the storage room.

[0108] In some embodiments, the refrigeration device may further include an antenna, which may be connected to the controller by wire. The controller may transmit a radio frequency signal through the antenna, and the transmitted radio frequency signal may cover a certain range around the antenna. When the electronic tag is within the range, it may receive the radio frequency signal, and transmit information stored in the electronic tag to the antenna by means of the energy obtained under the excitation of the radio frequency signal, such as the identification of the electronic tag; further, the controller may receive the information sent by the electronic tag through the antenna to determine the food corresponding to the electronic tag. In some embodiments, the antenna may be located in a label box, such as at the bottom of the label box, and the label box may be made of a material capable of shielding radio frequency signals, such as a material of the label box may include metal. At this time, the radio frequency signal transmitted by the antenna may only cover the space in the label box, and further, only the electronic tag in the label box may receive the radio frequency signal transmitted by the antenna, and transmit the information stored in the electronic tag to the antenna; the controller may only detect the electronic tag in the label box. In some embodiments, the controller may also adjust the transmission power of the radio frequency signal so that the radio frequency signal only covers the space in the label box, and the label box may be made of any material.

[0109] In some embodiments, the refrigeration device may further include a reader, which may be connected to an antenna, and is used to transmit a radio frequency signal through the antenna and receive information sent by the electronic tag through the antenna. The reader may be connected to the controller for communication, and after receiving the information sent by the electronic tag, the reader sends the information to the controller.

[0110] In another optional way of determining the ingredients, the user can operate on the display screen to input the current operation as an ingredient storage operation or ingredient removal operation each time the user deposits or takes out ingredients into the storage chamber, and can also input the type of the ingredients deposited and the type of the ingredients taken out. The controller can determine the ingredients currently stored in the storage chamber according to the operation type and the type of ingredients input by the user. In some embodiments, the user can also input other information such as the storage location of the ingredients, the volume of the ingredients, and the shelf life.

[0111] Step 602: The first refrigeration device sends information about the food stored therein to the server.

[0112] The information of the food may include the type of food. In some embodiments, the information of the food may also include the shelf life, volume, and storage location of the food in the storage room, etc., which is not limited in some embodiments of the present application. In some embodiments, the refrigeration device may also send an identification of the refrigeration device to the server when sending the information of the food stored therein, so that the server can store the information of the food from the device dimension.

[0113] Step 603: The server stores information about the food stored in the first refrigeration device.

[0114] The server may store the information about the food sent by the first refrigeration device in a storage space corresponding to the first refrigeration device, and the storage space may be used to specifically store the information about the food stored in the first refrigeration device.

[0115] In some embodiments, some embodiments of the present application take the example of a refrigeration device identifying ingredients, thereby determining the ingredients stored in the refrigeration device, and sending the information of the ingredients to a server. In some embodiments, the refrigeration device may also directly send the image of the photographed ingredients to the server, and the server may identify the ingredients based on the image, determine the ingredients stored in the refrigeration device, and then directly store the information of the ingredients stored in the refrigeration device. In some embodiments, the server may also send the information of the identified ingredients to the refrigeration device so that the refrigeration device can display and store the information of the ingredients.

[0116] In some embodiments, Fig.11 is a display interface diagram of a display screen of a refrigeration device provided in some embodiments of the present application, Fig.12 is a display interface diagram of another display screen of a refrigeration device provided in some embodiments of the present application, Fig.13 This is a display interface diagram of another display screen of a refrigeration device provided in some embodiments of the present application. In some embodiments, after closing the door of the refrigeration device, the user can operate on the display screen to trigger the display screen to display the food currently stored in the storage chamber. For example, a food management button can be displayed on the display screen, and the user can click the button to trigger the display screen to display Fig.11 In some embodiments, the storage compartment of the refrigeration device includes a plurality of compartments, namely, a refrigeration compartment, a temperature-changing compartment, and a freezing compartment. The controller can determine the food stored in each compartment and control the display screen to display information about the food stored in each compartment in different areas of the food management interface. Fig.11As shown, the food management interface includes: a first area D1 corresponding to the refrigerating chamber, a second area D2 corresponding to the temperature-changing chamber, and a third area D3 corresponding to the freezing chamber. Each area can display a representative picture P1 of each food stored in the corresponding compartment, and the name of the food is displayed below the representative picture P1 (such as Fig.11 tomatoes in the dish).

[0117] In some embodiments, Fig.11 As shown, multiple target controls can also be displayed in the ingredient management interface. For example, the multiple target controls can include a recipe recommendation control J1, an add ingredient control J2, and a photo control J3. The user can trigger the controller to execute the corresponding instruction by clicking the corresponding control. For example, when the user clicks the add ingredient control J2, it can trigger the controller to control the display screen to display a virtual keyboard. The user can enter the type of ingredient through the virtual keyboard, and then the controller can store the type of ingredient and control the display screen to display the representative picture and name of the ingredient type in the corresponding area in the ingredient management interface. For example, when the user clicks the photo control J3, the controller can control the image collector to collect images of the ingredients in the storage room, and identify the collected images, and then control the display screen to display Fig.12 The food identification interface shown.

[0118] In some embodiments, each compartment in the storage room may be provided with an image collector. Fig.12 As shown, the food identification interface may include images of food in each compartment collected by the image acquisition component, such as image 1 is an image of food in the refrigerator, image 2 is an image of food in the temperature-changing chamber, and image 3 is an image of food in the freezer. The food identification interface may also include the identification results of the food in each compartment, such as the identification result of each food can be displayed in the form of a representative picture P2, or can also be displayed in the form of a food name, or in the form of a representative picture combined with a food name. In some embodiments, the identification result of each food also has a corresponding deletion control J4, such as the deletion control J4 can be displayed with an icon of a small circle with a horizontal bar in the middle displayed in the upper right corner of the representative picture P2. The user can click the deletion control J4 to delete the corresponding identification result. In some embodiments, the food identification interface may also include a re-identification control J5 and a one-key entry control J6. The user can click the re-identification control J5 to trigger the controller to re-identify the food stored in the storage room. The user can click the one-key entry control J6 to trigger the controller to store and display the information of the identified food in each compartment, such as the controller can control the display screen to display Fig.13The input ingredient display interface shown. The input ingredient display interface may include information about the newly added ingredients in each compartment, such as a representative picture of each ingredient and the name of the ingredient. In this way, when the user triggers the display screen to display the ingredient management interface again, the ingredient management interface may include information about the newly entered ingredient, that is, the information about the ingredient in the input ingredient display interface.

[0119] Step 604: The second refrigeration device identifies the food stored therein. In some embodiments, step 604 may refer to step 601, and some embodiments of the present application will not be repeated.

[0120] Step 605: The second refrigeration device sends information about the food stored therein to the server. In some embodiments, step 605 may refer to step 602, and some embodiments of the present application will not be repeated.

[0121] Step 606: The server stores the information of the food stored in the second refrigeration device. In some embodiments, step 606 may refer to step 603, and some embodiments of the present application will not be repeated.

[0122] Step 607: When displaying dishes, the first refrigeration device receives a marking instruction for the displayed first dish.

[0123] Fig.14 This is another display interface diagram provided by some embodiments of the present application. In some embodiments, a recipe software may be installed in the refrigeration device. When the recipe software is running, the display screen may display a variety of dishes, such as Fig.14 In the interface shown, users can also mark the displayed dishes. If the user performs a specific operation (such as long pressing) on ​​the displayed dishes, the display screen will be triggered to display multiple operation controls, and the user can click the corresponding operation control to trigger the controller to perform the corresponding operation. Fig.14 As shown, the multiple operation controls may include: a collection control, a favorite control, a forwarding control, a save control, and a viewing control, etc. In some embodiments, other operation controls may also be provided, which is not limited in some embodiments of the present application. In some embodiments, the operation of the user clicking on these operation controls may be a marking operation, or the operation of the user clicking on certain specific operation controls may be a marking operation, such as the specific operation control may include: a collection control and a favorite control. When the refrigeration device detects a marking operation for a dish (such as the first dish), the refrigeration device may determine that a marking instruction for the first dish has been received.

[0124] In some embodiments, the above embodiments of the present application take the startup of the dish software to trigger the display screen to display the dish as an example. In some embodiments, at any time when the display screen of the refrigeration device displays the dish, the user can mark the dish. For example, when the user searches for a dish in the web page displayed on the display screen, the dish can also be marked, or the search operation for the dish can be directly determined as the marking operation for the dish.

[0125] Step 608: The first refrigeration device sends a marking request for the first dish to the server based on the marking instruction, and the marking request carries the target account number.

[0126] When the refrigeration device receives the marking instruction of the first dish, it can send a marking request for the first dish carrying the target account to the server, so that the server can store the information of the first dish from the account dimension.

[0127] Step 609: The server determines the first dish as a reference dish corresponding to the target account according to the marking request of the first dish.

[0128] In some embodiments of the present application, the reference dishes corresponding to the target account include dishes marked by the user of the target account, that is, dishes preferred by the user of the target account. The reference dishes corresponding to the target account can be used to reflect the personal preferences of the user of the target account. In some embodiments, when the server receives a marking request for a first dish, and the marking request carries the target account, the server can store the information of the first dish in the storage space corresponding to the target account, and the storage space can be used to specifically store the information of the reference dishes corresponding to the target account. In some embodiments, the target account can correspond to multiple reference dishes. After receiving each marking request for a dish, the server can determine the dish as a newly added reference dish, and does not make changes to other reference dishes corresponding to the target account.

[0129] Step 610: When displaying dishes, the second refrigeration device receives a marking instruction for the displayed second dish.

[0130] In some embodiments, step 610 may refer to step 607, which will not be described in detail in some embodiments of the present application.

[0131] Step 611: The second refrigeration device sends a marking request for the second dish to the server based on the marking instruction, and the marking request carries the target account number.

[0132] In some embodiments, step 611 may refer to step 608, which will not be described in detail in some embodiments of the present application.

[0133] Step 612: The server updates the reference dishes corresponding to the target account according to the marking request of the second dish, where the reference dishes corresponding to the target account include the first dish and the second dish.

[0134] In some embodiments, step 612 may refer to step 609, and the server may also determine the second dish as the reference dish corresponding to the target account. Since the reference dish corresponding to the target account also includes the first dish, the reference dish corresponding to the target account may include the first dish and the second dish, which is equivalent to the server updating the reference dish corresponding to the target account. The first dish and the second dish described in some embodiments of the present application are both target dishes, and the target dish refers to the marked dish among the dishes displayed by the refrigeration device.

[0135] In some embodiments of the present application, step 612 is performed after step 609 as an example. After step 612, the reference dishes corresponding to the target account determined by the server include the first dish and the second dish. In some embodiments, if before step 609, that is, before the server receives the marking request for the first dish, if a marking request for other dishes is received, and the marking request also carries the target account, then after step 609, all reference dishes corresponding to the target account may include the other dishes and the first dish. After step 611, the reference dishes corresponding to the target account include the first dish, the second dish, and the other dishes.

[0136] Step 613: After receiving the dish recommendation instruction, the first refrigeration device sends a dish recommendation request to the server. The dish recommendation request carries the identifier of the first refrigeration device and the target account.

[0137] In some embodiments, a recipe software may be installed in the refrigeration device. When the recipe software is running, the display screen may display recommended dishes. When a startup operation for the recipe software is detected, it can be confirmed that a dish recommendation instruction has been received. For example, a recipe application icon may be displayed on the display screen of the refrigeration device. When the user clicks on the recipe application icon, the startup operation for the recipe is performed, and the refrigeration device can receive the dish recommendation instruction. In some embodiments, when the user clicks on the recipe application icon, the refrigeration device can receive the dish recommendation instruction. Fig.11 When the recipe recommendation control J1 in the food management interface shown is pressed, the refrigeration device may also receive the dish recommendation instruction. There may also be other ways to trigger the refrigeration device to receive the dish recommendation instruction, which is not limited in some embodiments of the present application.

[0138] After receiving the dish recommendation instruction, the refrigeration device can send a dish recommendation request to the server. The dish recommendation request can carry the identification of the refrigeration device and the target account logged in to the refrigeration device, so that the server can determine the recommended dishes based on the refrigeration device and the target account.

[0139] Step 614: The server determines the ingredients stored in the first refrigeration device and the reference dishes corresponding to the target account according to the dish recommendation request.

[0140] In some embodiments, after receiving the dish recommendation request, the server can obtain the food information in the storage space corresponding to the first refrigeration device according to the identification of the first refrigeration device carried in the dish recommendation request to determine the food stored in the first refrigeration device. The server can also obtain the reference dish corresponding to the target account in the storage space corresponding to the target account according to the target account carried in the dish recommendation request.

[0141] Step 615: The server determines at least one dish that can be prepared using the ingredients stored in the first refrigeration device.

[0142] In some embodiments, the server may determine the at least one dish based on a recipe library, which may include methods for making a variety of dishes and the ingredients required for making each dish. If the server determines that the ingredients stored in the first refrigeration device include all the ingredients required for making a dish among the multiple dishes, it may determine that the dish is a dish that can be made using the ingredients stored in the first refrigeration device. In some embodiments, the at least one dish may include all dishes that can be made using the ingredients stored in the first refrigeration device among the multiple dishes in the recipe library. In some embodiments, the recipe library may be stored in the server, or may be obtained by the server on the Internet or other locations.

[0143] Step 616: The server determines the recommendation priority of each dish according to the similarity between each dish of the at least one dish and the reference dish corresponding to the target account.

[0144] After determining at least one dish that can be prepared using the ingredients stored in the first refrigeration device, the server can determine the recommendation priority of each dish in the at least one dish according to the reference dish corresponding to the target account. Since the reference dish corresponding to the target account reflects the user's preferences, the recommendation priority of the dish is determined according to the reference dish, and then the dish is recommended to the user according to the recommendation priority, which can ensure that the recommended dish is more in line with the user's preferences and improve the accuracy of the dish recommendation.

[0145] In some embodiments, the recommendation priority of each dish is positively correlated to the similarity between each dish and reference dishes, such as the recommendation priority of each dish is positively correlated to the sum of the similarities between each dish and all reference dishes. For example, the recommendation priority of each dish can be equal to the sum of the similarities between each dish and all reference dishes. In some embodiments, the similarity between any two dishes can include the similarity of taste and the overlap of ingredients used in making them.

[0146] For the sake of distinction, each of the at least one dish is referred to as a recommended dish below. In some embodiments, the server may compare the taste of each recommended dish in the at least one recommended dish with the taste of each reference dish to determine the similarity between the taste of each recommended dish and the taste of each reference dish; the server may also compare the ingredients used in each recommended dish in the at least one recommended dish with the ingredients used in each reference dish to determine the overlap between the ingredients used in each recommended dish and the ingredients used in each reference dish. Furthermore, the server may determine the sum of the similarity between the taste of a recommended dish and a reference dish and the overlap of the ingredients used as the similarity between the recommended dish and the reference dish.

[0147] In some embodiments, the similarity of taste and the overlap of the ingredients used may be respectively weighted, and the product of the similarity of taste and the corresponding weight plus the product of the overlap of the ingredients used and the corresponding weight may be used as the similarity of the two dishes. For example, the weight of the similarity of taste is lower than the weight of the similarity of the ingredients used, or the weight of the similarity of taste is higher than the weight of the similarity of the ingredients used. In some embodiments, different reference dishes may also be weighted. After determining the similarity of each reference dish for a recommended dish, the product of the similarities of various reference dishes and their weights may be added to obtain the recommendation priority of the final recommended dish.

[0148] Step 617: Determine the recommendation information of the dish, where the recommendation information of the dish includes: the at least one dish and the recommendation priority of the at least one dish.

[0149] After determining the at least one recommended dish and the recommendation priority of each recommended dish, the server can determine the dish recommendation information corresponding to the first refrigeration device. The dish recommendation information includes: the identification of the at least one recommended dish and the recommendation priority of each recommended dish in the at least one recommended dish, such as the identification of each recommended dish can include a representative picture and name of the recommended dish.

[0150] Step 618: Send the determined dish recommendation information to the first refrigeration device.

[0151] Step 619: The first refrigeration device displays the at least one dish according to the recommendation priority of the at least one dish in the received recommendation information of the dish.

[0152] Since the recommended dishes with higher recommendation priority have a higher similarity with the reference dishes preferred by the user and are more in line with the user's preferences, the information of the recommended dishes can be displayed in a more prominent position on the display screen so that the user can quickly notice the recommended dishes, thereby avoiding the frustration of the user when he / she fails to find a satisfactory dish after viewing multiple dishes, thereby improving the user experience. In some embodiments, since the user usually views the content on the display screen from left to right and from top to bottom, the refrigeration device can display the recommended dishes with higher recommendation priority in the received recommendation information of the dishes at a higher and left position on the display screen so that the user can quickly notice the recommended dishes and find the dishes that meet his / her needs quickly.

[0153] In some embodiments, the display position of the recommended dishes with a higher recommendation priority can also be set by the user. For example, if a user is accustomed to viewing the content displayed on the display screen from the middle position of the display screen, the user can set the middle area of ​​the display screen as the area for displaying the recommended dishes with a higher recommendation priority. Furthermore, the refrigeration device can display the recommended dishes with a higher recommendation priority in the middle area of ​​the display screen according to the received recommendation information of the dishes.

[0154] In some embodiments, the server may send the recommendation information of dishes to the refrigeration device, and the recipe of each dish may also be included. The user may operate on each dish displayed on the display screen to trigger the display screen to display the recipe of the dish. In some embodiments, the recommendation information of dishes may also include other dishes in addition to the at least one recommended dish mentioned above, such as the other dishes may include the dishes most popular with users, the dishes that are most convenient to prepare, the dishes with less fat content, etc. The ingredients required for the preparation of these dishes may have nothing to do with the ingredients stored in the refrigeration device.

[0155] In some embodiments, when the refrigeration device displays the recommended dish according to the received recommendation information of the dish, the user also performs a marking operation on the displayed recommended dish, and then the refrigeration device can determine that the recommended dish is a target dish, and determine that the marking instruction of the recommended dish is received, and then send a marking request for the recommended dish to the server. Please refer to the introduction of steps 607 to 609 for details, and some embodiments of the present application will not be repeated.

[0156] In some embodiments, the server can determine the recommendation information of the dish based on the preference information corresponding to the ingredients and the target account stored in the refrigeration device. The above embodiment of the present application is described by taking the preference information as reflected by the reference dishes marked by the user's history, and then determining the recommendation priority of the dish based on the similarity between the dish and the reference dish. In some embodiments, the preference information can also be reflected by other information entered by the user, such as the user can directly enter his or her own taste and ingredient preferences, such as liking sweeter tastes, liking to eat tomatoes, etc.; and then the server can determine the recommendation priority of the dish based on the other information. Alternatively, the server can also infer the user's taste and ingredient preferences based on the reference dishes marked by the user's history, and then determine the recommendation priority of the dish based on the taste and ingredient preferences.

[0157] In some embodiments of the present application, the at least one dish in the recommended information of dishes received by the refrigeration device may include: all dishes or some dishes that can be made with the ingredients stored in the refrigeration device, which is not limited in some embodiments of the present application. In some embodiments, the user can input information such as the taste and ingredients that he cannot accept through the display screen, and the refrigeration device can send the information and the target account to the server, and then when the server receives the dish recommendation request carrying the target account, the dishes in the determined recommended information may not include dishes made with the ingredients and dishes of the taste.

[0158] In some embodiments, the above embodiments are explained by taking the example that only one target account is logged in to the refrigeration device. In some embodiments, multiple target accounts can be logged in to the refrigeration device, such as a first target account and a second target account can be logged in to the first refrigeration device. The server can determine the recommendation information of the dish and the account carried by the marking request based on the currently selected target account among the multiple target accounts logged in to the refrigeration device. If the currently selected is the first target account, the server determines the recommendation information of the dish based on the reference dish corresponding to the first target account. If the currently selected is the second target account, the server determines the recommendation information of the dish based on the reference dish corresponding to the second target account. When a dish is marked in the page corresponding to the first target account, the generated marking request carries the first target account. When a dish is marked in the page corresponding to the second target account, the generated marking request carries the second target account. In some embodiments, please continue to refer to Fig.14 There are three target accounts that can be logged into the refrigeration device, and the three target accounts are marked by "Dad", "Mom" and "Dabao" respectively. The user can select his own target account from the three target accounts, and the target account selected by the user is considered to be the target account currently used by the refrigeration device. When the user selects different target accounts, the recommended dishes displayed on the display screen are also different.

[0159] In the related art, the information of ingredients stored in the refrigeration equipment is only marked by the account dimension. When the same account is logged in to multiple refrigeration equipment, each refrigeration equipment will make dish recommendations based on all the ingredients stored in the multiple refrigeration equipment. In this way, the ingredients for preparing the recommended dishes seen by the user on a certain refrigeration equipment may not be stored in the refrigerator, and the accuracy of the dish recommendation is low.

[0160] In some embodiments of the present application, the server can store the preference information of the user corresponding to the target account and the information of the ingredients stored in the refrigeration device, that is, the server stores the user-related information from the account dimension and the ingredient-related information in the device from the device dimension. The target account can be logged in on multiple refrigeration devices, and the server can determine the reference dishes corresponding to the target account as a whole according to the marking operation of the dishes performed by the user on the multiple refrigeration devices. In this way, it can be ensured that the server collects more comprehensive preference information of the user, and then can more accurately recommend dishes that meet the user's preferences, thereby improving the accuracy of the dish recommendation. And when determining the recommended information of the dish, the server can only determine the dishes that can be made with the ingredients stored in the refrigeration device that sends the dish recommendation request, and then the dishes recommended by the refrigeration device to the user are all directly prepared by the ingredients stored in the refrigeration device. It avoids the situation that the ingredients stored in each refrigeration device logged in by the target account are not stored in the refrigeration device currently operated by the user, thereby improving the effectiveness and practicality of the recommended information of the dish.

[0161] In some embodiments of the present application, the steps executed by the refrigeration device can be executed by the controller therein, and the steps executed by the server can be executed by the processing unit therein; the display steps executed by the refrigeration device can be implemented by the controller of the refrigeration device controlling the display screen; the steps executed by the refrigeration device for interacting with the server can be implemented by the controller in the refrigeration device through the communication interface; the steps executed by the server for interacting with the refrigeration device can be implemented by the processing unit in the server through the network interface unit.

[0162] In some embodiments, the first refrigeration device and the second refrigeration device in some embodiments of the present application are only any two refrigeration devices connected to the server and logged in with the same target account. The above steps can also be performed for other refrigeration devices logged in with the target account. In some embodiments of the present application, the information recommendation method is described by taking the example that both the first refrigeration device and the second refrigeration device receive the dish marking instruction. In some embodiments, only one refrigeration device may receive the dish marking instruction, and some embodiments of the present application do not limit this. In addition, in some embodiments of the present application, the information recommendation method is described by taking the example that the first refrigeration device receives the dish recommendation instruction. In some embodiments, if the second refrigeration device receives the dish recommendation instruction, the above steps 613 to 619 can also be executed.

[0163] In summary, in the information recommendation method provided in some embodiments of the present application, the dishes recommended by the refrigeration device to the user are determined based on the ingredients stored in the refrigeration device and the reference dishes corresponding to the target account logged in to the refrigeration device. If the reference dishes corresponding to the target account are different, the dishes recommended by the refrigeration device may also be different, thereby improving the flexibility of the dish recommendation of the refrigeration device.

[0164] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: monitor; User interface; Communicator, used to connect the display device with the server; The first controller connected to the display, the communicator and the user interface respectively is configured to: Receive a first operation input by a user, and play audio and video according to the received first operation; receiving a detection signal from a human body detector; When the detection signal indicates that the first user and the second user are in a preset viewing range of the display device and the first user leaves the preset viewing range, a playback record of the audio and video when the first user leaves the preset viewing range is sent to the server, and the display device is controlled to continue playing the audio and video; the playback record includes an audio and video identifier and a playback progress; Among them, the playback record is used to enable the server to send the playback record to the electronic device when receiving the acquisition request sent by the electronic device in response to the user's trigger operation; without pausing the playback of the audio and video of the display device, the electronic device continues to play the audio and video from the playback progress according to the received playback record.

2. The display device according to claim 1, characterized in that The first controller executes sending the playback record of the audio and video when the first user leaves the preset viewing range to the server, and is further configured to: The server stores the playback record based on the user account as a dimension, and when receiving an acquisition request sent by an electronic device with the same user account in response to the trigger operation, the server sends the playback record to the electronic device, so that the electronic device plays the audio and video from the playback progress according to the received playback record.

3. The display device according to claim 1, characterized in that The first controller is further configured to: In response to a pause operation input by the user, the audio and video are controlled to pause playing, and the playing record of the audio and video during the pause is sent to the server.

4. The display device according to claim 3, characterized in that After the first controller sends the playback record of the audio and video during the pause to the server, the first controller is configured to: Receive playback operations input by the user; If the play record of the server is updated during the time period between the pause operation and the play operation, the audio and video are played according to the updated play record in the server; If the play record of the server is not updated in the time period between the pause operation and the play operation, the audio and video are played according to the play record after the pause operation.

5. The display device according to claim 4, characterized in that The first controller is configured to: In response to the playing operation, sending a request for obtaining the playing record to the server; Receiving the playback record fed back by the server in response to the playback record acquisition request; When the feedback playback progress is different from the playback progress during the pause, the playback record of the server is updated; When the fed-back playing progress is the same as the playing progress during the pause, there is no update to the playing record of the server.

6. An electronic device, characterized in that: include: A display screen, used for displaying an interactive interface; The second controller is configured as: Receive trigger operation; In response to the trigger operation, sending a request to the server; Receive a playback record of the display device fed back by the server, wherein the playback record includes an audio and video identifier and a playback progress, the playback record is a playback record of the audio and video played by the display device before the electronic device sends an acquisition request to the server, and the playback record is a playback record of the audio and video when the first user leaves the preset viewing range when the display device plays the audio and video and receives a detection signal from a human body detector indicating that the first user and the second user are in the preset viewing range of the display device, and the display device continues to play the audio and video when the first user leaves the preset viewing range; The audio and video are acquired according to the audio and video identifier, and the audio and video are played from the playback progress.

7. The electronic device according to claim 6, characterized in that: The acquisition request is used to obtain the playback record of the display device under the same account as the electronic device.

8. The electronic device according to claim 6, characterized in that: The server includes a business server and an Internet of Things server; the acquisition request is also used to obtain the playback status of the display device corresponding to the electronic device, the playback record is obtained from the business server, and the playback status is obtained from the Internet of Things server; after the second controller receives the playback record of the display device fed back by the server, it is configured as follows: Receiving the playback status of the display device fed back by the server; If the playing state of the display device is the paused playing state, controlling the display screen to display a synchronization prompt message, wherein the synchronization prompt message is used to prompt the user whether to synchronously watch the audio and video played on the display device; If the playing state of the display device is a normal playing state, the display screen is controlled to display a specified interface.

9. The electronic device according to claim 6, characterized in that: The second controller receives a trigger operation, including: receiving at least one of an operation of starting the display screen, an operation of switching the display screen from a first mode to a second mode, an operation of opening a door, an operation of re-powering on, and an operation of re-logging into a user account; The first mode is a food management mode, and the second mode is a mode different from the food management mode.

10. An electronic device, characterized in that: include: Display screen; The third controller is configured as: Receiving a trigger operation reported or broadcast by other electronic devices, wherein the trigger operation is reported or broadcast by other electronic devices that do not have audio or video playback capabilities and are in the same local area network as the electronic device when receiving a trigger operation input by a user to instruct to continue playing audio or video; In response to the trigger operation, controlling the display screen to display synchronization prompt information, wherein the synchronization prompt information includes a viewing control and a cancel control; After the user selects the viewing control, sending a request to the server; Receive a playback record of the display device fed back by the server, wherein the playback record includes an audio and video identifier and a playback progress, the playback record is a playback record of the audio and video played by the display device before the electronic device sends an acquisition request to the server, and the playback record is a playback record of the audio and video when the first user leaves the preset viewing range when the display device plays the audio and video and receives a detection signal from a human body detector indicating that the first user and the second user are in the preset viewing range of the display device, and the display device continues to play the audio and video when the first user leaves the preset viewing range; The audio and video are acquired according to the audio and video identifier, and the audio and video are played from the playback progress.

11. A method for synchronizing audio and video progress, characterized in that: include: Receive a first operation input by a user, and play audio and video according to the received first operation; receiving a detection signal from a human body detector; When the detection signal indicates that the first user and the second user are in a preset viewing range of the display device and the first user leaves the preset viewing range, a playback record of the audio and video when the first user leaves the preset viewing range is sent to the server, and the display device is controlled to continue playing the audio and video; the playback record includes an audio and video identifier and a playback progress; Among them, the playback record is used to enable the server to send the playback record to the electronic device when receiving the acquisition request sent by the electronic device in response to the user's trigger operation; without pausing the playback of the audio and video of the display device, the electronic device continues to play the audio and video from the playback progress according to the received playback record.

12. A method for synchronizing audio and video progress, characterized in that: include: Receive trigger operation; In response to the trigger operation, sending a request to the server; Receive a playback record of the display device fed back by the server, wherein the playback record includes an audio and video identifier and a playback progress, the playback record is a playback record of the audio and video played by the display device before sending an acquisition request to the server, and the playback record is a playback record of the audio and video when the display device plays the audio and video and, when the first user leaves the preset viewing range in the case where a detection signal of a human body detector is received indicating that the first user and the second user are in the preset viewing range of the display device, the first user leaves the preset viewing range, and the display device continues to play the audio and video when the first user leaves the preset viewing range; The audio and video are acquired according to the audio and video identifier, and the audio and video are played from the playback progress.

Images