Display device, method, apparatus, storage medium, and program product

Abstract

The embodiment of the invention provides a display device, method and device, a storage medium and a program product. The display device comprises a first display screen used for displaying an image. And the at least one second display screen is used for displaying auxiliary information corresponding to the image. And the processor is connected with the first display screen and the second display screen and is configured to search the auxiliary information corresponding to the image in response to a search instruction triggered by a user for the image under the condition that the first display screen displays the image. And displaying the auxiliary information on the second display screen. The display device is used for achieving the effect of improving the efficiency and experience of providing associated information related to watched content for a user.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a display device, method, apparatus, storage medium, and program product. Background Technology

[0002] With evolving user needs, users now require information related to the content they watch on television. For example, when watching a movie, users need information beyond the visuals, such as actor details, synopsis, and music. Similarly, when watching sports, users need information about athletes and match schedules. However, current methods for providing this information are inefficient and offer a poor user experience.

[0003] Therefore, improving the efficiency and user experience of providing users with relevant information related to the content they are watching is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a display device, method, apparatus, storage medium, and program product to improve the efficiency and user experience of providing users with relevant information related to the content they are viewing.

[0005] In a first aspect, embodiments of this application provide a display device, including:

[0006] The first display screen is used to display images;

[0007] At least one second display screen is used to display auxiliary information corresponding to the image;

[0008] The processor connected to the first display screen and the second display screen is configured to:

[0009] When an image is displayed on the first display screen, the auxiliary information corresponding to the image is searched in response to a search command triggered by the user for the image;

[0010] The auxiliary information is displayed on the second display screen.

[0011] Optionally, the step of searching for the auxiliary information corresponding to the image in response to a search command triggered by the user for the image includes:

[0012] Obtain the user's focus in the image;

[0013] Search for the auxiliary information based on the search instructions and the points of interest.

[0014] Optionally, the display device further includes:

[0015] An eye-tracking sensor connected to a first display screen is used to track the user's gaze at the point of interest in the image.

[0016] The step of obtaining the user's focus point in the image includes:

[0017] Obtain the target position of the user's gaze in the image;

[0018] The point of interest is determined based on the target location and the image.

[0019] Optionally, the display device further includes:

[0020] A sound acquisition device is used to acquire the user's voice commands;

[0021] The step of responding to a user's search command for the image and searching for the auxiliary information corresponding to the image includes:

[0022] Receive the voice command;

[0023] If the voice command includes a search command, the auxiliary information is searched based on the voice command and the image.

[0024] Optionally, the processor is further configured to:

[0025] If the voice command includes an interactive command, the interactive information in the interactive command shall be obtained;

[0026] The interactive information is displayed on the second display screen.

[0027] Optionally, the interactive instructions include bullet screen instructions, and the interactive information includes bullet screen content.

[0028] Optionally, the display device further includes:

[0029] A retractable bracket connected to the first display screen and the second display screen is used to position the second display screen behind the first display screen when it is in the first state, and to be side by side with the first display screen when it is in the second state.

[0030] The processor is also configured to:

[0031] When the second display screen is in the first state, in response to the search command triggered by the user for the image, the second display screen is switched to the second state.

[0032] Optionally, switching the second display screen to the second state in response to the search command triggered by the user for the image includes:

[0033] Obtain the user's location information;

[0034] Based on the location information, the second display screen is switched to the target location corresponding to the location information in the second state.

[0035] Optionally, the processor is further configured to:

[0036] If the duration for which the auxiliary information is displayed on the second display screen is greater than or equal to a preset duration, the second display screen is switched back to the first state; and / or, in response to a user's control command, the second display screen is switched back to the first state; and / or, after the image displayed on the first display screen undergoes a change, the second display screen is switched back to the first state.

[0037] In a second aspect, embodiments of this application provide a display method applied to the display device described in any one of the first aspects, the method comprising:

[0038] When an image is displayed on the first display screen, the auxiliary information corresponding to the image is searched in response to a search command triggered by the user for the image;

[0039] The auxiliary information is displayed on the second display screen.

[0040] Thirdly, embodiments of this application provide a display device, the device comprising:

[0041] The processing module is configured to, when an image is displayed on the first display screen, search for the auxiliary information corresponding to the image in response to a search command triggered by the user for the image;

[0042] A control module is used to display the auxiliary information on the second display screen.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the second aspect and / or various possible implementations of the second aspect as described above.

[0044] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the second aspect and / or various possible implementations of the second aspect as described above.

[0045] The display device, method, apparatus, storage medium, and program product provided in this application include a first display screen, at least one second display screen, and a processor. The processor, while displaying an image on the first display screen, responds to a user's search command triggered by the image, searches for auxiliary information corresponding to the image, and displays the auxiliary information on the second display screen. Since users do not need to switch to other devices or interfaces to perform the search, and the main and secondary displays have clearly defined functions and do not interfere with each other, compared to the problems of poor timeliness, cumbersome operation, high difficulty in obtaining information, and low efficiency caused by obtaining related information through other personal devices or switching interfaces in the prior art, this method can present auxiliary information in real time and accurately while the user continuously views the main screen content, significantly improving information acquisition efficiency and viewing experience, thereby achieving an efficient, convenient, and immersive multi-tasking information interaction effect. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0047] Figure 1 A schematic diagram illustrating a scenario for obtaining associated information provided in this application;

[0048] Figure 2 This is a schematic diagram of the structure of a display device provided in an embodiment of this application;

[0049] Figure 3 This is a schematic diagram of the structure of another display device provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure of another display device provided in an embodiment of this application;

[0051] Figure 5 This is a schematic diagram of the structure of another display device provided in an embodiment of this application;

[0052] Figure 6 A schematic flowchart illustrating a display method provided in an embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the structure of a display device provided in an embodiment of this application.

[0054] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0055] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0056] First, let me explain the terms used in this application:

[0057] Eye tracking refers to a technology that uses sensors to capture eye movement information and uses algorithms to measure the direction of gaze, the position of the gaze point, and the eye movement pattern in real time. It can convert the physiological characteristics of the human eye into computable data and realize intelligent control of "gaze as interaction".

[0058] Artificial Intelligence (AI) models refer to models that, based on natural language processing and machine learning technologies, dynamically analyze user intent and call upon structured / unstructured data resources through semantic understanding, multi-turn dialogue management, and knowledge graph retrieval to achieve precise matching of needs and personalized responses.

[0059] Currently, when users use display devices, if they need to obtain information related to the content they are watching, they mainly obtain such information through the following two methods.

[0060] Method 1: Obtain associated information through other personal terminal devices.

[0061] Figure 1 A schematic diagram illustrating a method for obtaining related information provided in this application, such as... Figure 1 As shown, users watch content through a display device, which can be, for example, a television, projector, computer, mobile phone, or tablet. If a user needs to obtain related information about the content being played (such as actor information, video description, music, or other information beyond the video content itself, or athlete information and match schedules in sports events), the user typically uses other personal terminal devices such as computers, mobile phones, or tablets to search for the relevant information. For example, a user can enter their search terms into a search engine on another personal terminal device to search for the relevant information, or they can take a picture of the video playback and search for that image.

[0062] Method 2: Obtain related information by switching interfaces.

[0063] In this approach, taking the playback content viewed by the user as an example displayed on a computer, mobile phone, tablet, or other display device, the user can switch to a browser interface or AI assistant interface when they need to obtain related information corresponding to the playback content. They can then enter the required search information to display the searched related information in the browser interface, AI assistant interface, or other interfaces, thereby satisfying the user's need to obtain related information corresponding to the playback content.

[0064] Based on the scenarios described above, it is clear that existing technologies, whether obtaining related information through other personal devices or by switching interfaces, suffer from poor timeliness in information retrieval. Furthermore, users need to frequently switch between multiple interfaces or electronic devices, severely impacting their viewing experience. Moreover, if users are unfamiliar with the content being played, they cannot quickly search for content that meets their needs for related information, requiring them to resort to various methods of searching and inference, further exacerbating the technical problems of high difficulty and low efficiency in obtaining related information.

[0065] In view of this, this application provides a display device comprising multiple displays. A first display shows content viewed by a user (such as an image). The processor of the display device can respond to a user's search command triggered by the content displayed on the first display, retrieve auxiliary information corresponding to the viewed content, and display the auxiliary information on at least one second display. This display device enables the accurate retrieval of required auxiliary information based on the content displayed on the first display and the user's search command, without affecting the display of the content viewed on the first display (e.g., without pausing the display or switching the first display to another interface). This ensures that the display of auxiliary information does not interfere with the display of the content viewed by the user, thereby improving the efficiency and accuracy of auxiliary information retrieval and enhancing the user's viewing experience.

[0066] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0067] Figure 2 This is a schematic diagram of the structure of a display device provided in an embodiment of this application. Figure 2 As shown, the display device may include: a first display screen, at least one second display screen, and a processor.

[0068] The display device can be any device with a display screen, such as a television or an all-in-one computer. The first display screen can be the main display screen of the display device, and the second display screen can be a secondary display screen. The number of second display screens can be configured according to actual needs, such as one, two, or three. This application does not limit the size of the first and second display screens. Optionally, the second display screen can be located to the left, right, top, or bottom of the first display screen.

[0069] The processor is connected to both the first and second displays. The processor can be deployed inside the first or second display, or it can be deployed outside but connected to both displays. For example, the display device is a computer, the processor is deployed in the host, and the first and second displays are computer monitors. Specifically, the processor can be, for example, a Central Processing Unit (CPU) or a Graphics Processing Unit (GPU).

[0070] The first display screen is used to display images, such as video resources (e.g., movies, TV series, TV programs, live broadcasts, etc.) or image resources. Optionally, the first display screen can also play audio resources, such as playing the audio corresponding to the video while playing the video; or, the first display screen can also play audio resources independently.

[0071] When an image is displayed on the first screen, the processor can respond to a user's search command triggered by the image and search for corresponding auxiliary information.

[0072] One possible implementation is that users can input search commands through peripherals connected to the display device, such as remote controls, keyboards, or microphones. When using a remote control to input search commands, users can input text search commands, or, for remote controls with voice recognition capabilities, input voice search commands.

[0073] Another possible implementation is that users can activate the voice recognition function of the display device (such as by direct voice activation or by activating it via a function button on a remote control) and directly input voice search commands through the microphone deployed on the display device.

[0074] The processor can perform corresponding searches (such as searching from a local database and / or the network) based on the image currently displayed on the first display screen and the received search instructions to obtain auxiliary information corresponding to the image.

[0075] After obtaining the auxiliary information corresponding to the image, the auxiliary information can be displayed on the second display screen.

[0076] Optionally, when multiple second displays exist, different types of auxiliary information can be displayed on different second displays. For example, if the auxiliary information retrieved includes actor introductions, plot background, and related product recommendations, the actor introductions can be displayed on the left second display, the plot background on the right second display, and the related product recommendations on the top or bottom second display. Optionally, which display shows which auxiliary information can be adjusted according to preset layouts or user preferences.

[0077] Optionally, the layout of the auxiliary information displayed on the same second display screen can also be adjusted according to a preset layout or user preferences.

[0078] The display device provided in this application includes a first display screen, at least one second display screen, and a processor. When an image is displayed on the first display screen, the processor, in response to a user's search command triggered by the image, searches for auxiliary information corresponding to the image and displays the auxiliary information on the second display screen. Since the user does not need to switch to other devices or interfaces to perform the search, and the main and secondary display screens have clearly defined functions and do not interfere with each other, compared to the problems of poor timeliness, cumbersome operation, high difficulty in obtaining information, and low efficiency caused by obtaining related information through other personal devices or switching interfaces in the prior art, this device can present auxiliary information in real time and accurately while the user continuously views the main screen content, significantly improving information acquisition efficiency and viewing experience, thereby achieving an efficient, convenient, and immersive multi-tasking information interaction effect.

[0079] Optionally, the search command triggered by the user for an image can be a search command for the entire image displayed on the first display screen. Alternatively, it can be a search command for a portion of the image displayed on the first display screen. For example, assuming the first display screen is displaying a movie scene, the user can directly search for the movie's synopsis information based on that movie scene; in this case, the search command is a search command for the entire image displayed on the first display screen. Alternatively, the user can search for the actor's personal information for a specific character among multiple characters included in the movie scene; in this case, the search command is a search command for the image area where that character is located in the image displayed on the first display screen.

[0080] The following section details how to implement a search command for a portion of an image displayed on the first screen, and how to search for corresponding auxiliary information within that image.

[0081] The processor can first obtain the user's focus in the image, and then search for auxiliary information based on the search instructions and focus.

[0082] The point of interest can be input by the user through a ray device, such as pointing to a specific location in the image displayed on the first screen using a remote control or laser pointer. The processor determines the location of the user's point of interest in the image based on the pointing location, and then determines the content of the image area where the point of interest is located. For example, a user can point a remote control at the face of a specific character in the image, causing the processor to determine that the user's point of interest is that specific character.

[0083] Alternatively, the point of interest can also be determined by the user through voice input. For example, based on the received voice input from the user, the processor determines the location or region of the point of interest in the image based on the descriptive information related to the point of interest in the voice. For instance, the user can voice input "What is the name of the actor who plays the second character from the left in the video?", and the processor can determine the user's point of interest as the location of the second character from the left in the frame based on this descriptive information.

[0084] Alternatively, the focus can also be determined using eye-tracking. When determined using eye-tracking, Figure 3 This is a schematic diagram of another display device provided in an embodiment of this application. Figure 3 As shown, the display device may also include an eye-tracking sensor.

[0085] The eye-tracking sensor can be, for example, a pupil-corneal reflection sensor based on an infrared light source and an image sensor, a miniature eye tracker based on a near-eye display, or a visual camera module integrated into the bezel of a display device.

[0086] An eye-tracking sensor is connected to both a first display screen and a processor to track the user's gaze point in an image. Specifically, the eye-tracking sensor can accurately calculate the coordinates of the gaze point by capturing the relative positional change between the center of the user's pupil and the corneal reflective spot, and transmit this gaze point data to the processor so that the processor can determine the target position of the user's gaze in the image. How to determine the position of the user's gaze on the display screen through eye tracking can be found in existing technologies, and this application will not elaborate on that.

[0087] After the processor obtains the target position of the user's gaze in the image displayed on the first screen, it can determine the user's focus based on the target position and the image displayed on the first screen. Specifically, the processor can determine the user's focus within the image based on the image area corresponding to the target position. For example, when a user is watching a sports event, in addition to focusing on the athletes' faces, eye tracking can identify areas where the user's gaze is focused for an extended period, such as the score display area, a specific athlete's jersey number, advertising boards, or key match actions (like a shot). This allows the processor to present score updates, athlete data, advertising product information, or action replays as supplementary information. Similarly, when watching a movie, in addition to characters' faces, the processor can identify the user's attention to props (such as antiques appearing in the film), scenes (such as buildings from a specific historical period), special effects (such as explosion scenes), or hidden Easter eggs (such as humorous slogans in the background), and push relevant analysis or behind-the-scenes content.

[0088] The display device provided in this application embodiment can also achieve a more natural and seamless user interaction experience through an eye-tracking sensor. This means it can capture the user's gaze focus in real time, accurately identifying the area of ​​interest without requiring the user to actively operate peripherals or use voice input. Combined with content analysis of the area of ​​interest, it can search for corresponding auxiliary information. This method not only eliminates the steps and breaks in traditional search but also seamlessly integrates information presentation and content viewing through a multi-screen collaborative architecture. This allows users to obtain auxiliary information more quickly and conveniently while enjoying immersive viewing, improving information acquisition efficiency and convenience, and enhancing the user experience.

[0089] Below, taking voice commands as an example, we will provide a detailed explanation of how to respond to user-triggered search commands for images and search for the corresponding auxiliary information. Figure 4 This is a schematic diagram of the structure of another display device provided in an embodiment of this application. Figure 4 As shown, the display device may also include: a sound acquisition device.

[0090] The sound acquisition device can be, for example, a microphone, a microphone array, or a sound wave receiver. The sound acquisition device is used to collect the user's voice commands. Optionally, the sound acquisition device can be integrated into the display device bezel, remote control, or external device. Through high-sensitivity sound pickup and intelligent noise reduction algorithms, it accurately captures the user's voice content, ensuring accurate command recognition even under ambient noise interference.

[0091] One possible implementation is that the user can activate the voice recognition function, putting the sound acquisition device into a voice command acquisition state to collect the user's voice signal and obtain the voice command. For example, the user can activate the voice recognition function of the display device through a function button on a remote control, or through a specific voice wake-up command (such as "Hello, Xiao X").

[0092] Another possible implementation is that the user can directly input a voice command containing specific audio content. When the display device detects this specific audio content, it retrieves the command content from the voice command. For example, the user can say "Please help me search for XX content," and when the display device detects the specific audio content "Please help me search," it retrieves the command content "Search for XX content."

[0093] When a user issues a voice command, the sound acquisition device transmits the acquired voice signal to the processor. The processor can first preprocess the voice signal, including noise reduction and echo cancellation, to improve the accuracy of speech recognition. Then, the processor can call the speech recognition module to convert the voice signal into text commands. This speech recognition module can, for example, be trained based on deep learning algorithms and capable of understanding complex semantics in natural language.

[0094] For example, when a user says "What era is this prop from?" while watching a movie, the sound capture device receives the voice command and transmits it to the processor. The processor converts the speech into text and then uses natural language processing technology to interpret the intent of the command.

[0095] When the instruction intent includes a search instruction, the processor can directly search for auxiliary information based on the instruction intent recognized in the voice command according to the screen displayed on the first display device. Alternatively, the processor can search for auxiliary information based on the point of interest determined by any of the aforementioned methods (such as the point of interest pointed to by the remote control, the point of interest recognized by the eye-tracking sensor, etc.) and the instruction intent recognized in the voice command.

[0096] For example, the processor can call the AI ​​model to perform a corresponding search on the screen (or the point of interest in the screen) displayed by the first display device and the search command, so as to obtain auxiliary information corresponding to the screen (or point of interest) and the search command.

[0097] In one possible implementation, voice commands may include not only the aforementioned search commands but also interactive commands. These interactive commands could include, for example, commands for sending comments (bullet comments), emoticons, display effects, virtual gifts, and voting. Specifically, bullet comments are used by users to send comments via voice, which are displayed in real-time on a second screen for interaction with other viewers. Emoticons are used by users to trigger specific emoticons (such as likes or clapping) via voice, displaying dynamic emoticon effects on the second screen. Display effects are used by users to adjust screen parameters (such as brightness and color mode) or switch viewing angles (such as first-person / third-person perspective). Virtual gift commands are used by users to select virtual gifts (such as flowers or badges) via voice, with a gifting animation playing on the second screen. Voting commands are used by users to participate in real-time voting.

[0098] In this implementation, the processor can acquire interactive information from interactive instructions when the voice command includes such instructions. For example, the processor can recognize interactive instructions by identifying keywords related to interactive instructions in the voice command, such as "input a comment," "send an emoji," "adjust display effects," "switch perspective," and "give a gift." In this case, the processor can further identify content in the voice command other than keywords related to interactive instructions to obtain the interactive information included in the interactive instructions.

[0099] For example, when the image displayed on the first display screen is a live broadcast, the user can issue a voice command "Enter the comment 'This ball is so exciting'", and the processor can recognize the voice command as a comment interaction command by the keyword "Enter the comment", and extract the subsequent content "This ball is so exciting" as the interaction information (i.e. the comment content), and display the comment content in the comment area of ​​the second display screen.

[0100] For example, a user can issue a voice command "send an emoji 'laugh'". The processor recognizes the voice command as an emoji interaction command by the keyword "send an emoji" and extracts the subsequent content "laugh" as the interaction information. Then, an exaggerated "laugh" emoji animation is dynamically displayed in the interactive area of ​​the second display screen, accompanied by corresponding fun sound effects to enhance the interactive atmosphere when watching.

[0101] For example, a user can issue a voice command "adjust display effect to cinema mode". The processor can then determine that the voice command is an interactive command for display effects based on the keyword "adjust display effect". Based on the interactive information "cinema mode", it can automatically optimize the screen contrast and sound effects configuration to correspond to cinema mode.

[0102] The display device provided in this application embodiment uses a processor to accurately identify search commands or interactive command types by real-time parsing of keywords and semantic content in voice commands. It also uses multimodal interaction technologies (such as eye tracking and screen content analysis) to obtain attention information, dynamically calling AI search models or preset interaction rules to quickly generate auxiliary information strongly related to the screen content or execute real-time interactive feedback on the second display screen. This achieves seamless content expansion and immersive interactive experience driven by voice commands, allowing users to efficiently obtain in-depth information or participate in social interactions while maintaining viewing continuity, and reducing the impact on the screen on the first display screen.

[0103] Figure 5 This is a schematic diagram of another display device provided in an embodiment of this application. Figure 5 As shown, the display device may also include:

[0104] A retractable bracket connected to the first and second display screens. This retractable bracket can be, for example, a hydraulically driven lifting bracket or an electrically adjustable push rod bracket. Alternatively, the display device may also include a foldable bracket connected to the first and second display screens. The retractable and foldable brackets serve the same purpose: to support and fix the first and second display screens, and to control the position of the first and / or second display screens when needed.

[0105] Taking a display device including a retractable bracket as an example, the retractable bracket is used to position the second display screen behind the first display screen when it is in the first state, and to place it side by side with the first display screen when it is in the second state.

[0106] When the user does not need to use the second display screen, the position of the second display screen can be controlled by the retractable bracket to keep it in its first state, even if it is located behind the first display screen. This reduces the interference of the second display screen on the user's view of the first display screen.

[0107] When the user needs to use the second display screen, the position of the second display screen can be controlled by the retractable bracket to put it in a second state, where the second display screen is placed side by side with the first display screen. For example, the second display screen can be placed anywhere to the left, right, top, or bottom of the first display screen. In this way, if the user needs to view auxiliary information corresponding to the image displayed on the first display screen while viewing the image on the first display screen, they can view the content displayed on the second display screen to obtain that auxiliary information.

[0108] One possible implementation is that the user can manually adjust the position of the first or second display screen by adjusting the retractable bracket. For example, the position of the second display screen can be adjusted manually by adjusting the retractable bracket to place it in either the first or second state. Alternatively, the position of the first display screen can also be adjusted manually by adjusting the retractable bracket to place the second display screen in either the first or second state.

[0109] Another possible implementation is that the processor can automatically adjust the position of the first or second display screen by adjusting the retractable bracket according to the received user control command, so that the second display screen is in either a first state or a second state. When the second display screen is in the first state, the processor can control the adjustment of the position of the first or second display screen according to the received first user control command, so that the second display screen switches from the first state to the second state; when the second display screen is in the second state, the processor can control the adjustment of the position of the first or second display screen according to the received second user control command, so that the second display screen switches from the second state to the first state.

[0110] Optionally, when the second display screen is in the second state, its location on which side of the first display screen it is located can be adaptively adjusted according to a preset strategy or user preference, and this application does not limit this.

[0111] The following section will detail how to control the position of the second display screen by taking the example of the processor automatically adjusting the retractable bracket to adjust the position of the second display screen.

[0112] First, let's explain when to control the second display screen to switch from the first state to the second state:

[0113] In one possible implementation, the processor can switch the second display screen to a second state in response to a user's search command triggered by an image while the second display screen is in the first state. In this implementation, since the processor has received the user's search command for the image, it needs to display the auxiliary information found based on the image and the search command on the second display screen. Therefore, it is necessary to move the second display screen to a position convenient for the user to view. Thus, the processor can control a retractable stand to switch the second display screen to the second state, making it easier for the user to view the auxiliary information on the second display screen. In this case, the search command is the aforementioned first user control command.

[0114] Another possible implementation is that, while the second display screen is in the first state, the processor can switch the second display screen to the second state based on a received control command specifically for controlling the position of the second display screen. In this implementation, the control command can be, for example, a voice command (captured by the display device itself, or by a remote control, external microphone, or other peripheral device), a control command triggered by a user pressing a dedicated function button on the display device, or a control command triggered by the user pressing a dedicated function button on the remote control corresponding to the display device. In this case, the user can freely adjust when to switch the second display screen from the first state to the second state according to their own needs.

[0115] Next, we will explain when to control the second display screen to switch from the second state to the first state:

[0116] One possible implementation is that the processor can switch the second display screen back to the first state if the duration of the auxiliary information displayed on the second display screen is greater than or equal to a preset duration. The preset duration can be pre-configured by the display device or set by the user according to personal preference, such as 10 seconds, 30 seconds, or 1 minute. When the duration of the auxiliary information displayed on the second display screen is greater than or equal to the preset duration, it indicates that the user has finished viewing the auxiliary information. Switching the second display screen back to the first state at this time can reduce the interference of the second display screen on the user's viewing of the first display screen and improve the user's viewing experience. Alternatively, the processor can detect updates to the auxiliary information on the second display screen. If no update occurs within the preset duration, the second display screen can also be switched back to the first state; if an update occurs, the timing restarts.

[0117] Another possible implementation is that the processor can respond to user control commands and switch the second display back to the first state. Specifically, users can issue control commands in various ways, such as directly triggering the second display to turn off or switch states via specific function buttons on the remote control (such as the "Turn off auxiliary screen" button); or users can speak specific voice commands (such as "Turn off second screen" or "End auxiliary information display"). After the voice acquisition device collects the voice command, it is transmitted to the processor. After the processor recognizes the command content, it immediately controls the second display to switch back to the first state. This interaction method is more convenient and efficient, meeting users' personalized operation needs in different scenarios.

[0118] Another possible implementation is that the processor can switch the second display back to the first state after the image displayed on the first display changes. For example, when the first display switches from playing the current video to an advertisement, program change, or transitions to another scene (such as from a movie screen to a menu interface), the content changes significantly, and the auxiliary information previously displayed for the previous screen may no longer be applicable or the user's attention to it may decrease. In this case, the processor automatically detects the change in the image on the first display and promptly switches the second display back to the first state. This avoids mismatch between auxiliary information and the new image or distraction of the user from the new image, ensuring that the user can focus on the new content on the first display and improving the overall smoothness and comfort of the viewing experience.

[0119] The display device provided in this application embodiment, by configuring a retractable bracket and designing a manual or automatic dual-mode control mechanism, combined with multi-dimensional triggering conditions such as user commands, changes in screen content, and duration of auxiliary information display, can realize intelligent switching between hiding and side-by-side display of the second display screen. This ensures the immersive viewing experience of the main screen while providing auxiliary information display space as needed, and takes into account both user operation convenience and display device resource optimization.

[0120] In one possible implementation, the processor can also obtain the user's location information and, based on the location information, switch the second display screen to the target location corresponding to the location information in the second state.

[0121] The display device can use a built-in position detection sensor to acquire the user's location information, thereby determining the user's distance and orientation relative to the display device. This position detection sensor can include, for example, a camera, an infrared sensor, or a millimeter-wave radar. Alternatively, if the display device has a corresponding remote control, the display device can also infer the user's location information based on the remote control's position.

[0122] After acquiring the user's location information, the processor can switch the second display screen to the target position corresponding to the location information in the second state. Specifically, the processor can adjust the orientation and / or position of the second display screen in the second state based on the location information. For example, the second display screen can be oriented towards the user's location based on the user's location information. Alternatively, the position of the second display screen can be adjusted based on the distance between the user's location and the display device's location, maintaining a suitable distance between the second display screen and the user's location to improve the user's viewing experience. Optionally, the distance between the second display screen and the user's location can be a preset strategy of the display device or a user-defined value.

[0123] The display device provided in this application embodiment obtains user location information and dynamically adjusts the position of the second display screen so that the second display screen is always in the user's comfortable field of vision, avoiding viewing fatigue or information omissions caused by improper position, while meeting the personalized viewing needs of different users for auxiliary information in different scenarios, thereby significantly improving the flexibility of human-computer interaction and user experience of the display device.

[0124] Figure 6 This is a flowchart illustrating a display method provided in an embodiment of this application. Figure 6 As shown, the method is applied to the processor of the display device mentioned in any of the foregoing embodiments, and the method may include:

[0125] S601, When an image is displayed on the first display screen, in response to a search command triggered by the user for the image, search for auxiliary information corresponding to the image.

[0126] S602, Display auxiliary information on the second display screen.

[0127] Optionally, S601 may specifically include: acquiring the user's focus points in the image; and searching for auxiliary information based on the search instructions and focus points.

[0128] Optionally, when the display device further includes an eye-tracking sensor connected to the first display screen for tracking the user's gaze point in the image, obtaining the user's gaze point in the image may specifically include: obtaining the target position of the user's gaze in the image; and determining the gaze point based on the target position and the image.

[0129] Optionally, when the display device further includes a sound acquisition device for acquiring user voice commands, S601 may specifically include: receiving voice commands. If the voice command includes a search command, auxiliary information is searched based on the voice command and the image.

[0130] Optionally, the method may further include: if the voice command includes an interactive command, acquiring interactive information from the interactive command; and displaying the interactive information on a second display screen. The interactive command may include a bullet screen command, and the interactive information may include bullet screen content.

[0131] Optionally, when the display device further includes a retractable bracket connected to the first display screen and the second display screen, and used to position the second display screen behind the first display screen when in the first state and alongside the first display screen when in the second state, the method may further include: when the second display screen is in the first state, switching the second display screen to the second state in response to a user-triggered search command for an image.

[0132] Optionally, in response to a user's search command triggered by an image, the second display screen is switched to a second state, which may specifically include: obtaining the user's location information. Based on the location information, the second display screen is switched to the target location corresponding to the location information in the second state.

[0133] Optionally, the method may further include: switching the second display screen back to the first state if the duration for which the auxiliary information is displayed on the second display screen is greater than or equal to a preset duration; and / or switching the second display screen back to the first state in response to a user's control command; and / or switching the second display screen back to the first state after a change in the image displayed on the first display screen.

[0134] It should be noted that, Figure 6 The specific execution process of the steps shown in the embodiments can be found in any of the above embodiments, and will not be repeated here.

[0135] Figure 7 This is a schematic diagram of the structure of a display device provided in an embodiment of this application. Figure 7 As shown, the display device includes: a processing module 11 and a control module 12.

[0136] The processing module 11 is used to search for auxiliary information corresponding to the image in response to a search command triggered by the user for the image when the image is displayed on the first display screen.

[0137] The control module 12 is used to display auxiliary information on the second display screen.

[0138] Optionally, processing module 11 is specifically used to obtain the user's focus points in the image. Based on the search instructions and focus points, it searches for auxiliary information.

[0139] Optionally, if the display device also includes an eye-tracking sensor connected to the first display screen, and the eye-tracking sensor is used to track the user's gaze in the image, the processing module 11 is specifically used to obtain the target position of the user's gaze in the image. Based on the target position and the image, the point of interest is determined.

[0140] Optionally, if the display device also includes a sound acquisition device for acquiring user voice commands, the processing module 11 is specifically used to receive the voice commands. If the voice command includes a search command, auxiliary information is searched based on the voice command and the image.

[0141] Optionally, the processing module 11 is further configured to acquire interactive information from the interactive instructions when the voice instructions include interactive instructions. The control module 12 is further configured to display the interactive information on a second display screen.

[0142] Optionally, interactive commands include bullet screen commands, and interactive information includes bullet screen content.

[0143] Optionally, the display device also includes a retractable bracket connected to the first display screen and the second display screen. The retractable bracket is used to position the second display screen behind the first display screen when it is in the first state and to place it side by side with the first display screen when it is in the second state. In this case, the control module 12 is also used to switch the second display screen to the second state in response to a user's search command triggered by an image when the second display screen is in the first state.

[0144] Optionally, the control module 12 is specifically used to acquire the user's location information. Based on the location information, the second display screen is switched to the target location corresponding to the location information in the second state.

[0145] Optionally, the control module 12 is also configured to switch the second display screen back to the first state when the duration of the auxiliary information displayed on the second display screen is greater than or equal to a preset duration, and / or switch the second display screen back to the first state in response to a user's control command, and / or switch the second display screen back to the first state after the image displayed on the first display screen changes.

[0146] The display device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0147] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0148] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0149] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0150] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0151] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0152] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0153] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0154] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0155] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0156] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A display device, characterized in that, The display device includes: The first display screen is used to display images; At least one second display screen is used to display auxiliary information corresponding to the image; The processor connected to the first display screen and the second display screen is configured to: When an image is displayed on the first display screen, the auxiliary information corresponding to the image is searched in response to a search command triggered by the user for the image; The auxiliary information is displayed on the second display screen.

2. The display device according to claim 1, characterized in that, The step of responding to a user's search command for the image and searching for the auxiliary information corresponding to the image includes: Obtain the user's focus in the image; Search for the auxiliary information based on the search instructions and the points of interest.

3. The display device according to claim 2, characterized in that, The display device further includes: An eye-tracking sensor connected to a first display screen is used to track the user's gaze at the point of interest in the image. The step of obtaining the user's focus point in the image includes: Obtain the target position of the user's gaze in the image; The point of interest is determined based on the target location and the image.

4. The display device according to claim 1, characterized in that, The display device further includes: A sound acquisition device is used to acquire the user's voice commands; The step of responding to a user's search command for the image and searching for the auxiliary information corresponding to the image includes: Receive the voice command; If the voice command includes a search command, the auxiliary information is searched based on the voice command and the image.

5. The display device according to claim 4, characterized in that, The processor is also configured to: If the voice command includes an interactive command, the interactive information in the interactive command shall be obtained; The interactive information is displayed on the second display screen.

6. The display device according to claim 5, characterized in that, The interactive commands include bullet screen commands, and the interactive information includes bullet screen content.

7. The display device according to claim 3 or 4, characterized in that, The display device further includes: A retractable bracket connected to the first display screen and the second display screen is used to position the second display screen behind the first display screen when it is in the first state, and to be side by side with the first display screen when it is in the second state. The processor is also configured to: When the second display screen is in the first state, in response to the search command triggered by the user for the image, the second display screen is switched to the second state.

8. The display device according to claim 7, characterized in that, The step of switching the second display screen to the second state in response to the search command triggered by the user for the image includes: Obtain the user's location information; Based on the location information, the second display screen is switched to the target location corresponding to the location information in the second state.

9. The display device according to claim 7, characterized in that, The processor is also configured to: If the duration for which the auxiliary information is displayed on the second display screen is greater than or equal to a preset duration, the second display screen will be switched back to the first state. And / or, In response to the user's control command, the second display screen is switched back to the first state; And / or, After the image displayed on the first display screen changes, the second display screen is switched back to the first state.

10. A display method, characterized in that, Applied to the display device according to any one of claims 1-8, the method comprises: When an image is displayed on the first display screen, in response to a search command triggered by the user for the image, auxiliary information corresponding to the image is searched. The auxiliary information is displayed on a second display screen.

11. A display device, characterized in that, The device includes: The processing module is configured to, when an image is displayed on the first display screen, search for auxiliary information corresponding to the image in response to a search command triggered by the user for the image; A control module is used to display the auxiliary information on a second display screen.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in claim 10.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method of claim 10.

Images