Interface display method, device, system, equipment, storage medium and program product

Abstract

The present disclosure provides an interface display method, device, system, electronic device, computer readable storage medium and computer program product, relating to the technical field of computer. The method comprises: displaying a screen projection window and a local window on a current display interface, and obtaining first running data of an application in the screen projection window and second running data of an application in the local window; generating predicted operation information based on the display mode of the screen projection window and the local window on the current display interface, the first running data, the second running data and historical operation information, wherein the predicted operation information comprises predicted operations associated with the application in the screen projection window and / or predicted operations associated with the application in the local window; generating and displaying an operation interface for executing the predicted operations in the predicted operation information based on the predicted operation information, wherein the operation interface comprises a screen projection operation window and / or a local operation window. The method greatly improves the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an interface display method, apparatus, system, electronic device, and computer-readable storage medium. Background Technology

[0002] Currently, screen mirroring between electronic devices is widely used, providing users with a rich experience. However, while screen mirroring offers a wealth of experiences, it also requires users to perform numerous operations or issue a large number of commands, consuming a significant amount of user effort. Summary of the Invention

[0003] This disclosure provides an interface display method, apparatus, system, electronic device, computer-readable storage medium, and computer program product that can meet users' service needs while reducing the amount of user effort required, thus greatly improving the user experience.

[0004] In a first aspect, embodiments of this disclosure propose an interface display method applied to a target device for screen projection, comprising: displaying a projection window and a local window on a current display interface, and acquiring first running data of an application in the projection window and second running data of an application in the local window; generating predicted operation information based on the display mode of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information, wherein the predicted operation information includes predicted operations associated with the application in the projection window and / or predicted operations associated with the application in the local window; generating and displaying an operation interface for executing the predicted operations in the predicted operation information based on the predicted operation information, wherein the operation interface includes a projection operation window and / or a local operation window.

[0005] In one possible implementation, the method provided in the first aspect further includes: in response to determining that no actual operation information has been obtained within a first time period, operating the application in the projection operation window and / or the local operation window based on the predicted operation information, generating and displaying a predicted interface, wherein the predicted interface includes a predicted projection window and / or a predicted local window; and in response to obtaining actual operation information, obtaining and displaying an actual interface based on the actual operation information. Even when the user does not perform an actual operation, the relevant application can be automatically operated based on the predicted operation information, and the predicted interface can be automatically generated and displayed, enabling the user to obtain relevant services without needing to operate or issue commands, thus achieving an optimal user experience.

[0006] In one possible implementation, the predicted operation information includes predicted operations associated with applications in the projection window and predicted operations associated with applications in the local window. The process of operating the projection window and / or the applications in the local window based on the predicted operation information to generate and display a predicted interface further includes: sending information about the predicted operations associated with applications in the projection window to the projection source; receiving information about the predicted projection window from the projection source; operating the applications in the local window based on the predicted operations associated with applications in the local window to generate information about the predicted local window; and generating and displaying a predicted interface based on the predicted projection window information and the predicted local window information. When it is predicted that the user wants to operate the applications in the projection window and the applications in the local window, the predicted operation information associated with the applications in the projection window is sent to the projection source, which then generates information about the predicted projection window. Receiving the predicted projection window information from the projection source enables data synchronization between devices.

[0007] In one possible implementation, the aforementioned predictive display interface includes: generating a predicted display mode based on display mode preference information, wherein the predicted display mode includes a predicted display mode for the projection window and / or a predicted display mode for the local window; and displaying the predicted projection window and / or the predicted local window based on the predicted display mode. This can make the displayed predictive interface more in line with personalized needs.

[0008] In one possible implementation, the method provided in the first aspect further includes: in response to determining that the predicted operation information includes predicted operations associated with an application in the projection window but does not include predicted operations associated with an application in the local window, generating a control command instructing to adjust the audio-visual parameters associated with the application in the local window. Thus, in response to determining that operation of the application in the projection window is required but operation of the application in the local window is not required, if the application in the local window is currently running with audio-visual effects, a control command instructing to adjust the audio-visual parameters associated with the application in the local window is generated. This control command weakens the audio-visual parameters, reducing the impact of the local volume and lights on the user, allowing the user to focus their attention on the application in the projection window.

[0009] In one possible implementation, the target device for screen projection is an in-vehicle infotainment system. The method provided in the first aspect further includes: generating and outputting a recommendation message based on historical operation information and current driving information. The recommendation message may include content recommendations, providing users with recommendations that match the current driving environment and reflecting their driving experience.

[0010] In one possible implementation, the method provided by the first aspect further includes: determining whether an operation to stop displaying the prediction interface is obtained during a second time period; and, in response to no, storing the prediction operation information in a historical operation information database. Storing the prediction operation information as historical operation data can be used for subsequent prediction operations, improving prediction accuracy and further enhancing the user experience.

[0011] Secondly, embodiments of this disclosure provide an interface display device, comprising: a display acquisition module configured to display a projection window and a local window on a current display interface, and acquire first running data of an application in the projection window and second running data of an application in the local window; a first generation module configured to generate predicted operation information based on the display modes of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information, wherein the predicted operation information includes predicted operations associated with the application in the projection window and / or predicted operations associated with the application in the local window; and a second generation module configured to generate and display an operation interface for executing the predicted operations in the predicted operation information based on the predicted operation information, wherein the operation interface includes a projection operation window and / or a local operation window.

[0012] In one possible implementation, the apparatus provided in the second aspect further includes: a display generation module configured to, in response to determining that no actual operation information has been obtained within a first time period, operate the application in the projection operation window and / or the local operation window based on the predicted operation information, generate and display a predicted interface, wherein the predicted interface includes a predicted projection window and / or a predicted local window. An acquisition display module configured to, in response to obtaining actual operation information, obtain and display the actual interface based on the actual operation information.

[0013] In one possible implementation, the prediction operation information includes prediction operations associated with applications in the projection window and prediction operations associated with applications in the local window; the apparatus provided in the second aspect further includes: a sending module configured to send information about prediction operations associated with applications in the projection window to the projection source end; a receiving module configured to receive information about the predicted projection window from the projection source end; a third generating module configured to operate the applications in the local window based on the prediction operations associated with applications in the local window, and generate information about the predicted local window; and a generating display module further configured to generate and display a prediction interface based on the predicted projection window information and the predicted local window information.

[0014] In one possible implementation, the apparatus provided in the second aspect further includes: a fourth generation module configured to generate a predicted display mode based on display mode preference information, wherein the predicted display mode includes a predicted display mode for the projection window and / or a predicted display mode for the local window; and a display module configured to display the predicted projection window and / or the predicted local window based on the predicted display mode.

[0015] In one possible implementation, the apparatus provided in the second aspect further includes: a fifth generation module configured to generate a control command in response to determining that the prediction operation information includes prediction operations associated with an application in the projection window but does not include prediction operations associated with an application in the local window, indicating that the audio-visual parameters associated with the application in the local window should be adjusted.

[0016] In one possible implementation, the target device for screen projection is an in-vehicle infotainment system. The apparatus provided in the second aspect further includes: a generation output module configured to generate and output a recommendation message based on historical operation information and current driving information.

[0017] In one possible implementation, the apparatus provided in the second aspect further includes: a determining module configured to determine whether an operation to stop displaying the prediction interface is obtained during a second time period; and a storage module configured to store the prediction operation information in a historical operation information database in response to whether or not the operation is stopped.

[0018] Thirdly, this disclosure provides a screen mirroring system, including a screen mirroring source and a screen mirroring destination. The screen mirroring source is configured to establish a screen mirroring link with the screen mirroring destination. The screen mirroring destination is configured to: display a screen mirroring window and a local window on the current display interface; acquire first running data of an application in the screen mirroring window and second running data of an application in the local window; generate predicted operation information based on the display mode of the screen mirroring window and the local window on the current display interface, the first running data, the second running data, and historical operation information, wherein the predicted operation information includes predicted operations associated with applications in the screen mirroring window and / or predicted operations associated with applications in the local window; and generate and display an operation interface for executing the predicted operations in the predicted operation information based on the predicted operation information, wherein the operation interface includes a screen mirroring operation window and / or a local operation window.

[0019] Optionally, the target device for screen mirroring can implement the interface display method described in any of the implementation methods in the first aspect.

[0020] Fourthly, embodiments of this disclosure provide an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the interface display method as described in any implementation of the first aspect.

[0021] Fifthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the interface display method described in any implementation of the first aspect when executed.

[0022] In a sixth aspect, embodiments of this disclosure provide a computer program product including a computer program, which, when executed by a processor, can implement the interface display method as described in any implementation of the first aspect.

[0023] The technical solution provided in this disclosure involves an execution entity acquiring first running data of an application in a projection window and second running data of an application in a local window. By analyzing the display modes of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information, the entity predicts the user's next desired operation, generates predicted operation information, and displays an operation interface for performing the predicted operation based on this information. This method predicts the user's next desired operation based on the current running status of the projection window and the local window and the user's historical operation information, and automatically displays the operation interface without requiring the user to perform any related operations or issue any commands. This reduces the user's effort required while meeting their service needs, significantly improving the user experience.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0025] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0026] Figure 1 This is an exemplary system architecture to which this disclosure can be applied;

[0027] Figure 2 A flowchart illustrating an interface display method provided in this embodiment of the disclosure;

[0028] Figure 3 A flowchart illustrating another interface display method provided in this embodiment of the disclosure;

[0029] Figure 4 A structural block diagram of an interface display device provided in an embodiment of this disclosure;

[0030] Figure 5 This is a schematic diagram of the structure of an electronic device suitable for performing an interface display method, provided as an embodiment of the present disclosure. Detailed Implementation

[0031] To better understand this application, various aspects of this application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are merely illustrative of exemplary embodiments of this application and are not intended to limit the scope of this application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0032] It should be understood that expressions such as "comprising," "including," "having," "containing," and / or "comprising" are open-ended rather than closed-ended expressions in this specification, indicating the presence of the stated features, elements, and / or components, but not excluding the presence of one or more other features, elements, components, and / or combinations thereof. Furthermore, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features, not just individual elements in the list. Additionally, when describing embodiments of this application, the word "may" is used to mean "one or more embodiments of this application." And the term "exemplary" is intended to refer to examples or illustrations.

[0033] Unless otherwise specified, all terms used herein (including engineering and technical terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that, unless expressly stated herein, terms defined in common dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or overly formalized meaning.

[0034] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. Furthermore, unless explicitly limited or contradicted by the context, the specific steps included in the methods described in this application are not limited to the order in which they are described, but can be performed in any order or in parallel. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the interface display methods, apparatus, systems, electronic devices, and computer-readable storage media of the present disclosure can be applied.

[0036] like Figure 1 As shown, system architecture 100 may include a projection destination 101 and a projection source 102, which can be connected via a network. The network serves as the medium for providing a communication link between the projection destination 101 and the projection source 102. The network may include various connection types, such as wired or wireless communication links or fiber optic cables. System architecture 100 may be a projection system.

[0037] The projection destination 101 and projection source 102 can be either hardware or software. When the projection destination 101 is hardware, it can be various electronic devices, including but not limited to in-vehicle devices, central control screens, passenger-side screens, rear-seat screens, vehicle infotainment systems, smartphones, smart screens, tablets, laptops, and desktop computers, etc. When the projection destination 101 is software, it can be installed in the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitations are made here. When the projection source 102 is hardware, it can be various electronic devices, including but not limited to in-vehicle devices, central control screens, passenger-side screens, rear-seat screens, vehicle infotainment systems, smartphones, smart screens, tablets, laptops, and desktop computers, etc. When the projection source 102 is software, it can be installed in the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitations are made here.

[0038] It should be understood that Figure 1 The number of projection destinations and projection sources shown is merely illustrative. Depending on implementation needs, there can be any number of projection destinations and projection sources.

[0039] Please refer to Figure 2 , Figure 2 A flowchart of an interface display method provided in this disclosure embodiment, applied to a projection destination, wherein process 200 includes the following steps:

[0040] Step 201: Display the projection window and the local window on the current display interface, and obtain the first running data of the application in the projection window and the second running data of the application in the local window.

[0041] For example, the current display interface of the target device shows the casting window of the source device and the local window of the target device. It acquires first running data of the application displayed in the casting window and second running data of the application displayed in the local window. The first running data includes: the application's currently displayed page and / or ongoing events, etc. The second running data includes: the application's currently displayed page and / or ongoing events, etc.

[0042] Step 202: Generate predicted operation information based on the display modes of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information.

[0043] For example, display modes include embedded mode, full-screen display mode, and split-screen display mode.

[0044] For example, historical operation information includes various historical operation data performed by the user based on the interface before entering the application after powering on and / or the interface after launching the application, such as historical operations associated with the application in the screen mirroring window, historical operations associated with the application in the local window, and / or historical operations for setting the display mode, etc.

[0045] For example, the prediction operation information may include prediction operations associated with applications in the projection window, prediction operations associated with applications in the local window, prediction display modes, prediction operations associated with the projection window, and / or prediction operations associated with the local window. Optionally, the prediction operation information may also include prediction operations associated with applications that are not currently running.

[0046] For example, the target device can predict user behavior based on the display modes of the projection window and the local window in the current display interface, the first running data of the application in the projection window, the second running data of the application in the local window, the historical operations of the display mode, and the historical operations associated with the application. This includes predicting the operations that the user may perform on the application in the projection window, the operations that the user may perform on the application in the local window, and / or predicting the display modes of the projection window and the local window.

[0047] Optionally, the preset operations included in the predicted operation information can be independent operations within the projection window and the local window, or interactive operations executed between the projection window and the local window.

[0048] For example, independent operations can include operating an application within the projection window, or operating an application within the local window (taking an application as an example). Interactive operations can include customizing the content of the local window through the projection window, or customizing the content of the projection window through the local window, such as defining the interface theme, background, and layout.

[0049] Predicted action information can include a predicted action. For example, the action to close the navigation application after navigation is complete.

[0050] Predictive operation information can also include multiple predicted operations. For example, assuming the navigation application in the local window is running, the music application in the projected window is running, and the current display mode is not a split-screen display of the navigation application and the music application, the aforementioned execution entity detects the user's history of navigating to the first address multiple times in the past few days at this time, and generates a predicted operation to navigate to the first address. It also detects the user's history of using the music application to play playlists after performing navigation operations, and generates a predicted operation to play playlists. Furthermore, it can generate corresponding embedded mode predicted operations by detecting the user's usual habit of embedding the music application window in the navigation application window.

[0051] In this way, by analyzing the current state of the projection window and the local window, as well as the user's historical operation information, it is possible to make personalized predictions for the user's next one or more operations.

[0052] Optionally, the aforementioned execution entity can utilize the large model to generate predictive operation information based on the display modes of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information.

[0053] Step 203: Generate and display an operation interface based on the prediction operation information to perform the prediction operations in the prediction operation information.

[0054] For example, the user interface includes a screen mirroring window and / or a local operation window.

[0055] After the aforementioned execution entity generates the prediction operation information, it can display the operation interface for executing the prediction operation in the prediction operation information to the user. The user can perform the next operation on this operation interface, or the aforementioned execution entity can perform the next operation.

[0056] In the method provided in this embodiment, the executing entity obtains the first running data of the application in the projection window and the second running data of the application in the local window. Based on the display mode of the projection window and the local window in the current display interface, the first running data, the second running data, and historical operation information, it predicts the operation that the user wants to perform next, generates predicted operation information, and displays an operation interface for performing the predicted operation based on the predicted operation information. In this way, by predicting the operation that the user wants to perform next based on the current running status of the projection window and the local window and the user's historical operation information, and automatically displaying the operation interface, the operation interface can be displayed without the user having to perform related operations or issue commands. This can meet the user's service needs while reducing the user's effort, and greatly improve the user experience.

[0057] Please refer to Figure 3 , Figure 3A flowchart of another interface display method provided in this disclosure embodiment, applied to the target device for screen projection, wherein process 300 includes the following steps:

[0058] Step 301: Display the projection window and the local window on the current display interface, and obtain the first running data of the application in the projection window and the second running data of the application in the local window.

[0059] Step 302: Generate predictive operation information based on the display modes of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information.

[0060] Step 303: Generate and display an operation interface for performing the prediction operations in the prediction operation information based on the prediction operation information.

[0061] For the specific implementation of steps 301-303, please refer to steps 201-203 above, which will not be repeated here.

[0062] Step 304: In response to determining that no actual operation information was obtained within the first time period, the predicted operation information is used to operate the screen projection operation window and / or the application in the local operation window to generate and display the predicted interface.

[0063] For example, operational information includes the actual actions performed on the application or interface.

[0064] For example, the prediction interface may include a prediction projection window and / or a prediction local window. For instance, based on a prediction operation associated with an application in the projection window, the application in the projection operation window is manipulated to generate a prediction projection window. Similarly, based on a prediction operation associated with an application in the local window, the application in the local operation window is manipulated to generate a prediction local window. Optionally, the prediction operation information may also include prediction operations associated with the projection window and / or with the local window, such as closing the window (projection window and / or local window), enlarging or shrinking the window, etc.

[0065] In this embodiment, after seeing the operation interface, the user can either perform the operation they want to perform on the interface or choose not to, and the aforementioned execution entity will perform the predicted operation. Before performing the predicted operation, the execution entity can determine whether to perform the predicted operation by checking whether the user has actually performed the operation within a first time period. If it is determined that no actual operation information has been obtained within the first time period, the predicted operation is performed, and the application in the projection operation window and / or the local operation window is executed, and the predicted interface is generated and displayed. If it is determined that actual operation information has been obtained within the first time period, indicating that the user has performed the operation they wanted to perform, the aforementioned execution entity will not perform the predicted operation.

[0066] Optionally, after the aforementioned execution entity displays the operation interface, the user, upon seeing the interface, can issue a return command via touch, gesture, or voice, indicating that the user does not wish to perform the prediction operation. The aforementioned execution entity then returns to the previous interface based on the received return command. Alternatively, the user can issue a continue command via touch, gesture, or voice, instructing the aforementioned execution entity to perform the preset operation in the prediction operation information. Upon receiving this continue command, the aforementioned execution entity operates the projection operation window and / or the application in the local operation window based on the prediction operation information, generating and displaying the prediction interface.

[0067] For example, the aforementioned execution entity may, in response to determining that no actual operation information has been obtained and / or a continuation instruction has been received within the first time period, operate the projection operation window and / or the application in the local operation window based on the predicted operation information, and generate and display the predicted interface.

[0068] Step 305: In response to obtaining actual operation information, obtain and display the actual interface based on the actual operation information.

[0069] In this embodiment, after the user sees the operation interface, they perform the operation they want to perform on the operation interface. The execution entity obtains the actual operation information based on the operation, and obtains and displays the actual interface based on the actual operation information.

[0070] In some embodiments, the method provided in this disclosure may further include: determining whether an operation to stop displaying the prediction interface is obtained during a second time period; and in response to no, storing the prediction operation information in a historical operation information database.

[0071] For example, the operation of stopping the display of the prediction interface may include a user issuing a command to return to the previous interface, stop the prediction operation, or close the operation interface via touch, gesture, or voice. In response to determining that no operation to stop the display of the prediction interface has been received within a preset second time period, the execution entity stores the prediction operation information in a historical operation information database, storing the prediction operation information as historical operation data for use in subsequent prediction operations. Alternatively, in response to determining that an operation to stop the display of the prediction interface has been received within the preset second time period, the execution entity does not store the prediction operation information.

[0072] In some embodiments, the predicted operation information includes predicted operations associated with applications in the projection window and predicted operations associated with applications in the local window. Step 203 above, which operates the projection operation window and / or the applications in the local operation window based on the predicted operation information to generate and display the predicted interface, may include: sending the predicted operation information associated with the applications in the projection window to the projection source; receiving the predicted projection window information from the projection source; operating the applications in the local window based on the predicted operations associated with the applications in the local window to generate the predicted local window information; and generating and displaying the predicted interface based on the predicted projection window information and the predicted local window information.

[0073] For example, the aforementioned execution entity sends information about the predicted operation associated with the application in the projection window to the projection source end, and operates the application in its own window based on the predicted operation associated with the application in its own window, generating information predicting the local window. The projection source end receives the information about the predicted operation associated with the application in the projection window, generates information predicting the projection window based on this information, and sends the information predicting the projection window to the aforementioned execution entity, achieving data synchronization. The aforementioned execution entity receives the information predicting the projection window, and generates a predicted interface based on the predicted projection window information and the information predicting the local window.

[0074] Thus, when the prediction operation information includes prediction operations associated with the application in the projection window and prediction operations associated with the application in the local window, the prediction operation information associated with the application in the projection window is sent to the projection source, and the projection source generates the prediction information for the local window. The aforementioned execution entity combines the prediction information for the projection window and the prediction information for the local window to generate the prediction interface.

[0075] In some embodiments, the above-described display prediction interface may include: generating a prediction display mode based on display mode preference information; and displaying a prediction projection window and / or a prediction local window based on the prediction display mode.

[0076] For example, predicting the display mode includes predicting the display mode of the projected window and / or predicting the display mode of the local window.

[0077] Optionally, display mode preference information can be generated based on historical data of user-used display modes, or it can be generated based on historical data of user-used display modes when launching applications in the casting window and applications in the local window. For example, if a user most frequently uses the display mode of embedding a music application window in a navigation application window, then the predicted display mode can include embedding the music application window in the navigation application window. This can make the predicted display interface more personalized.

[0078] In some embodiments, the method provided in this disclosure may further include: in response to determining that the prediction operation information includes prediction operations associated with an application in the projection window but does not include prediction operations associated with an application in the local window, generating a control command that instructs to adjust the audio-visual parameters associated with the application in the local window.

[0079] For example, in response to determining that the application in the projection window needs to be operated but the application in the local window does not need to be operated, if the application in the local window is running with sound and light, a control command is generated to adjust the sound and light parameters associated with the application in the local window. The sound and light parameters are weakened by the control command, reducing the impact of the volume and light on the user, so that the user can concentrate on the application in the projection window.

[0080] For example, a music application in the local window is playing a song, and the ambient light on the local device moves in sync with the rhythm and melody of the song. A communication application in the projected window receives a message, and the message card flashes gently with a brief notification, such as "New message from Zhang." Based on this operational data and historical operation information, the predicted operation information includes viewing Zhang's message. It is determined that the predicted operation does not include operations related to the music application in the local window, and a control command is generated to lower the audio-visual parameters associated with the music application in the local window. After executing this control command, the song volume on the local device decreases and the ambient light dims. The operation interface generated based on the predicted operation information displays the message page of the communication application. After executing the operation to view Zhang's message, the predicted interface including Zhang's message is displayed.

[0081] Subsequently, if Zhang's message is a voice message, prediction can determine whether to convert it to text or play the voice message directly. If Zhang's message asks for directions, the aforementioned execution entity can also generate predicted action information, including a reply message, based on navigation data and historical operation information from the navigation application. For example, the reply message could include "I'll be there soon!" This reply message could be entered into the message card's input box and sent to Zhang. Alternatively, the user can confirm and reply using the predicted message before sending it to Zhang. This reduces manual operation for the user, improves the user experience, and will not affect driving safety in driving scenarios.

[0082] After the aforementioned executing entity sends a reply message, it can generate control commands to restore the audio-visual parameters associated with the application in the local window, such as increasing the audio-visual parameters to restore the previous music playback state.

[0083] In some embodiments, the target device for screen projection is an in-vehicle system. The method provided in this disclosure may further include: generating a recommendation message based on historical operation information and current driving information, and outputting the recommendation message.

[0084] For example, driving information may include, but is not limited to: vehicle driving status information, road condition information, and / or driver driving behavior. For instance, driving status information may include, but is not limited to, one or more of the following: vehicle speed, whether it is accelerating or decelerating, and fuel tank information. Driving behavior includes, but is not limited to: reaction speed, fatigue level, and / or the driving mode used. Recommended messages may include operation entry points or content recommendations.

[0085] Optionally, the recommendation message generated by the aforementioned execution entity may include one or more recommendation items. The highest priority recommendation item can be determined from multiple recommendation items based on time urgency and / or historical operation count and output to the user.

[0086] For example, if there is severe traffic congestion on the current road section, the historical operation information includes the user's history of listening to the radio, and a recommendation message is generated to encourage the user to listen to the radio, thereby alleviating the user's anxiety.

[0087] The method provided in this disclosure can be applied to screen mirroring scenarios or to an electronic device. For example, the screen mirroring window can be replaced with the window of a first application, and the local window can be replaced with the window of a second application. Exemplarily, the electronic device can display both the first and second application windows on the current display interface, and obtain the running data of the first and second applications. Based on the current display mode, the running data of the first and second applications, and historical operation information, predictive operation information is generated. This predictive operation information includes predictive operations associated with the application in the screen mirroring window and / or predictive operations associated with the second application. An operation interface for executing the predicted operations in the predictive operation information is generated and displayed based on the predictive operation information. This operation interface includes the operation window of the first application and / or the operation window of the second application. Alternatively, the corresponding application can be operated based on the predictive operation information to generate and display a predictive interface, and so on.

[0088] For ease of understanding, some application scenarios that can implement the methods of the embodiments of this disclosure are provided below.

[0089] For example, Mr. Li issues a navigation command, "Navigate to the nearest coffee shop." The vehicle's infotainment system detects the command and displays a navigation window, providing driving directions. While driving, Mr. Li suddenly wants to listen to a song and issues a play command, "Play song." The infotainment system detects this command, displays a split-screen playback window, and its historical operation information includes actions such as lowering the music volume during simultaneous music playback and navigation. The system can generate predicted operation information, including a predicted action to lower the volume of the currently playing song. Based on this predicted operation information, it generates and displays an interface for lowering the volume of the currently playing song. It can also lower the volume based on this predicted operation information to obtain a predicted interface after the volume is reduced. When flexible adjustments are needed, such as approaching an exit, the system intelligently recognizes and automatically enlarges the navigation window to ensure route changes are not missed. It can also enlarge the navigation window and hide the playback window based on historical operation information.

[0090] This disclosure does not limit the type of application; it can be an application with various functions, such as navigation, music, weather, news, or web pages.

[0091] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an interface display device, which corresponds to the above method embodiment and can be specifically applied to various electronic devices.

[0092] like Figure 4 As shown, the interface display device 400 of this embodiment may include: a display acquisition module 401, a first generation module 402, and a second generation module 403. The display acquisition module 401 is configured to display a projection window and a local window on the current display interface, and acquire first running data of the application in the projection window and second running data of the application in the local window. The first generation module 402 is configured to generate predicted operation information based on the display mode of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information, wherein the predicted operation information includes predicted operations associated with the application in the projection window and / or predicted operations associated with the application in the local window. The second generation module 403 is configured to generate and display an operation interface for executing the predicted operations in the predicted operation information based on the predicted operation information, wherein the operation interface includes a projection operation window and / or a local operation window.

[0093] In this embodiment, the specific processing of the display acquisition module 401, the first generation module 402, and the second generation module 403 in the interface display device 400 and the resulting technical effects can be referred to the relevant descriptions of the above method embodiments, and will not be repeated here.

[0094] In one possible implementation, the interface display device 400 further includes: a display generation module configured to, in response to determining that no actual operation information has been obtained within a first time period, operate the application in the projection operation window and / or the local operation window based on the predicted operation information, generate and display a predicted interface, wherein the predicted interface includes a predicted projection window and / or a predicted local window. A display acquisition module configured to, in response to obtaining actual operation information, obtain and display the actual interface based on the actual operation information.

[0095] In one possible implementation, the predicted operation information includes predicted operations associated with applications in the projection window and predicted operations associated with applications in the local window; the interface display device 400 further includes: a sending module configured to send information about predicted operations associated with applications in the projection window to the projection source end; a receiving module configured to receive information about the predicted projection window from the projection source end; a third generation module configured to operate the applications in the local window based on the predicted operations associated with applications in the local window to generate information about the predicted local window; and a generation and display module further configured to generate and display a predicted interface based on the predicted projection window information and the predicted local window information.

[0096] In one possible implementation, the interface display device 400 further includes: a fourth generation module configured to generate a predicted display mode based on display mode preference information, wherein the predicted display mode includes a predicted display mode for the projection window and / or a predicted display mode for the local window; and a display module configured to display the predicted projection window and / or the predicted local window based on the predicted display mode.

[0097] In one possible implementation, the interface display device 400 further includes a fifth generation module configured to generate a control command in response to determining that the predicted operation information includes predicted operations associated with an application in the projection window but does not include predicted operations associated with an application in the local window, and to instruct the adjustment of audio-visual parameters associated with the application in the local window.

[0098] In one possible implementation, the target device for screen projection is the vehicle's infotainment system, and the interface display device 400 further includes: a generation output module configured to generate and output a recommendation message based on historical operation information and current driving information.

[0099] In one possible implementation, the interface display device 400 further includes: a determining module configured to determine whether an operation to stop displaying the prediction interface is obtained during a second time period; and a storage module configured to store the prediction operation information in a historical operation information database in response to whether or not the operation is stopped.

[0100] This embodiment exists as a device embodiment corresponding to the above method embodiment. The technical effects brought about by the rescue call transfer device provided in this embodiment can be referred to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0101] According to embodiments of this disclosure, a screen mirroring system is also provided, including: a screen mirroring source end and a screen mirroring destination end. The screen mirroring source end is configured to establish a screen mirroring link with the screen mirroring destination end. The screen mirroring destination end is configured to: display a screen mirroring window and a local window on the current display interface, and acquire first running data of an application in the screen mirroring window and second running data of an application in the local window; generate predicted operation information based on the display mode of the screen mirroring window and the local window on the current display interface, the first running data, the second running data, and historical operation information, wherein the predicted operation information includes predicted operations associated with applications in the screen mirroring window and / or predicted operations associated with applications in the local window; and generate and display an operation interface for executing the predicted operations in the predicted operation information based on the predicted operation information, wherein the operation interface includes a screen mirroring operation window and / or a local operation window.

[0102] Optionally, the target device for screen projection can implement the interface display method described in any of the above embodiments.

[0103] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the interface display method described in any of the above embodiments when executed.

[0104] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the interface display method described in any of the above embodiments when executed.

[0105] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the interface display method described in any of the above embodiments.

[0106] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of locks, such as floor locks, smart locks, and other suitable locks. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0107] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0108] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506; output unit 507; storage unit 508; and communication unit 509. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0109] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the interface display method. For example, in some embodiments, the interface display method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the interface display method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the interface display method by any other suitable means (e.g., by means of firmware).

[0110] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0114] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0115] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem to address the management difficulties and weak business scalability inherent in traditional physical hosts and Virtual Private Servers (VPS) services. Servers can also be servers for distributed systems or servers integrated with blockchain technology.

[0116] According to the technical solution of this disclosure embodiment, the execution entity obtains the first running data of the application in the projection window and the second running data of the application in the local window. By using the display mode of the projection window and the local window in the current display interface, the first running data, the second running data, and historical operation information, it predicts the operation that the user wants to perform next, generates predicted operation information, and displays an operation interface for performing the predicted operation based on the predicted operation information. In this way, by predicting the operation that the user wants to perform next based on the current running status of the projection window and the local window and the user's historical operation information, and automatically displaying the operation interface, the operation interface can be displayed without the user having to perform related operations or issue commands. This can meet the user's service needs while reducing the user's effort, and greatly improve the user experience.

[0117] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0118] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for displaying an interface, applied to a target device for screen projection, comprising: Display the projection window and the local window on the current display interface, and obtain the first running data of the application in the projection window and the second running data of the application in the local window; Based on the display mode of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information, predictive operation information is generated, wherein the predictive operation information includes predictive operations associated with the application in the projection window and / or predictive operations associated with the application in the local window. An operation interface for performing the prediction operation in the prediction operation information is generated and displayed based on the prediction operation information, wherein the operation interface includes a screen projection operation window and / or a local operation window.

2. The method according to claim 1, wherein, The method further includes: In response to determining that no actual operation information is obtained within the first time period, the application in the projection operation window and / or the local operation window is operated based on the predicted operation information to generate and display a prediction interface, wherein the prediction interface includes a prediction projection window and / or a prediction local window. In response to obtaining the actual operation information, the actual interface is obtained and displayed based on the actual operation information.

3. The method according to claim 2, wherein, The prediction operation information includes prediction operations associated with applications in the projection window and prediction operations associated with applications in the local window. The method of operating the projection window and / or the application in the local operation window based on the prediction operation information to generate and display the prediction interface further includes: Send information about the predicted operation associated with the application in the projection window to the projection source. Receive information from the predicted projection window from the projection source; Based on the prediction operation associated with the application in the local window, the application in the local window is operated to generate the information of the prediction local window; Based on the information of the prediction projection window and the information of the prediction local window, the prediction interface is generated and displayed.

4. The method according to claim 3, wherein, The prediction interface is displayed, including: A predicted display mode is generated based on display mode preference information, wherein the predicted display mode includes the predicted display mode of the projection window and / or the predicted display mode of the local window; Based on the prediction display mode, the prediction projection window and / or the prediction local window are displayed.

5. The method according to any one of claims 1-4, wherein, The method further includes: In response to determining that the predicted operation information includes predicted operations associated with the application in the projection window but does not include predicted operations associated with the application in the local window, a control command is generated to instruct the adjustment of the audio-visual parameters associated with the application in the local window.

6. The method according to any one of claims 1-4, wherein, The target device for screen projection is a vehicle-mounted system, and the method further includes: A recommendation message is generated based on the historical operation information and the current driving information, and then the recommendation message is output.

7. The method according to any one of claims 1-4, wherein, The method further includes: Determine whether the operation to stop displaying the prediction interface is obtained during the second time period; If no response is received, the predicted operation information is stored in the historical operation information database.

8. An interface display device, comprising: The display acquisition module is configured to display a projection window and a local window on the current display interface, and acquire the first running data of the application in the projection window and the second running data of the application in the local window; The first generation module is configured to generate predicted operation information based on the display mode of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information, wherein the predicted operation information includes predicted operations associated with the application in the projection window and / or predicted operations associated with the application in the local window. The second generation module is configured to generate and display an operation interface for performing the prediction operation in the prediction operation information based on the prediction operation information, wherein the operation interface includes a screen projection operation window and / or a local operation window.

9. A screen projection system, comprising: The source and destination of the projection; The projection source is configured to establish a projection link with the projection destination; The target device for screen projection is configured as follows: Display the projection window and the local window on the current display interface, and obtain the first running data of the application in the projection window and the second running data of the application in the local window; Based on the display mode of the projection window and the local window on the current display interface, the first running data, the second running data, and historical operation information, predictive operation information is generated, wherein the predictive operation information includes predictive operations associated with the application in the projection window and / or predictive operations associated with the application in the local window. An operation interface for performing the prediction operation in the prediction operation information is generated and displayed based on the prediction operation information, wherein the operation interface includes a screen projection operation window and / or a local operation window.

10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the interface display method according to any one of claims 1-7.

11. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the interface display method according to any one of claims 1-7.

12. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the interface display method according to any one of claims 1-7.

Images