Response method of projection end and display end, electronic device and storage medium

Abstract

The embodiments of the present application provide a response method, electronic device, and storage medium for a projection terminal and a display terminal. In some embodiments, the response method of the projection terminal includes: obtaining user operation information; in response to the user operation information indicating that the application module is to be projected, obtaining the projection data of the application module through a first projection method; or, in response to the user operation information indicating that the third-party application is to be projected, obtaining the projection data of the third-party application through a second projection method; and transmitting the projection data to the display terminal to display the projection data through at least one display screen of the display terminal. The embodiments of the present application can realize the projection of the projection terminal and the display terminal.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and more specifically, to a response method for a projection terminal and a display terminal, an electronic device, and a storage medium. Background Art

[0002] With the development of automotive technology, cars are becoming increasingly equipped with more functions, such as navigation and video playback. This increase in car functionality places increasing demands on the processing and storage capabilities of the car's central control system, driving up vehicle costs. Summary of the Invention

[0003] The embodiments of the present application provide a response method for a projection terminal and a display terminal, an electronic device, and a storage medium that can at least partially solve the above-mentioned problems existing in the prior art.

[0004] In the first aspect, an embodiment of the present application provides a response method of a projection end, wherein the projection end is installed with a projection application and a third-party application, and multiple application modules are set in the projection application; the method includes: obtaining user operation information; in response to the user operation information indicating the projection of the application module, obtaining the projection data of the application module through a first projection method; or, in response to the user operation information indicating the projection of the third-party application, obtaining the projection data of the third-party application through a second projection method; and transmitting the projection data to the display end so as to display the projection data through at least one display screen of the display end.

[0005] In the second aspect, an embodiment of the present application provides a response method of a display end, including: displaying the identification information of an application module in a projection application of a projection end interconnected with the display end, and the identification information of a third-party application with projection permission on the projection end through at least one display screen of the display end; obtaining and transmitting user operation information of at least one display screen of the display end to the projection end; and obtaining projection data determined by the projection end, and displaying the projection data through at least one display screen of the display end.

[0006] In a third aspect, an embodiment of the present application provides an electronic device comprising: at least one processor; a communication unit communicatively connected to the at least one processor, for performing data transmission under the control of the processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the response method of the projection end as mentioned in the above embodiment, where the electronic device is the projection end; or, execute the response method of the display end as mentioned in the above embodiment, where the electronic device is the display end.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the response method mentioned in the above embodiment is implemented.

[0008] According to one embodiment of the present application, the projection end can project application modules and third-party applications in the projection application to the display end through different projection methods, so that the user can obtain the identification information of the application modules and third-party applications on the projection end through the display end. In addition, the user can operate the application modules in the projection application and the third-party applications on the projection end through the display end, thereby allowing the display end to have the functions of the application modules and third-party applications without installing these application modules and third-party applications, which can reduce the processing power requirements of the display end, thereby reducing the hardware cost of the display end. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings, in which:

[0010] Figure 1 2 is a schematic diagram of a screen projection system composed of a screen projection end and a display end according to an embodiment of the present application;

[0011] Figure 2 This is a block diagram of a screen projection terminal provided by an embodiment of the present application;

[0012] Figure 3 This is a block diagram of a display terminal using a vehicle as an example provided by an embodiment of the present application;

[0013] Figure 4 is a schematic diagram of a display interface of at least one display screen of a display terminal provided in an embodiment of the present application;

[0014] Figure 5 is a schematic diagram of a display interface of at least one display screen of a display terminal provided in an embodiment of the present application;

[0015] Figure 6 This is a flowchart of a response method of a projection terminal provided according to an embodiment of the present application;

[0016] Figure 7 is a flow chart of a response method of a display terminal provided according to an embodiment of the present application;

[0017] Figure 8 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] For a better understanding of the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are merely descriptions of exemplary embodiments of the present application and are not intended to limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0019] It should be noted that in this specification, the terms "first," "second," "third," etc., are used solely to distinguish one feature from another and do not limit the features, and in particular do not indicate any order of precedence. For example, the first projection terminal discussed in this application could also be referred to as the second projection terminal, and vice versa, without departing from the teachings of this application.

[0020] It should also be understood that expressions such as "comprises," "including," "having," "includes," and / or "comprising" are open rather than closed expressions in this specification, indicating the presence of the stated features, elements, and / or components, but do not exclude the presence of one or more other features, elements, components, and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application." And, the term "exemplary" is intended to refer to an example or illustration.

[0021] Unless otherwise defined, all words used herein (including engineering terms and scientific and technological terms) have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that, unless otherwise specified in this application, words defined in commonly used dictionaries should be interpreted as having the same meaning as they do in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0022] It should be noted that, unless otherwise specified or inconsistent with the context, the embodiments and features of the embodiments in this application may be combined with each other. Furthermore, unless expressly limited or inconsistent with the context, the specific steps included in the methods described in this application are not necessarily limited to the order in which they are described, but may be performed in any order or in parallel. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0023] In addition, when the term “connected” is used in the present application, it may indicate that the corresponding components are in direct contact or indirect contact, unless otherwise clearly defined or inferred from the context.

[0024] An embodiment of the present application provides a screen projection system, which includes a screen projection end and a display end. The screen projection end and the display end are communicatively connected to present data from the screen projection end on the display end.

[0025] As an example, the projection terminal 11 may be a mobile terminal, and the display terminal 12 may be a vehicle equipped with a display screen. The mobile terminal may be a smartphone (e.g., a mobile phone), a personal computer, a tablet computer, a laptop computer, a wearable device (e.g., a smart bracelet), an ultrabook, etc., and this application does not limit this. The vehicle may be a car, a tram, a train, a ship, an airplane, a helicopter, an amusement park vehicle, construction equipment, etc., and this embodiment of the application does not specifically limit this.

[0026] The following uses a mobile terminal as the projection end and a vehicle as the display end as an example to illustrate the projection system, the response method of the projection end, and the response method of the display end provided by the embodiments of this application. It should be understood that the projection end and the display end may also be other electronic devices without departing from the teachings of this application, and the method for implementing the projection can refer to the relevant description of this application.

[0027] Figure 1 is a schematic diagram of a screen projection system composed of a screen projection end and a display end according to an embodiment of the present application. Figure 2 It is a block diagram of the projection terminal 11 provided in the implementation manner of the present application.

[0028] like Figure 1 As shown, the projection end and the display end are connected in communication. Figure 2 As shown, the screen projection terminal 11 includes a body 111 and a display screen 112. A processor (not shown) and a memory (not shown) connected to the display screen 112 are installed in the body 111. The memory stores instructions related to the response method of the screen projection terminal 11 mentioned in the embodiment of the present application, so that the processor executes the above instructions to implement the response method of the screen projection terminal 11 mentioned in the embodiment of the present application. The screen projection terminal 11 is installed with a screen projection application 1121 and other third-party applications 1122. The icons of the screen projection application 1121 and other third-party applications 1122 can be displayed on the display screen 112 for user use. Among them, the screen projection application 1121 can be used to transmit the projection data of the screen projection terminal 11 to the display terminal 12.

[0029] As an example, multiple application modules (not shown) are set in the screen projection application. Each application module can be an independently running mini-program, and multiple application modules can also interact with each other. This application does not impose any restrictions on this.

[0030] It should be understood that Figure 2This is just an example. The number of third-party applications 1122 installed on the projection terminal 11, the number of application modules of the projection application 1121, and the arrangement of the application modules and third-party applications 1122 can be adjusted as needed.

[0031] For example, the screen projection application 1121 can be used to create a virtual screen. Relative to the actual display screen 112 of the projection terminal 11, the virtual screen can also be called a virtual display screen. This virtual screen can be created through software programming and is used to carry the content of the operating interface of the application module during operation. Displaying the application module's operating interface on the virtual screen is equivalent to display encoding of the application module in the background.

[0032] For example, each application module in the screen projection application 1121 may have some functions applicable to the display terminal. For example, if the display terminal 12 is a vehicle, the application modules may include: a navigation module, a video module, etc.

[0033] For example, third-party applications may include chat software, entertainment software, learning software, etc., which is not limited in this application.

[0034] Figure 3 FIG. 1 is a block diagram of a display terminal 12 provided in an embodiment of the present application, taking a vehicle as an example. One or more display screens 121 may be installed in the display terminal 12. Figure 3 As shown, the display screen 121 of the display terminal 12 may include: a central control display screen 1211 located in the main driver's field of view, a co-pilot display screen 1212 located in the co-pilot's field of view, a main driver's rear pillow screen 1213 installed on the back side of the main driver's seat back, and a co-pilot's rear pillow screen 1214 installed on the back side of the co-pilot's seat back. The above-mentioned display screen 121 can be formed by coating a material with a display function (such as a flexible display screen), or it can be a liquid crystal display screen, an LED display screen, etc., and this application is not limited to this. A projected screen application (not shown) can be installed on the display terminal 12. The projected screen application can control the display terminal 12 to establish a communication connection with the projection terminal 11 if authorized by the user, and has an application program compatible with the projection application 1121. The projected screen application having an application program compatible with the projection application 1121 can refer to the projected screen application having any of the following functions:

[0035] 1. The projected application can process and display the projected screen data transmitted to the projection application 1121 according to the preset decryption, decompression and other data processing methods corresponding to the encryption, compression and other processing methods of the projection data in the projection application 1121.

[0036] 2. The projected application can obtain the identification information of the application module installed in the projection application 1121, and if authorized, can obtain the identification information of the third-party application 1122 installed in the projection terminal 11 through the projection application 1121.

[0037] It should be understood that without departing from the teachings of this application, Figure 3 The number and position of the display screens 121 can be adjusted as needed, and this application does not impose any restrictions on this.

[0038] As an option, after the screen casting application is started, the display interface is initialized on the virtual screen created based on the central control display screen of the display end, so that it displays the main operation interface of the screen casting application. Among them, if the screen casting application is authorized, the identification information (such as name and icon) of the application module installed in the screen casting application and the identification information (such as name and icon) of the third-party application can be displayed in the main operation interface. The screen casting application transmits the drawn virtual screen to the screen casting application on the display end, so that the central control display screen of the display end displays the main operation interface of the screen casting application. For example, Figure 4 It is a schematic diagram of the display interface of at least one display screen 121 (such as a central control display screen) of the display terminal provided by the embodiment of the present application. Among them, application module 1 and application module 2 can be, for example, a navigation module and a video module, and chat software and game software are third-party applications. In order to facilitate users to understand and distinguish between applications (i.e., application modules and third-party applications), the display position of the icons of each application can be adjusted so that the icons of the application modules are displayed in the same area of ​​the operation interface (such as the left area of ​​the operation interface), and the icons of third-party applications are displayed in the same area of ​​the operation interface (such as the right area of ​​the operation interface). It should be understood that, Figure 4 This is just an example. The types and number of third-party applications actually installed on the projection end can be set as needed.

[0039] It should be understood that, without departing from the teachings of this application, the main operation interface can also be drawn by the projected application. For example, the projected application transmits the identification information of the application module and the identification information of the third-party application to the projected application, and the projected application draws the main operation interface on the central control display based on the above information. This application does not impose any restrictions on this.

[0040] As an example, the display end displays the identification information of the application module in the projection application of the projection end interconnected with the display end through at least one display screen of the display end, as well as the identification information of the third-party application with projection permission on the projection end, to obtain and transmit the user operation information of at least one display screen of the display end to the projection end. The projection end obtains the user operation information and determines the projection data based on it. The display end obtains the projection data determined by the projection end and displays the projection data through at least one display screen of the display end. In the above manner, the user can operate the application module in the projection application of the projection end and the third-party application of the projection end through the display end, thereby enabling the display end to have the functions of the application module and the third-party application without installing these application modules and third-party applications, which can reduce the requirements on the processing power of the display end, thereby reducing the hardware cost of the display end.

[0041] It should be understood that without departing from the teachings of this application, users can also directly operate the application modules and third-party applications of the projection application on the projection end, that is, the projection end obtains user operation information through input devices such as the display screen of the projection end. This application does not limit the source of user operation information.

[0042] In some embodiments of the present application, the way in which the projection end processes user operation information may be: the projection end first determines the application operated by the user based on the user operation information. If the user operation information indicates that the application operated by the user is a projection application, the projection application determines the application module operated by the user based on the display screen (the display screen of the projection end or the display screen of the display end) that obtains the user operation information, and transmits the user operation information to the determined application module, which processes the user operation information in combination with its own processing logic. If the user operation information indicates that the application operated by the user is a third-party application, the projection end may transmit the user operation information to the third-party application, and the third-party application will process the user operation information in combination with its own processing logic.

[0043] In some embodiments of the present application, the function programs of the screen projection application and the third-party application can be installed on the screen projection terminal or on the cloud. The screen projection terminal stores the address of the cloud that can realize the functions of the screen projection application and the third-party application, and can establish a connection with the cloud server through the address.

[0044] As an example, the program for implementing the functions of each application module can be installed on the projection end. For example, the data of the projection application includes the program for implementing the functions of each application module. The program for implementing the functions of each application module can also be installed on the cloud (i.e., its remote server or server group). For example, the data of the projection application includes the address of the cloud that can implement the functions of each application module. When the projection application needs to call the functions of any application module, it can request related services from the cloud based on the address information of the cloud.

[0045] For example, the projection end obtains user operation information; determines the application module triggered by the user based on the user operation information; and calls the application module's server to respond to the user operation information through the communication interface of the determined application module. The projection end receives feedback data from the server and determines the interface information of the application module based on the server feedback data.

[0046] In some embodiments of the present application, Figure 4 On the main operation interface shown, after the projected application detects that the user has triggered application module 1, it transmits the user operation information indicating the triggering of application module 1 to the projected application on the projecting end. The projected application on the projecting end starts application module 1 and draws the operation interface of application module 1 on a virtual screen created based on the central control display. The projected end transmits the screen interface of this virtual screen to the display end, which then displays the operation interface of application module 1 and provides the user with services related to application module 1.

[0047] As an option, Figure 5 Schematic diagram of the display interface of at least one display screen of the display terminal provided by the embodiment of the present application. Figure 5 As shown, at least one display screen 121 displays projection data through the first display area 121-1, and displays identification information of unactivated application modules and / or identification information of third-party applications through the second display area 121-2, so that users can start projection of new applications based on projection of a certain application.

[0048] As an example, the way in which the screen projection application draws the operating interface of the application module on the virtual screen includes: drawing the first area of ​​the virtual screen according to the operating interface of the application module, and drawing the second area of ​​the virtual screen according to the identification information of the application module and / or the identification information of the third-party application. After the screen projection application transmits the projection data determined according to the screen interface of the virtual screen to the projected application, the projected application presents the projection data on at least one display screen 121 as shown in FIG. Figure 5 The interface shown.

[0049] As another example, the screen projection application draws a virtual screen according to the operation interface of the application module, and transmits the projection data determined according to the screen interface of the virtual screen to the screen projection application. The screen projection application displays the projection data in the first display area 121-1 of at least one display screen 121, and displays the icon of the application module and / or the icon of the third-party application in the second display area 121-2 according to the identification information of the application module and / or the identification information of the third-party application, thereby presenting Figure 5 The interface shown.

[0050] The following is an example explanation of the startup conditions of the screen projection application on the projection end.

[0051] In some embodiments of the present application, after the projection end detects the projection instruction, it starts the projection application, and the triggering methods of the projection instruction include but are not limited to method one, method two and method three.

[0052] Method 1

[0053] In some embodiments of the present application, a user may start a screen casting application on the screen casting terminal. In response to the screen detection data of the screen casting terminal indicating that the triggered application icon is the icon of the screen casting application, the screen casting terminal triggers a screen casting instruction to start the screen casting application.

[0054] Method 2

[0055] In some embodiments of the present application, a screen projection instruction is automatically triggered when the projection terminal is close to the display terminal. In response to detecting a screen projection request, the projection terminal triggers the screen projection instruction, where the triggering condition for the screen projection request includes at least the projection terminal being close to the display terminal. When the projection terminal is close to the display terminal, the projection application is automatically launched and the projection operation is performed, simplifying the process for users to trigger the projection operation, reducing the complexity of the projection operation, and improving the user experience.

[0056] In some embodiments of the present application, a screen projection request may be sent by the display end to the projection end after determining that the projection end is close to the display end. For example, the display end detects the distance between the projection end and the display end and determines whether the projection end is close to the display end based on this distance. In response to determining that the projection end is close to the display end, the display end sends a screen projection request to the projection end.

[0057] As an example, a distance detection module may be installed on the display terminal to sense the distance between the projection terminal and the display terminal. The display terminal determines that the projection terminal is close to the display terminal in response to the distance between the projection terminal and the display terminal being less than a predetermined distance threshold, or the distance between the projection terminal and the display terminal indicating that the projection terminal is moving closer to the display terminal.

[0058] For example, the distance detection module can be an ultra-wideband (UWB) module. Specifically, one or more UWB modules are installed on the display end and the projection end respectively (hereinafter, the UWB module of the display end is referred to as a UWB anchor point, and the UWB module of the projection end is referred to as a UWB tag). The UWB anchor point can obtain the distance between it and the UWB tag when the UWB tag enters its sensing range. The display end can determine whether the projection end is close to the display end based on the distance value sensed by the UWB anchor point. Due to the higher precision of the UWB module, the distance value it measures is more accurate, which improves the accuracy of the response method.

[0059] It should be understood that without departing from the teachings of this application, the display end can also determine the distance between the projection end and the display end by, for example, analyzing the distance between the user and the vehicle at the projection end in the image captured by the image capture device, and this application does not impose any restrictions on this.

[0060] It should be understood that, without departing from the teachings of this application, the projection end can also measure the distance between the projection end and the display end, and when it is determined that the distance between the projection end and the display end indicates that the projection end is close to the display end, a projection request is triggered and automatic projection is started. This application does not impose any restrictions on this.

[0061] Method 3

[0062] In some embodiments of the present application, when the projection end is close to the display end and its own state meets the requirements, the projection instruction is automatically triggered. After detecting the projection request, the projection end obtains the status information of the projection end, and the triggering condition of the projection request at least includes that the projection end is close to the display end. The projection end determines whether to trigger the projection instruction based on whether the status information of the projection end meets the projection requirements. In response to detecting the projection instruction, the projection end transmits the projection data to the display end. It is worth mentioning that after receiving the projection request, the projection end detects the status of the projection end, and after determining that the status of the projection end meets the projection requirements, the projection application is started, which can reduce the situation where the projection data has no projection meaning due to the status of the projection end, and the projection operation affects the normal use of the projection end.

[0063] It should be noted that the triggering method of the screen projection request in method 3 can refer to the relevant description of method 2, which will not be repeated here. The following is an exemplary description of the method for determining whether the screen projection terminal meets the screen projection conditions.

[0064] In some embodiments of the present application, the status information of the projection terminal may include at least one of battery status information of the projection terminal and usage status information of the projection terminal. The usage status information includes at least one of the following: screen status information of the projection terminal, audio channel status information of the projection terminal, screen image information of the projection terminal, and feature information of the currently running application on the projection terminal. The following is an exemplary description of each status information and its corresponding projection requirements.

[0065] 1. Example of battery status information

[0066] In some embodiments of the present application, the status information of the projection terminal includes battery status information, and the projection requirements include at least battery status requirements. Among them, the battery status requirements may include: the battery status information of the projection terminal indicates that the power of the projection terminal is greater than a predetermined power threshold; or, the battery status information of the projection terminal indicates that the battery of the projection terminal is in a charging state. After the projection terminal turns on the projection, its power consumption increases. If the power of the projection terminal is insufficient, it may affect the use of other functions of the projection terminal. Judging the battery status information of the projection terminal before projection can reduce the impact of the projection operation on other functions of the projection terminal.

[0067] For example, the power threshold can be a fixed value, such as any value between 50% and 90%, or a dynamic value. For example, the projection end dynamically adjusts the power threshold based on the power consumption of the projection application. If the projection application consumes a lot of power, the power threshold is high; if the projection application consumes a little power, the power threshold is low. This allows the projection end to flexibly decide whether to trigger the projection operation based on the power consumption of the projection application.

[0068] It should be understood that those skilled in the art may adjust the battery status requirement setting based on other purposes, and this application does not limit this.

[0069] 2. Example of screen status information

[0070] In some embodiments of the present application, the status information of the screen projection end includes screen status information, and the screen projection requirements include at least screen status requirements. The screen status requirements may include: the screen status information indicates that the screen projection end is on. In other words, the screen projection end does not trigger automatic screen projection when it is in the screen-off state. In some cases, even if the screen projection end turns on the screen projection, the screen projection data it obtains has little or no viewing significance for the user, that is, the screen projection data has no screen projection significance. For example, the screen projection end uses screen recording to project the screen, but the screen projection end is not currently in the screen-off state, and the screen projection data it records may be the lock screen interface, which has little viewing significance for the user. The screen projection end detects the screen status information of the screen projection end and automatically projects the screen after determining that the screen projection end is on, thereby reducing the waste of resources caused by triggering the screen projection operation when the screen projection end does not need to project the screen.

[0071] It should be understood that those skilled in the art may adjust the screen status requirements based on other purposes, and this application does not limit this.

[0072] 3. Example of Audio Channel Status Information

[0073] In some embodiments of the present application, the status information of the screen projection end includes audio channel status information, and the screen projection requirements include at least audio status requirements.

[0074] As an example, the audio status requirement may include: audio channel status information indicates that the projected end is not in a call state. In other words, if the projected end is in a call, the projected end does not trigger the automatic screen casting instruction. When the projected end is in a call state, its screen casting data includes the operating interface of the call application, which may contain private information. The projected end detects the audio channel status information to determine whether the audio input channel and audio output channel of the projected end are simultaneously open, and judge whether the projected end is in a call state, thereby reducing privacy leaks and other problems.

[0075] As another example, the audio status requirement may also include: the audio status information indicates that the projection end has audio output, that is, the projection end can trigger automatic projection when playing audio. Generally, when using the projection end to watch videos, navigate and other functions, the projection end will turn on the audio output channel. Therefore, the projection end can detect the audio channel status information to determine whether there is an application that may need to be projected in the running applications of the projection end. If there is an application that may need to be projected on the projection end, triggering automatic projection can reduce the waste of resources caused by turning on projection when there is no projection demand.

[0076] It should be understood that the audio status requirement can be set as needed without departing from the teachings of this application, and this application does not impose any restrictions on this.

[0077] IV. Explanation of Screen Image Information

[0078] In some embodiments of the present application, the status information of the projection end includes screen image information, and the projection requirements include screen display requirements. For example, the screen display requirements include: the screen image information indicates that the application currently running on the projection end has a projection requirement. Among them, the currently running application may include the application running in the foreground of the projection end. When the projection end is close to the display end, if the application running on the projection end is an application with small changes in interface information and the user mainly obtains information through hearing, such as making calls, listening to music, etc., there is not much difference between projecting the projection end to the display end to provide services to the user through the display end and continuing to use the projection end to provide services to the user. In this case, the projection end may not automatically trigger the projection. After the user determines that projection is needed, he or she can choose to manually trigger the projection operation to reduce the power loss and computing resource loss caused by the projection operation.

[0079] In some embodiments of the present application, the projection end obtains multiple screenshot images of the display screen of the projection end according to the preset screenshot rules as the screen image information, and judges whether the screen image information meets the screen display requirements based on the similarity between the multiple screen images and the preset demand judgment rules. The following is an exemplary description of the way in which the projection end obtains the screenshot image. Among them, the similarity between the two screen images can be determined based on the change in the value of the image parameter of the pixel points corresponding to the two screen images. The image parameter of the pixel point can be, for example, the RGB value of the pixel point, etc., which is not limited here.

[0080] As an example, the screen projection end can periodically capture screenshots. For example, the screen projection end captures the first screenshot at time t1, the second screenshot at time t1+m, the third screenshot at time t1+2m, and so on, until it determines, based on the demand determination rules, whether the currently running application has a screen projection requirement and ends the screenshot capture. The predetermined screenshot interval m can be set to 1s, 0.5s, etc.

[0081] As another example, the projection end may capture screenshot images non-periodically. For example, the projection end captures M screenshot images at a preset screenshot interval, where M is greater than 1; and dynamically adjusts the screenshot interval based on the similarity between the M screenshot images.

[0082] For ease of understanding, the following example is explained using M=2. The projection end captures a first screenshot image, and after a predetermined screenshot interval, captures a second screenshot image; based on the similarity between the first screenshot image and the second projection image, and the preset interval adjustment rules, the screenshot interval is updated, and based on the updated screenshot interval, a third screenshot image is captured; the second screenshot image is used as the first screenshot image, and the third screenshot image is used as the second screenshot image, and the step of updating the screenshot interval is continued until it is determined whether the currently running application has a projection requirement, and the screenshot is ended.

[0083] It should be understood that M can also be an integer greater than 2. The projection end calculates the similarity between two consecutive screenshot images, and adjusts the screenshot interval based on the calculated multiple similarities and preset interval adjustment rules. This application does not impose any restrictions on this.

[0084] Alternatively, the interval adjustment rule indicates: if the similarity between the M screenshots is greater than a first threshold, increase the screenshot interval; if the similarity between the M screenshots is less than or equal to the first threshold, decrease the screenshot interval. The projection end dynamically adjusts the screenshot interval based on the similarity between two consecutive screenshots. When the similarity is high, the screenshot interval can be increased to reduce resource consumption caused by the screenshot operation; when the similarity is low, the screenshot interval can be shortened to increase the accuracy of the judgment result.

[0085] For example, M=2, and the demand determination rule indicates that if the similarity between two consecutive screenshots is less than the similarity threshold, it is determined that the currently running application has a screen projection requirement. The screen projection end captures the first screenshot image at time t1, and the second screenshot image at time t1+m (predetermined screenshot interval), calculates the similarity between the first screenshot image and the second screenshot image, and in response to the similarity between the first screenshot image and the second screenshot image being greater than or equal to the similarity threshold, and the similarity between the first screenshot image and the second screenshot image being greater than the first threshold, the number of judgments is updated, and the third screenshot image is captured at time t1+m+l1. The screenshot image at time t1+m is used as the first screenshot image, and the screenshot image at time t1+m+l1 is used as the second screenshot image. The similarity between the first screenshot image and the second screenshot image is calculated again. In response to the similarity between the first screenshot image and the second screenshot image being less than the similarity threshold, it is determined that the currently running application has a screen projection requirement. Wherein, l1>m, and the first threshold may be greater than the similarity threshold.

[0086] It should be understood that the method of capturing screenshot images can be adjusted as needed without departing from the teachings of this application, and this application does not impose any restrictions on this.

[0087] The above is an exemplary description of the method of capturing screenshot images. The following is an exemplary description of the demand determination rules mentioned in the implementation of the present application.

[0088] As an example, the requirement determination rule may indicate: if at least one of the similarities between any two screenshots among N screenshots is less than a similarity threshold, the currently running application is determined to have a screen projection requirement; if the similarity between any two screenshots among N screenshots is greater than or equal to the similarity threshold, the currently running application is determined not to have a screen projection requirement. N may be greater than or equal to M or less than M, and this application does not impose any restrictions on this.

[0089] For ease of understanding, the following example explanation is given with N equal to 3. In response to the similarity between the first screenshot image and the second screenshot image being less than a predetermined similarity threshold, the screen projection end determines that the currently running application has a screen projection requirement. In response to the similarity between the first screenshot image and the second screenshot image being greater than or equal to the similarity threshold, the screen projection end determines the similarity between the second screenshot image and the third screenshot image. In response to the similarity between the second screenshot image and the third screenshot image being less than the similarity threshold, the screen projection end determines that the currently running application has a screen projection requirement. In response to the similarity between the second screenshot image and the third screenshot image being greater than or equal to the similarity threshold, the screen projection end determines the similarity between the first screenshot image and the third screenshot image. In response to the similarity between the first screenshot image and the third screenshot image being less than the similarity threshold, the screen projection end determines that the currently running application has a screen projection requirement. In response to the similarity between the first screenshot image and the third screenshot image being greater than or equal to the similarity threshold, the screen projection end determines that the currently running application does not have a screen projection requirement.

[0090] As another example, the demand determination rule may indicate: obtaining the similarity of two consecutive screenshot images from multiple screenshot images, in response to at least one of the similarities of the two consecutive screenshot images being less than a similarity threshold, determining that the currently running application has a screen projection requirement, and in response to the similarities of the two consecutive screenshot images being greater than or equal to the similarity threshold, determining that the currently running application does not have a screen projection requirement.

[0091] It should be understood that, without departing from the teachings of this application, the similarity threshold can be set based on experience or based on the similarity calculation results of screenshot images of applications such as listening to music, and this application does not impose any restrictions on this.

[0092] It should be understood that the demand determination rules may be adjusted as needed without departing from the teachings of this application, and this application does not impose any restrictions on this.

[0093] 5. Example Description of Feature Information of Currently Running Applications

[0094] In some embodiments of the present application, the status information of the screen projection terminal includes characteristic information of the currently running application, and the screen projection requirement includes a screen projection application requirement. The screen projection application requirement includes: the characteristic information of the currently running application indicates that the currently running application has a screen projection requirement.

[0095] For example, the characteristic information of the currently running application includes the name of the currently running application. The projection end determines whether the currently running application requires projection based on the name of the currently running application. The projection end determines whether the currently running application requires projection based on the name of the application, so that the projection end can decide whether to project the screen based on the characteristics of the currently running application, reducing unnecessary resource consumption.

[0096] For example, a first data table is stored in the screen projection terminal, and the first data table contains pre-set names of applications with screen projection requirements. The screen projection terminal can obtain the name of the currently running application through, for example, the TopActivity tool. In response to a successful match between the name of the currently running application and the first data table (i.e., the name of the currently running application exists in the first data table), it is determined that the currently running application has a screen projection requirement. In response to a failure to match the name of the currently running application and the first data table (i.e., the name of the currently running application does not exist in the first data table), it is determined that the currently running application does not have a screen projection requirement.

[0097] As another example, the screen projection terminal stores a second data table and a third data table. The second data table contains a correspondence between application names and application types, and the third data table contains pre-set application types that require screen projection. The screen projection terminal obtains the name of the currently running application through, for example, the TopActivity tool, and determines the application type corresponding to the name of the currently running application based on the first data table. In response to a successful match between the determined application type and the third data table, the currently running application is determined to have a screen projection requirement. In response to a failed match between the determined application type and the third data table, the currently running application is determined not to have a screen projection requirement.

[0098] It should be understood that without departing from the teachings of this application, other methods can be used to determine whether the currently running application has a screen projection requirement, and this application does not impose any restrictions on this.

[0099] The above provides examples of various status information. It should be understood that, in some embodiments, the screen projection end may jointly decide whether to automatically trigger the screen projection instruction based on one status information or multiple status information. For example, in some embodiments of the present application, the status information of the screen projection end includes at least two different status information (first status information and second status information). The status information of the screen projection end includes first status information and second status information; determining whether to trigger the screen projection instruction based on whether the status information meets the screen projection requirements includes: in response to the first status information meeting the first status requirement in the screen projection requirements, determining whether to trigger the screen projection instruction based on whether the second status information meets the second status requirement in the screen projection requirements. The screen projection end determines whether to trigger automatic screen projection based on multiple status information in sequence, so that it can consider whether the screen projection end has screen projection requirements from multiple dimensions, thereby improving the accuracy of the screen projection requirement judgment result.

[0100] As an example, the first status information includes the battery status information of the projection end, and the second status information includes the usage status information of the projection end. The projection requirements may include battery status requirements, and usage status requirements corresponding to the usage status information (such as screen status requirements, audio status requirements, screen display requirements, and projection application requirements). In this example, when the projection end meets the battery status requirements, that is, when projection does not affect other functions of the projection end, it is determined whether to project the screen in combination with the usage status of the projection end, which can reduce the waste of computing resources caused by subsequent judgments when the battery of the projection end no longer meets the battery status requirements.

[0101] As another example, the first status information includes the screen status information of the projection end, and the second status information includes at least one of the audio channel status information of the projection end, the screen image information of the projection end, and the feature information of the currently running application of the projection end. The projection requirements may include screen status requirements, and at least one of audio status requirements, screen display requirements, and projection application requirements. In this example, when the screen of the projection end is on, the projection end determines whether to project the screen based on whether the application opened on the projection end has a projection requirement, which can improve the effectiveness of the projection.

[0102] As another example, the first status information includes the audio channel status information of the projection end, and the second status information includes at least one of the screen image information of the projection end and the characteristic information of the currently running application of the projection end. The projection requirements may include audio status requirements, and at least one of screen display requirements and projection application requirements. In this example, the projection end determines whether to project the screen based on whether the currently running application has projection requirements while protecting the privacy of the call, which can improve the security and effectiveness of the projection.

[0103] It should be understood that, for ease of understanding, in the implementation of the present application, an example is given in which the projection end starts the projection application to execute the response method of the projection end mentioned in the present application after detecting the projection command. In other implementations, the projection end may also automatically start the projection application after the projection end is turned on, and transmit the projection data after detecting the projection command. The present application does not impose any restrictions on the startup conditions of the projection application.

[0104] As an option, the screen projection end starts screen projection after determining that the status information meets the requirements and the user confirms the screen projection. For example, the screen projection end responds to the status information meeting the screen projection requirements by feeding back screen projection confirmation information; in response to detecting the screen projection confirmation instruction based on the feedback of the screen projection confirmation information, the screen projection instruction is triggered. In other words, when the screen projection end determines that its status information meets the screen projection requirements, it feeds back the screen projection confirmation information through a pop-up window or the like. After the screen projection end detects that the user has triggered the screen projection confirmation instruction, it triggers the screen projection instruction to start the subsequent screen projection operation. Turning on screen projection when the user confirms the screen projection can improve the security of user information while simplifying the screen projection operation.

[0105] In some embodiments of the present application, the projection end is a mobile terminal, and the display end is a vehicle. The mobile terminal triggers a projection command when it determines that its status information meets the projection requirements and the mobile terminal is not in the driving position. By not triggering the projection operation while driving a vehicle, the user can reduce the impact of the projection operation on vehicle driving and improve driving safety.

[0106] It should be understood that, without departing from the teachings of this application, the projection end may also directly trigger the projection instruction after determining that its status information meets the projection requirements, and this application does not impose any restrictions on this.

[0107] The above is an exemplary description of the startup conditions of the screen projection application on the projection end. The following is an exemplary description of the startup conditions of the screen projection application on the display end.

[0108] In some embodiments of the present application, the display end opens the projected application in response to detecting that screen detection data of the display screen used to display the projected application indicates that the application icon in the triggered area is the icon of the projected application.

[0109] Optionally, after launching the projected app, the display side sends a projection request to the projection side to trigger the projection operation. Users can directly launch the projected app on the display side and trigger the projection operation, meeting their real-time projection needs.

[0110] In other embodiments of the present application, the display terminal opens the projected screen application in response to detecting that the projection terminal is close to the display terminal. When the projection terminal is close to the display terminal, the projected screen shadow is automatically opened and the projection is started, which simplifies the projection operation and improves the user experience.

[0111] In other embodiments of the present application, the display end sends a screen projection request to the screen projection end in response to detecting that the screen projection end is close to the display end, so that the screen projection end determines whether to trigger the screen projection instruction in response to detecting the screen projection request, according to whether the status information meets the screen projection requirements, and transmits the screen projection data to the display end after triggering the screen projection instruction. In response to receiving the connection request sent by screen projection 1 (such as the first connection request or the second connection request mentioned below), the screen projection application is started, and a data link for transmitting the screen projection data is established with the screen projection end. The display end receives the screen projection data and displays the screen projection data through at least one display screen of the display end. Starting the screen projection application after the screen projection end determines that there is a screen projection demand can reduce the resource consumption caused by running the screen projection application.

[0112] It should be understood that other startup conditions may be set for the projected application without departing from the teachings of this application, and this application does not impose any restrictions on this.

[0113] The above is an exemplary description of the startup conditions of the screen projection application and the screen projection application. The following is an exemplary description of the method of establishing a data link between the screen projection end and the display end for transmitting screen projection data.

[0114] Example 1

[0115] In some embodiments of the present application, after detecting the projection instruction, the projection end selects a connection mode from the first connection mode and the second connection mode based on the network device information of the projection end and the network device information of the display end, and establishes a data link for transmitting the projection data based on the selected connection mode.

[0116] As an example, the first connection mode indicates: a first Internet network is provided by the network device of the projection end, and a data link is established between the display end and the projection end through the first Internet network. The second connection mode indicates: a second Internet network is provided by the network device of the display end, and a data link is established between the display end and the projection end through the second Internet network. In response to the network device information of the projection end and the network device information of the display end indicating that the network conditions of the projection end are better than those of the display end, the projection end selects the first connection mode; in response to the network device information of the projection end and the network device information of the display end indicating that the network conditions of the display end are better than those of the projection end, the projection end selects the second connection mode. Among them, the network device information may include indication information of whether there is a network device for establishing a data link, network speed information of the network device, network traffic upper limit information of the network device, etc.

[0117] Taking the projection end as a mobile terminal and the display end as a vehicle as an example, the network device of the projection end is the hotspot of the mobile terminal, and the network device of the display end is the Wi-Fi device of the vehicle. Before establishing a data link for transmitting projection data, the mobile terminal and the vehicle can establish a UWB connection through their respective UWB modules for data transmission. The projection end and the display end establish a Wi-Fi connection through the hotspot of the mobile terminal or the Wi-Fi device of the vehicle to transmit the projection data more quickly. If the selected connection method is the first connection method, the mobile terminal turns on the hotspot of the mobile terminal after detecting the projection instruction. The mobile terminal initiates a first connection request to the vehicle based on the improvement of the UWB connection. The first connection request includes relevant information of the hotspot (such as the hotspot name and hotspot password). After receiving the first connection request, the vehicle automatically connects to the hotspot of the mobile terminal based on the relevant information of the hotspot, and establishes a data link with the mobile connection for transmitting projection data. If the selected connection method is the second connection method, the mobile terminal sends a second connection request to the vehicle for requesting relevant information of the vehicle's Wi-Fi device after detecting the projection instruction. After receiving the second connection request, the vehicle feeds back the relevant information of the Wi-Fi device (the name and password of the Wi-Fi device) to the mobile terminal. The mobile terminal automatically connects to the WiFi device based on the relevant information of the WiFi device and establishes a data link with the vehicle for transmitting the projection data.

[0118] It is worth mentioning that the projection end can choose a connection method that is more suitable for the current scenario based on the network device information of the projection end and the display end, and establish a data link for transmitting projection data, making the data transmission between the projection end and the display end more secure.

[0119] As another example, the user can set priorities for the first connection mode and the second connection mode, and the projection end selects one connection mode from the first connection mode and the second connection mode according to the priority of the first connection mode, the priority of the second connection mode, the network device information of the projection end, and the network device information of the display end. For example, the projection end determines whether the device corresponding to the higher-priority connection mode has a network device that can provide a data link for transmitting projection data based on the network device information of the device (projection end or display end) corresponding to the higher-priority connection mode; if it is determined to be yes, the higher-priority connection mode is selected; if it is determined not to be yes, the lower-priority connection mode is selected.

[0120] For example, the projection end is a mobile terminal and the display end is a vehicle. The priority of the second connection method is higher than that of the first connection method. The mobile terminal determines whether the vehicle has a wifi device based on the network device information of the vehicle; if it is determined to be, a second connection request for requesting relevant information of the vehicle's wifi device is sent through the UWB connection, and a data link is established based on the relevant information of the wifi device; if it is determined not to be, the hotspot is turned on, and a first connection request containing the name and password of the mobile terminal's hotspot is sent to the vehicle through the UWB connection, so that the vehicle and the mobile terminal establish a data link.

[0121] It is worth mentioning that users can set their preferred connection method according to the network configuration of the projection end and the display end, making the projection process more personalized.

[0122] Example 2

[0123] In some embodiments of the present application, after detecting the screen projection instruction, the screen projection end sends a second connection request requesting relevant information of the network device to the display end through the UWB connection, and receives the feedback result of the display end for the second connection request through the UWB connection. The screen projection end responds to the feedback result of the second connection request, indicating that the display end has a network device that can provide a data link for transmitting the projection data. According to the name and password of the network device of the display end in the feedback result of the second connection request, the display end is connected to form a data link for transmitting the projection data. The screen projection end responds to the feedback result of the second connection request, indicating that the display end does not have a network device that can provide a data link for transmitting the projection data. The screen projection end turns on the hotspot and sends a first connection request including the name and password of the hotspot to the display end, so that the display end establishes a connection with the screen projection end through the name and password of the hotspot to form a data link for transmitting the projection data.

[0124] It is worth mentioning that the projection end preferentially uses the network equipment of the display end to form a data link, which can reduce the traffic cost of the projection end.

[0125] It should be understood that without departing from the teachings of this application, in addition to WiFi connection and UWB connection, the projection end and the display end can also interact with various types of data through network connections such as USB connection, Bluetooth connection, 2G / 3G / 4G / 5G, etc. This application does not limit the connection method between the projection end and the display end.

[0126] The above is an exemplary description of the connection method between the projection end and the display end. The following is an exemplary description of the method in which the projection end responds to the projection of the application module or third-party application.

[0127] Figure 6 This is a flow chart of a response method 200 of a screen projection terminal provided according to an embodiment of the present application, see Figure 6 , the response method 200 of the projection end may include the following steps:

[0128] S21, obtaining user operation information.

[0129] S22, in response to the user operation information indicating that the application module is to be projected, the projection data of the application module is obtained through the first projection method specified by the projection application; or, in response to the user operation information indicating that the third-party application is to be projected, the projection data of the third-party application is obtained through the second projection method specified by the projection end.

[0130] S23, transmitting the projection data to the display end so as to display the projection data through at least one display screen of the display end.

[0131] According to the implementation mode of the present application, the projection end can project the application modules and third-party applications in the projection application to the display end through different projection methods, so that the user can obtain the identification information of the projection end through the display end.

[0132] The following is an illustrative description of the various steps of the response method 200 of the projection terminal provided in the implementation manner of the present application.

[0133] Step S21

[0134] In some embodiments of the present application, the projection end obtains user operation information from the display end. The display end displays identification information of an application module in a projection application on the projection end that is interconnected with the display end, as well as identification information of third-party applications with projection permission on the projection end, through at least one display screen of the display end, and obtains and transmits user operation information on at least one display screen of the display end to the projection end.

[0135] It should be understood that without departing from the teachings of this application, users can also directly operate the application modules and third-party applications of the projection application on the projection end, that is, the projection end obtains user operation information through input devices such as the display screen of the projection end. This application does not limit the source of user operation information.

[0136] Step S22

[0137] In some embodiments of the present application, the first screen projection method indicates that: the screen projection application draws the screen interface of the virtual screen created by the screen projection application based on the interface information of the application module, and determines the projection data based on the screen interface of the virtual screen. The relevant content of the first screen projection method can be referred to the description of the response method of the projection terminal below and is not described in detail here.

[0138] It is worth mentioning that the operation interface of the application module is projected to the display end through the virtual screen, so that the projection end can project the screen when the screen is off, which improves the projection effect.

[0139] In some embodiments of the present application, the second projection mode can be adjusted according to the projection mode of the projection end itself.

[0140] As an example, the second screen projection method indicates: drawing the screen interface of the display screen of the projection end according to the interface information of the third-party application, and obtaining the projection data by recording the display screen of the projection end.

[0141] It is worth mentioning that the operating interface of the third-party application is displayed on the display screen of the projection end, and the projection data is obtained by screen recording, so that the third-party application can be projected.

[0142] As another example, the second screen projection method indicates: obtaining the virtual screen created by the screen projection application through the operating system of the screen projection end, and drawing the screen interface of the virtual screen according to the interface information of the third-party application; determining the screen projection data according to the screen interface of the virtual screen through the screen projection application.

[0143] It is worth mentioning that a virtual screen is created through the projection application, and the projection end draws the virtual screen based on the interface information of the third-party application, so that the projection end can project the third-party application when the screen is off, thereby improving the projection effect.

[0144] It should be understood that, without departing from the teachings of this application, the first screen projection mode and the second screen projection mode may also be other screen projection modes, and this application does not impose any restrictions on this.

[0145] In some embodiments of the present application, the projection end transmits the projection data to the display end through the projection application. In other words, whether it is the projection data of a third-party application or the projection data of an application module, it is transmitted to the display end through the projection application.

[0146] It should be understood that the transmission method of the projection data can be adjusted as needed without departing from the teachings of this application, and this application does not impose any restrictions on this.

[0147] The following introduces the response method of the projection end provided by the implementation method of this application when the projection end mainly adopts the first projection mode.

[0148] In some embodiments of the present application, the projection end draws the screen interface of the virtual screen created by the projection application based on the interface information of the application module; determines the projection data based on the screen interface of the virtual screen; and transmits the projection data to the display end so as to display the projection data through at least one display screen of the display end. Since the projection end sets part of the application to be projected in the form of an application module in the projection application, the projection application has the authority to obtain the relevant data of the application module, and there is no need to request permission from the application to be projected, thereby reducing the probability of projection failure due to lack of permission to obtain data of the application to be projected. In addition, the projection end projects the virtual screen drawn based on the interface information of the application module to the display end, and the display end can display the response results of these application modules to the user's operations, so that the display end can provide users with the services of the application module without installing the application module, reducing the requirements for the processing power of the display end, thereby reducing the hardware cost of the display end.

[0149] The following is an exemplary description of how the projection end draws the screen interface of the virtual screen created by the projection application based on the interface information of the application module.

[0150] In some embodiments of the present application, the method for creating a virtual screen by the projection application on the projection end may include: obtaining the size information of the display screen of the display end; and creating a virtual screen by calling the virtual screen creation interface of the projection end according to the size information of the display screen of the display end. The size of the created virtual screen is adapted to the size of the display screen of the display end, that is, the size of the virtual screen is equal to or slightly smaller than the size of the display screen of the display end. After the virtual screen is created, the method for drawing the virtual screen by the projection application may be: display encoding of the application module according to the interface information of the application module.

[0151] In some embodiments of the present application, the method for drawing the screen interface of the virtual screen on the projection end includes: the projection end determining the virtual screen corresponding to the application module; and drawing the screen interface of the virtual screen corresponding to the application module based on the interface information of the application module. Because the projection end may create multiple virtual screens, it is necessary to first determine the virtual screen corresponding to the application module before drawing the operation interface of the application module.

[0152] In some implementations of the present application, methods for determining the virtual screen corresponding to the application module include but are not limited to method one and method two.

[0153] Method 1

[0154] In some embodiments of the present application, the projection end can use a pre-designated virtual screen as the virtual screen corresponding to the application module. Because the correspondence between the virtual screen and the display screen of the display end is fixed, the operation interface of the application module will be displayed on the pre-designated display screen of the display end. Displaying the operation interface of the application module on the pre-designated display screen of the display end facilitates the user to obtain the identification information of each application module from the same display screen of the display end and switch it to another display screen as needed.

[0155] Method 2

[0156] In some embodiments of the present application, the method for the projection end to determine the virtual screen corresponding to the application module may include: determining a candidate virtual screen for the application module; in response to the candidate virtual screen having drawn interface information of other application modules other than the application module, and the other application modules and the application module satisfy a first constraint relationship, creating a new virtual screen as the virtual screen corresponding to the application module.

[0157] For ease of understanding, the following provides an exemplary description of a method for determining a candidate virtual screen.

[0158] In some implementations of the present application, methods for determining candidate virtual screens include but are not limited to method 1, method 2, and method 3.

[0159] Method 1

[0160] In some embodiments of the present application, the projection end determines the candidate virtual screen of the application module based on the application type of the application module, the correspondence between the application type and the display screen of the display end, and the correspondence between the display screen of the display end and the virtual screen. For example, after the projection end determines the application module that needs to be displayed, it first determines the application type of the application module based on the correspondence between the application module and the application type, and then determines the display screen corresponding to the application type to which the application module belongs based on the correspondence between the application type and the display screen of the display end, and then determines whether a virtual screen has been created for the determined display screen based on the correspondence between the display screen of the display end and the virtual screen. If it is determined to be yes, the virtual screen corresponding to the determined display screen is used as a candidate virtual screen. If it is determined not to be, a new virtual screen is created, and the new virtual screen is used as a candidate virtual screen, and the correspondence between the display screen of the display end and the virtual screen is updated.

[0161] It is worth mentioning that the display screen used to display its operation interface is determined according to the application type of the application module. The operation interface can be displayed on a display screen that is more suitable for displaying the application module in combination with the characteristics of each application type, thereby improving the user's viewing experience.

[0162] As an example, the correspondence between application modules and application types, the correspondence between application types and display screens of the display end, and the correspondence between the display screens of the display end and the virtual screen can be stored in a table format on the projection end.

[0163] It should be understood that without departing from the teachings of this application, the above-mentioned correspondence relationship can also be stored in other forms on the projection end, and this application does not limit the storage form of the correspondence relationship.

[0164] It should be understood that the same application type may correspond to display screens of multiple display terminals, and the display screen of the same display terminal may correspond to multiple virtual screens, and this application does not impose any restrictions on this.

[0165] It should be understood that when multiple virtual screens are determined, all of the multiple virtual screens can be used as candidate virtual screens.

[0166] Method 2

[0167] In some embodiments of the present application, the projection end determines the candidate virtual screen of the application module based on the identification information of the application module, the correspondence between the application module and the display screen of the display end, and the correspondence between the display screen of the display end and the virtual screen. For example, after the projection end determines the application module that needs to be displayed, it determines the display screen of the display end corresponding to the application module based on the correspondence between the application module and the display screen of the display end. The projection end determines whether there is a virtual screen corresponding to the determined display screen based on the correspondence between the display screen of the display end and the virtual screen. If so, the virtual screen corresponding to the determined display screen is used as a candidate virtual screen. If not, a new virtual screen is created, the new virtual screen is used as a candidate virtual screen, and the correspondence between the display screen of the display end and the virtual screen is updated.

[0168] It is worth mentioning that, in combination with the characteristics of each application module, its operation interface is displayed on a display screen that is more suitable for displaying the application module, thereby improving the user's viewing experience.

[0169] Method 3

[0170] The projection end uses the pre-specified virtual screen as a candidate virtual screen for determining application modules. For example, the projection end may use the virtual screen corresponding to the central control display as a candidate virtual screen, so that each application module is first displayed on the central control display after being turned on, making it easier for the driver to obtain the identification information of each application module and switch it to other displays as needed.

[0171] It should be understood that, without departing from the teachings of this application, the candidate display screens of the application module may also be determined in other ways, and this application does not impose any limitation thereto.

[0172] The above describes an example of a method for determining candidate virtual screens of an application module. The following describes an example of a method for determining a virtual screen of an application module based on candidate virtual screens of the application module.

[0173] In some embodiments of the present application, in response to a candidate virtual screen already having interface information drawn for application modules other than the application module, and the other application modules and the application module satisfy a first constraint, a new virtual screen is created as the virtual screen corresponding to the application module, and the display screen corresponding to the new virtual screen is the same as the display screen corresponding to the candidate virtual screen. For example, after determining the candidate virtual screens for the application module, the projection end determines whether there are any idle virtual screens among the candidate virtual screens, that is, whether there is at least one candidate virtual screen that does not have interface information drawn for any application module. If the projection end determines that there are idle virtual screens, it can arbitrarily select an idle virtual screen as the virtual screen for the application module. If the projection end determines that there are no idle virtual screens, that is, all candidate virtual screens have interface information drawn for other application modules, a candidate set is obtained based on the candidate virtual screens. For each virtual screen in the candidate set, it is determined whether the application module and the application module corresponding to the virtual screen satisfy the first constraint. If so, the virtual screen is removed from the candidate set, and the display screen of the display end continues to display the screen interface of the virtual screen. If not, the virtual screen is retained. After completing the above operations for all virtual screens in the candidate set, the projection end determines whether there are any remaining virtual screens in the candidate set. If it is determined to be, it can arbitrarily select one virtual screen from the remaining virtual screens in the candidate set as the virtual screen corresponding to the application module, or it can select the virtual screen corresponding to the application module with the lowest priority as the virtual screen of the application module based on the priority of the application modules corresponding to the remaining virtual screens. The screen interface of the selected virtual screen will no longer be displayed on the display screen of the display end. If the projection end determines that the candidate set is an empty set, it will create a new virtual screen as the virtual screen corresponding to the application module. In the above manner, on the basis of displaying multiple virtual screens on the display screen of a display end at the same time, the number of virtual screens displayed at the same time can be controlled to avoid the excessive number of virtual screens displayed at the same time affecting user viewing.

[0174] It should be noted that the first constraint relationship can be set as needed. The first constraint relationship is exemplarily described below.

[0175] As an example, the first constraint relationship includes: the application type of the other application modules and the application module are the same. In other words, if the application module and the other application modules corresponding to the candidate virtual screen are of the same application type, a new virtual screen is created for the application module. For example, if a user opens chat application module 3 while the user interface of chat application module 1 is displayed on the display screen of the display terminal, a new virtual screen is created for application module 3, and the contents of the two virtual screens are displayed in split-screen mode on the same display screen of the display terminal.

[0176] It should be understood that, without departing from the teachings of this application, when the application types of other application modules and application modules are different, the projection end can directly determine that the two do not satisfy the first constraint relationship, or set other conditions for further judgment. This application does not impose any restrictions on this.

[0177] As another example, the first constraint relationship includes: the other application module or application module belongs to a pre-specified application type. The pre-specified application type may include a navigation type and / or a call type. In other words, if one of the other application module and the application module belongs to the navigation module or the call type, the projection end determines that the other application module and the application module satisfy the first constraint relationship.

[0178] As another example, the first constraint relationship includes multiple constraint conditions. When any one constraint condition is met, the projection end determines that the other application modules and the application module meet the first constraint relationship. For example, the first constraint relationship includes constraint condition 1 and constraint condition 2. Constraint condition 1 is that the application type of the other application module and the application module is the same, and constraint condition 2 is that the other application module or the application module belongs to a pre-specified application type. If the application type of the other application module and the application module meets the following conditions, the projection end determines that the other application module and the application module meet the first constraint relationship:

[0179] Case 1: The application type in other application modules is the same as that in the application module;

[0180] Case 2: The application types of the other application module and the application module are different, but one of the other application module and the application module belongs to a pre-specified application type.

[0181] It should be understood that the first constraint relationship can be set as needed without departing from the teaching of this application, and this application does not impose any restrictions on this.

[0182] It should be understood that other methods can also be used to determine the virtual screen corresponding to the application module without departing from the teachings of this application. For example, the projection terminal provides the user with three methods, namely, method 1, method 2, and method 3, and the user can choose the method for determining the virtual screen corresponding to the application module as needed. This application does not limit the method for determining the virtual screen corresponding to the application module.

[0183] The following is an exemplary description of how the projection end determines the projection data based on the screen interface of the virtual screen.

[0184] In some embodiments of the present application, the projection end determines the projection data based on the number of screen interfaces of the virtual screen that can be presented by the display screen. For example, if the display screen of the display end can present the screen interface of a virtual screen, the projection end can use the entire screen interface of the virtual screen as the projection data. If the display screen of the display end can present the screen interfaces of multiple virtual screens, the projection end determines the projection data in the following manner: determining the display screen of the display end corresponding to the virtual screen, and obtaining the number of screens of the virtual screen corresponding to the determined display screen of the display end; if the number of screens is equal to 1, the projection end can use the entire screen interface of the virtual screen as the screen data. If the number of screens is greater than 1, the projection end determines the size information of the screen interface to be projected extracted from the screen interface of the virtual screen based on the obtained number of screens and the predefined cropping rules, and extracts the screen interface to be projected from the screen interface of the virtual screen based on the determined size information, and then determines the projection data based on the screen interface to be projected. Among them, the predefined cropping rules may indicate: the correspondence between the number of screens and the size of the screen interface to be projected of each virtual screen.

[0185] It should be understood that without departing from the teachings of this application, the cropping rules can be set as needed. As an example, in the case of multiple virtual screens being displayed in split screen, the cropping rules may indicate: the vertical size of the interface of the virtual screen to be projected is equal to or slightly smaller than the vertical size of the display screen of the display end, and the horizontal size of the interface of the virtual screen to be projected is equal to or slightly smaller than the quotient of the horizontal size of the display screen of the display end divided by the number of screens, so that multiple virtual screens are displayed in split screen horizontally on the display screen of the display end. The cropping rules may also indicate: the horizontal size of the interface of the virtual screen to be projected is equal to or slightly smaller than the horizontal size of the display screen of the display end, and the vertical size of the interface of the virtual screen to be projected is equal to or slightly smaller than the quotient of the vertical size of the display screen of the display end divided by the number of screens, so that multiple virtual screens are displayed in split screen vertically on the display screen of the display end. The cropping rules may also indicate: if the number of screens is divisible by N, the vertical size of the interface to be projected of the virtual screen is determined according to the quotient of the vertical size of the display screen of the display terminal and the number of screens and N, and the horizontal size of the interface to be projected of the virtual screen is determined according to the horizontal size of the display screen of the display terminal and N; if the number of screens is not divisible by N, the value of [number of screens / N]+1 is calculated, and the vertical size of the interface to be projected of the virtual screen is determined according to the vertical size of the display screen of the display terminal and the calculated value, and the horizontal size of the interface to be projected of the virtual screen is determined according to the horizontal size of the display screen of the display terminal and N, so that multiple virtual screens can be displayed in split screen on the display screen of the display terminal. Wherein, N may be an integer greater than or equal to 2. As another example, when one of the multiple virtual screens is displayed in full screen and the other virtual screens are displayed through picture-in-picture, the cropping rules may indicate: according to the size information of the set display window of the picture-in-picture, the size information of the interface to be projected of the virtual screen is determined. This application does not impose any restrictions on the cropping rules.

[0186] For ease of understanding, the process of determining the projection data is explained by taking the example of creating virtual screen 1, virtual screen 2, and virtual screen 3 on the projection side, virtual screen 1 and virtual screen 2 corresponding to the central control display screen of the display side, virtual screen 3 corresponding to the co-pilot display screen of the display side, and the cropping rule indicating horizontal split-screen display. For virtual screen 1 and virtual screen 2, after the projection side determines that the number of screens corresponding to the central control display screen of the display side is greater than 1, it determines, according to the cropping rule, the size information of the screen interface to be projected extracted by virtual screen 1 and virtual screen 2 as follows: the horizontal size is equal to the horizontal size of the central control display screen / 2, and the vertical size of the screen interface to be projected extracted by virtual screen 1 and virtual screen 2 is equal to the vertical size of the central control display screen; the projection side extracts the screen interface to be projected that meets the determined size information from virtual screen 1 and virtual screen 2 respectively, wherein the center of the extracted area can be the center of the area of ​​interest determined according to the area of ​​interest determination rule of the application module corresponding to virtual screen 1 and virtual screen 2. For virtual screen 3, after determining that the number of screens corresponding to the co-pilot display screen of the display side is equal to 1, the projection side extracts the entire screen interface of virtual screen 3 as the screen interface to be projected. The projection end extracts the screen interface to be projected from virtual screen 1, virtual screen 2 and virtual screen 3 as the screen data in the projection data.

[0187] It should be understood that, without departing from the teachings of this application, the rules for determining the regions of interest of different application modules can be set according to the functions of each application module. For example, if the function of the application module is navigation (i.e., navigation module), its rules for determining the regions of interest can indicate that the region of interest in the map is a circular area with the user's current location as the center and a predetermined distance value as the radius; if the function of the application module is video playback (i.e., video module), its rules for determining the regions of interest can indicate that the region of interest is a rectangular area with the geometric center of the video playback interface or the center of the area where any face is located as the center and a predetermined size as the side length. This application does not impose any restrictions on the rules for determining the regions of interest.

[0188] It is worth mentioning that when supporting one display screen corresponding to multiple virtual screens, the screen interface to be projected is obtained by cropping rather than adjusting the display ratio of the display interface of the application module. This makes the obtained screen interface to be projected clearer, and the picture ratio changes less than when one display screen corresponds to one virtual screen, which has less visual impact on the user and improves the user's viewing experience.

[0189] It should be understood that without departing from the teachings of this application, the projection data may be determined by other means, and this application does not impose any restrictions on this.

[0190] Step S23

[0191] In some embodiments of the present application, the projection end may adjust the data transmission method of the projection data according to the number of display screens of the display end.

[0192] As an example, if the display terminal has one display screen, the method for the projection terminal to determine the projection data based on the screen interface of the virtual screen may include: determining the screen data based on the screen interface of the virtual screen, and using the screen data as the projection data. If the display terminal has two display screens, the method for the projection terminal to determine the projection data based on the screen interface of the virtual screen may include: determining the screen data based on the screen interface of the virtual screen, and using the screen data and the correspondence between the virtual screen and the display screen of the display terminal as the projection data, so that the display terminal determines the display screen used to display the screen interface of each virtual screen, thereby realizing multi-channel projection.

[0193] As another example, the display end and the projection end can establish multiple data links (i.e., interconnection channels) for transmitting projection data. The projection end determines the projection data based on the screen interface of the virtual screen, and determines the data link corresponding to each virtual screen based on the correspondence between the virtual screen and the display screen of the display end, and the correspondence between the display screen of the display end and the data link. The projection end transmits the projection data determined based on each virtual screen to the display end through the data link corresponding to the virtual screen. The display end displays the projection data received by each data link through the display screen corresponding to each data link, thereby realizing multi-channel projection.

[0194] In some embodiments of the present application, the projection end may obtain and transmit the projection data to the display end after determining to project the screen, or may set prerequisites for the transmission of the projection data, and then obtain and transmit the projection data to the display end when the prerequisites are met.

[0195] For example, the projection end is a mobile terminal, and the display end is a vehicle. The prerequisite is that the mobile terminal enters the vehicle. After entering the vehicle, the mobile terminal obtains and transmits the projection data to the vehicle, reducing network resource waste caused by the mobile terminal transmitting the projection data when the user cannot view the projection data.

[0196] It should be understood that other prerequisites may be set without departing from the teachings of this application, and this application does not limit this.

[0197] The above example illustrates how the projection end responds when the user operation information instructs the application to be projected. The following example illustrates how the projection end responds when the user operation information instructs the screen to be switched.

[0198] In some embodiments of the present application, the projection end switches screens in response to user operation information, changing the correspondence between the screen interface to be switched indicated by the user operation information and the display screen of the display end to update the projection data. The projection end changes the correspondence between the screen interface and the display screen of the display end based on the user operation information, thereby realizing screen switching operations in the case of multiple screens on the display end, allowing the user to select the screen for displaying the operation interface of each application module or the operation interface of a third-party application as needed, thereby improving the user experience.

[0199] In some embodiments of the present application, the screen interfaces that need to be switched include the screen interface of the display screen of the projection end and the screen interface of the virtual screen. The projection end performs screen switching in response to the user operation information instruction, and changes the correspondence between the screen interface of the display screen of the projection end and the screen interface of the virtual screen and the display screen of the display end (referred to as the first correspondence). Among them, the way to change the first correspondence can be: the projection end carries the identification information of the display screen of the display end in the projection data corresponding to the virtual screen according to the changed first correspondence, and the identification information of the display screen of the display end carried in the projection data corresponding to the display screen of the projection end.

[0200] In some embodiments of the present application, the screen interfaces to be switched are all virtual screen interfaces. The projection end switches screens in response to user operation information, changing the correspondence between the screen interface to be switched indicated by the user operation information and the display screen of the display end. Methods for changing the correspondence between the virtual screen interface and the display screen of the display end include, but are not limited to, Method 1 and Method 2.

[0201] Method 1

[0202] In some embodiments of the present application, the application module in the screen projection application may include a virtual screen management module. The virtual screen management module is used to create a virtual screen, manage the correspondence between other currently running application modules and the virtual screen (hereinafter referred to as the second correspondence), manage the correspondence between the virtual screen and the display screen of the display terminal (hereinafter referred to as the third correspondence), draw the operation interface of each application module to the corresponding virtual screen, etc. The correspondence between the application module that needs to switch screens and the display screen of the display terminal is changed through the virtual screen management module.

[0203] As an example, the process of the virtual screen management module changing the correspondence between the screen interface to be switched and the display screen of the display terminal may include: adjusting the second correspondence based on the user operation information to change the correspondence between the screen interface to be switched and the display screen of the display terminal. Due to the adjustment of the second correspondence, the screen interface of each virtual screen will change, and the correspondence between the screen interface and the display screen of the display terminal will also be adjusted.

[0204] As another example, the process of the virtual screen management module changing the correspondence between the screen interface to be switched and the display screen of the display terminal may include: adjusting the third correspondence based on the user operation information to change the correspondence between the screen interface to be switched and the display screen of the display terminal. Since the third correspondence is adjusted, the identification information of the display screen of the display terminal carried in the projection data determined based on the virtual screen is adjusted, and the correspondence between the screen interface and the display screen of the display terminal is also adjusted.

[0205] For ease of understanding, let's use the second correspondence relationship indicating that application module 1 corresponds to virtual screen 3 and application module 2 corresponds to virtual screen 4, and the third correspondence relationship indicating that virtual screen 1 corresponds to the central control display, virtual screen 2 corresponds to the passenger display, virtual screen 3 corresponds to the driver's rear screen, and virtual screen 4 corresponds to the passenger's rear screen as an example. Before switching, the driver's rear screen on the display side displays the user interface of application module 1, and the passenger's rear screen on the display side displays the user interface of application module 2. The user performs a switching operation on the display side, such as clicking the switch button or swiping right with two fingers on the driver's rear screen. After detecting the user operation, the display side transmits the operation information to the projection side, which transmits it to the projection application. The projection application determines that the user operation information indicates a switch between the driver's rear screen and the passenger's rear screen and transmits it to the virtual screen management module. The virtual screen management module adjusts the first correspondence relationship so that application module 1 corresponds to virtual screen 4 and application module 2 corresponds to virtual screen 3. Alternatively, the virtual screen management module adjusts the second correspondence, so that virtual screen 3 corresponds to the front passenger rear screen, and virtual screen 4 corresponds to the front passenger rear screen. This operation enables screen switching between the front passenger and rear passenger screens. Similarly, switching between other screens can be described in the relevant description of this application.

[0206] Method 2

[0207] In some embodiments of the present application, the projection end draws a virtual screen through various application modules, and adjusts the correspondence between the screen interface to be switched and the display screen of the display end through the application module corresponding to the screen interface to be switched. For example, the screen interface to be switched is the screen interface of a first virtual screen, which corresponds to a first application module. The first application module determines the identification information of the virtual screen corresponding to the switched display screen; and updates the identification information of the virtual screen used to draw the operating interface of the application module to the identification information of the determined virtual screen.

[0208] For example, virtual screen 3 corresponds to the main driver's rear screen, virtual screen 4 corresponds to the co-driver's rear screen, application module 1 corresponds to virtual screen 3, and application module 2 corresponds to virtual screen 4. Before switching, the main driver's rear screen on the display end displays the operation interface of application module 1, and the co-driver's rear screen on the display end displays the operation interface of application module 2. After the screen projection application calculates the user operation information based on the user operation information and instructs the user operation information to switch the screen interface of virtual screen 3 and the screen interface of virtual screen 4, it can transmit the identification information of virtual screen 4 to application module 1, and transmit the identification information of virtual screen 3 to application module 2. Application module 1 draws its own operation interface to virtual screen 4, and application module 2 draws its own operation interface to virtual screen 3. Through the above operations, the screen switching between the main driver's rear screen and the co-driver's rear screen can be realized. Similarly, the switching between other screens can also refer to the relevant description of this application.

[0209] It should be understood that screen switching can be achieved in other ways without departing from the teachings of this application, and this application does not limit this.

[0210] After completing the exemplary description of the response method of the projection end, the following exemplary description of the response method of the display end is given.

[0211] Figure 7 This is a flow chart of a response method 300 of a display terminal provided according to an embodiment of the present application, see Figure 7 , the response method 300 of the display terminal may include the following steps:

[0212] S31, displaying, through at least one display screen of the display end, identification information of an application module in a screen projection application of a screen projection end interconnected with the display end, and identification information of a third-party application with screen projection permission on the screen projection end.

[0213] S32, obtaining and transmitting user operation information of at least one display screen of the display end to the projection end.

[0214] S33, obtaining the projection data determined by the projection end through the first projection method or the second projection method, and displaying the projection data through at least one display screen of the display end.

[0215] According to the implementation mode of the present application, the user can operate the application modules in the projection application of the projection end and the third-party applications of the projection end through the display end, so that the display end has the functions of the application modules and third-party applications without installing these application modules and third-party applications, which can reduce the requirements on the processing power of the display end, thereby reducing the hardware cost of the display end.

[0216] The following is an exemplary description of each step of the display terminal response method 300 provided in the embodiment of the present application.

[0217] Step S31

[0218] In some embodiments of the present application, the display terminal can be, for example, Figure 4 The following method displays the identification information of the application module and the identification information of the third-party application. For specific display formats, please refer to Figure 4 The relevant instructions will not be repeated here.

[0219] Step S32

[0220] In some embodiments of the present application, the display terminal obtains and transmits user operation information to the projection terminal, so that the projection terminal responds to the user operation, allowing the user to control the projection terminal through the display screen of the display terminal. The manner in which the projection terminal responds to the user operation information can be referred to the relevant description of the projection terminal response method 200, which will not be repeated here.

[0221] For example, if all of the multiple display screens on the display terminal are touch screens, the display terminal may obtain user operation information by: obtaining detection data transmitted by at least one display screen on the display terminal, indicating the display area where the user operated; and determining the user operation information based on the identification information of the at least one display screen on the display terminal and the detection data. In other words, the user operation information determined by the display terminal carries the identification information of the display screen, allowing the projection terminal to determine the display screen actually operated by the user, and then determine the application module actually operated by the user based on the correspondence between the display screen on the display terminal and the currently running application module.

[0222] Step S33

[0223] In some embodiments of the present application, the display end receives projection data transmitted by the projection end interconnected with the display end. The projection data is determined according to the screen interface of the virtual screen created by the projection application of the projection end, or is obtained by recording the screen of the projection display of the projection end. The screen interface of the virtual screen is drawn according to the interface information of the application module in the projection application. The projection data is displayed by at least one display screen of the display end. In the above manner, the application to be projected is set in the projection application in the form of an application module, so that the projection application has the authority to obtain the relevant data of the application module, and there is no need to request permission from the application to be projected, thereby reducing the probability of projection failure due to lack of permission to obtain data of the application to be projected. In addition, the projection end projects the virtual screen drawn based on the interface information of the application module to the display end, and the display end can display the response results of these application modules to the user's operation, so that the display end can provide users with the service of the application module without installing the application module, thereby reducing the processing power and other requirements of the display end, thereby reducing the hardware cost of the display end.

[0224] In some embodiments of the present application, before the projection end and the display end start projection, the display end may initiate a projection request. For example, after the display end detects that the projection end is close to the display end, it sends a projection request to the projection end. After receiving the projection request, the projection end may trigger the projection instruction, or determine whether to trigger the projection instruction based on whether its own status information meets the projection request. After triggering the projection instruction, the projection end transmits the projection data to the display end. Among them, the process of the projection end determining whether to trigger the projection instruction can refer to the relevant description of the response method 200 of the projection end, which will not be repeated here.

[0225] In some embodiments of the present application, after the projection end triggers the projection instruction, it establishes an interconnection channel (i.e., the data link mentioned above for transmitting projection data) with the display end, and transmits the projection data to the display end through the interconnection channel.

[0226] As an example, the projection end includes a first projection end and a second projection end. In response to receiving an interconnection request from the first projection end, and the display end and the second projection end have established an interconnection channel, the display end determines whether to allow the first projection end and the display end to establish an interconnection channel for transmitting the projection data of the first projection end according to the preset multi-end interconnection rules. In other words, after the display end establishes an interconnection channel with the second projection end, if it receives an interconnection request from the first projection end, it determines whether to allow the first projection end to access according to the multi-end interconnection rules of the display end. If the judgment result indicates that the first projection end is allowed to access, it can directly establish an interconnection channel with the first projection end, or it can establish an interconnection channel with the first projection end after determining that the permission information of the first projection end is qualified. If the judgment result indicates that the first projection end is not allowed to access, no interconnection channel is established with the first projection end. In compliance with the multi-end interconnection rules, the display end allows multiple projection ends to access, thereby improving the utilization rate of the display end.

[0227] It should be understood that, without departing from the teachings of this application, the interconnection request may be the connection request mentioned above (the first connection request or the second connection request). For example, after receiving the connection request, the display end determines whether to allow the display end and the projection end to establish an interconnection channel according to the multi-terminal interconnection rules. If the judgment result is allowed, the interconnection channel is established through the first Internet network or the second Internet network. The interconnection request may also be a request different from the connection request mentioned above. After receiving the interconnection request, the display end determines whether to establish an interconnection channel with the projection end according to the multi-terminal interconnection rules. This application does not impose any restrictions on this.

[0228] For ease of understanding, the following is an illustrative explanation of the multi-terminal interconnection rules.

[0229] As an example, the multi-terminal interconnection rule indicates that in response to the priority of the first projection terminal being higher than the priority of the second projection terminal, the first projection terminal is allowed to access. In other words, the display terminal determines whether to allow the new projection terminal to access based on the priority of each projection terminal.

[0230] It should be understood that, without departing from the teachings of this application, if the priority of the first projection terminal is lower than or equal to the priority of the second projection terminal, the display terminal can send authentication information to the second projection terminal and determine whether to allow access to the first projection terminal based on the feedback data from the second projection terminal. The display terminal can also directly deny access to the first projection terminal. This application does not impose any restrictions on this.

[0231] It is worth mentioning that after setting the priority for each projection end, the display end can create an interconnection channel with a projection end, and then determine whether to allow the new projection end to access based on the priority of the projection end, making the projection process more intelligent.

[0232] As another example, the multi-terminal interconnection rule indicates: Send authentication information to the second projection terminal, and determine whether to allow the first projection terminal to access based on the feedback data from the second projection terminal based on the authentication information. In other words, the display terminal follows the principle that the projection terminal that connects first has the right to decide on the display terminal's use. After receiving an interconnection request from a new projection terminal, it inquires with the existing projection terminal whether to allow the new projection terminal to access.

[0233] It should be understood that, without departing from the teachings of this application, after the display end determines to allow the first projection end to access, it can choose to disconnect from the second projection end or continue to maintain the connection with the second projection end. There is no restriction here.

[0234] In some embodiments of the present application, after the display end and the projection end establish an interconnection channel, the projection data is received by the projection application on the display end. For example, data can be transmitted between the projection application on the projection end and the projection application on the display end. It should be understood that data can be transmitted directly between the projection application on the projection end and the projection application on the display end, or data can be transmitted through other devices. For example, the projection application on the projection end transmits the projection data to the cloud device of the projection service, and the cloud device then transmits the projection data to the projection application on the display end. This application does not impose any restrictions on this.

[0235] In some embodiments of the present application, after the projection end is connected to the display end, the display end selects a display screen for displaying the projection data for the projection end, and displays the projection data of the projection end through the selected display screen.

[0236] The following is an exemplary description of how the display end selects the display screen corresponding to the projection end.

[0237] Method 1

[0238] In some embodiments of the present application, a display terminal allows access to a projection terminal. The display terminal may select a display screen for displaying the projection data of the projection terminal, wherein the display terminal uses a designated display screen as the display screen corresponding to the projection terminal, or uses the display screen closest to the projection terminal as the display screen corresponding to the projection terminal; and displays the projection data of the projection terminal on the display screen corresponding to the projection terminal.

[0239] It should be understood that, without departing from the teachings of this application, the display end can be set with a designated display screen, and all projection ends correspond to the designated display screen. The display end can also set designated display screens for different projection ends. After the projection end is connected, the corresponding designated display screen is determined based on the identification information of the projection end. This application does not impose any restrictions on this.

[0240] Method 2

[0241] In some embodiments of the present application, a display terminal allows access to multiple projection terminals. The display terminal selects a display screen for displaying the projection data of the projection terminal, which may include: the display terminal determines a candidate display screen corresponding to the projection terminal, and selects a display screen for displaying the projection data from the candidate display screens based on the occupancy of the candidate display screens.

[0242] As an example, the candidate display screen corresponding to the projection end can be a designated display screen, or it can be a display screen determined based on the location information of the projection end. For example, based on the proximity principle, the display screen of the display end closest to each projection end can be used as the candidate display screen corresponding to the projection end.

[0243] For example, the projection end is a mobile terminal, the display end is a vehicle, and the location information of the projection end is the seating position of the mobile terminal. The seating position of the mobile terminal can be determined by the UWB module of the mobile terminal and the UWB module of the vehicle, and by using UWB positioning technology. The seating position of the mobile terminal can also be determined by performing face recognition on the in-car image through the in-car camera, and determining the seating position of the mobile terminal corresponding to each user based on the correspondence between the user and the mobile terminal. The seating position of the mobile terminal can also be determined by detecting the doors opened by the user and the order in which the user enters the vehicle through the camera outside the vehicle, and determining the seating position of the mobile terminal corresponding to the user based on the doors opened by the user, the order in which the user enters, and the correspondence between the user and the mobile terminal.

[0244] It should be understood that, without departing from the teachings of this application, the display end can also determine the candidate display screens of the projection end based on other rules, and this application does not limit the method of determining the candidate display screens.

[0245] After completing the exemplary description of the method of determining the candidate display screens, the following exemplary description is given of the method in which the display end selects a display screen for displaying projection data from the candidate display screens based on the occupancy of the candidate display screens.

[0246] In some embodiments of the present application, the way in which the display end selects the display screen corresponding to the projection end from the candidate display screens may include: in response to the existence of an unoccupied candidate display screen, the display end selects the display screen corresponding to the projection end from the unoccupied candidate display screens.

[0247] For example, if the number of unoccupied candidate display screens is 1, the display end uses the unoccupied display screen as the display screen corresponding to the projection end. If the number of unoccupied candidate display screens is greater than 1, the display end may randomly select an unoccupied candidate display screen as the display screen corresponding to the projection end, or the display end may select the candidate display screen with the highest priority among the unoccupied candidate display screens. The priority of each candidate display screen can be determined based on its distance from the projection end, or it can be set by the user, and this application does not impose any restrictions on this.

[0248] In some embodiments of the present application, the display end may select the display screen corresponding to the projection end from the candidate display screens, including: in response to the candidate display screens being occupied, the display end may return an occupation request, wherein the occupation request indicates a request to occupy the candidate display screen; and in response to detecting an occupation permission instruction, the candidate display screen is selected as the selected display screen. In other words, if the candidate display screen of the projection end is occupied, the projection end is allowed to use the candidate display screen after obtaining the occupation permission.

[0249] As an example, the display end may provide feedback on the occupation request by displaying relevant information about the occupation request, such as the login user information of the projection end and a summary of the projection data, on the candidate display screen. Based on the user operation information detected by the candidate display screen, the display end determines whether the user has triggered an occupation permission instruction. Another method for the display end to provide feedback on the occupation request is to send the occupation request to the occupying end of the candidate display screen. The occupying end displays relevant information about the occupation request and, based on the detected user operation information, determines whether to send an occupation permission instruction to the display end.

[0250] It should be understood that, without departing from the teachings of this application, the display terminal may also feedback the occupancy request in other ways, and this application does not impose any restrictions on this.

[0251] As an example, if the number of candidate displays is equal to 1, the display end will feedback an occupancy request for the candidate display. If the number of candidate displays is greater than 1, the display end can feedback an occupancy request for each candidate display. If an occupancy permission instruction is received for a candidate display, the candidate display will be selected as the display. If occupancy permission instructions are received for multiple candidate displays, the display corresponding to the projection end can be selected based on the priority of the multiple candidate displays, or the display corresponding to the projection end can be randomly selected.

[0252] It should be understood that, without departing from the teachings of this application, if the number of candidate display screens is greater than 1, the display end may also use other methods to feedback the occupancy request. For example, the occupancy request may be fed back in sequence according to the priority of each candidate display screen. If an occupancy permission instruction is detected, the current candidate display screen will be used as the selected display screen. If an occupancy rejection instruction is detected, the occupancy request will be fed back for the next candidate display screen.

[0253] It should be understood that, without departing from the teachings of this application, if the occupancy request fed back to the candidate display screen is an occupancy rejection instruction, the display end can feedback the projection failure information to the projection end, etc., or take other measures, and this application does not impose any restrictions on this.

[0254] In some embodiments of the present application, the display end may select the display screen corresponding to the projection end from the candidate display screens by the display end: in response to the candidate display screens being occupied and the priority of the projection end being higher than the priority of the occupying end of the candidate display screen, the display end selects the candidate display screen as the selected display screen. In other words, if the candidate display screen is occupied, but the priority of the projection end is higher than the priority of the occupying end, the projection end may occupy the candidate display screen.

[0255] It should be understood that those skilled in the art can understand that the above-mentioned method of determining the display screen corresponding to the projection end can be performed at any step before displaying the projection data after determining that the projection end is close to the display end, and the order of implementation of each step can be set as needed. For example, the projection end determines the projection data through the virtual screen created by the projection application. After determining to trigger the projection instruction, the display end can determine the display screen corresponding to the projection end through the above example, and send the size information of the display screen corresponding to the projection end to the projection end. The projection application of the projection end creates a virtual screen according to the size information of the display screen corresponding to the projection end. For another example, the projection end obtains the projection data through other methods such as screen recording. After receiving the projection data, the display end can determine the display screen corresponding to the projection end through the above steps, and display the received projection data through the display screen corresponding to the projection end. This application does not limit the timing of triggering the projection instruction by the projection end to determine the display screen corresponding to the projection end.

[0256] The above is an example of how to schedule the display screen of the display terminal for different projection terminals. The following is an example of how to schedule the display screen of the display terminal for different projection data.

[0257] In some embodiments of the present application, the way in which the display end displays projection data through at least one display screen may include: in response to detecting that the display screens of the display end corresponding to multiple projection data obtained based on different virtual screens are all first display screens, according to a preset scheduling strategy, controlling the first display screen to display at least one of the multiple projection data.

[0258] For ease of understanding, the scheduling strategy is illustrated below.

[0259] Example 1 of Scheduling Strategy

[0260] In some embodiments of the present application, the display terminal supports split-screen display, and the scheduling strategy indicates: through split-screen display, the projection data of multiple different virtual screens corresponding to the first display screen are displayed on the first display screen.

[0261] As an example, virtual screen 1 of the projection end and virtual screen 2 of the projection end both correspond to the first display screen of the display end, and projection data 1 obtained based on virtual screen 1 of the projection end and projection data 2 obtained based on virtual screen 2 of the projection end both indicate that they are displayed through the first display screen of the display end. Based on embodiment 1 of the scheduling strategy, the display end displays projection data 1 and projection data 2 on the first display screen in split-screen mode.

[0262] In some embodiments of the present application, the scheduling strategy also indicates: according to a preset division rule, determine the display position of multiple projection data on the first display screen. For example, if the first display screen of the display end corresponds to multiple projection data, the display end may first divide the first display screen of the display end into multiple display areas according to the number of projection data corresponding to the first display screen, and number each display area on the display screen according to the rule from top to bottom and from left to right (for example, numbered 1, 2...). The display end determines the display area for displaying each projection data according to the preset division rule.

[0263] For ease of understanding, the division rules are exemplified below.

[0264] As an example, the division rule indicates: according to the priority of the application modules corresponding to the multiple projection data, determine the display positions of the multiple projection data on the first display screen. For example, the display terminal allocates display areas with smaller numbers to the projection data corresponding to the application modules with higher priorities according to the priority of the application modules corresponding to the multiple projection data. The display terminal displays each projection data through the display area allocated to each projection data. The display terminal can display the application modules that are used more frequently or have higher attention in the process of the user using the display terminal in a display area that is more convenient for the user to watch, so that the user can use the functions of the application module more conveniently.

[0265] As another example, the division rule indicates: determine the display positions of multiple projection data on the first display screen according to the application types to which the application modules corresponding to the multiple projection data belong. For example, the display terminal allocates display areas with smaller numbers to projection data corresponding to application types with higher priorities according to the priorities of the application types to which the application modules corresponding to the multiple projection data belong. If there are projection data of the same application type, display areas can be randomly allocated to the projection data of the same application type, or they can be allocated display areas according to the priorities of the corresponding application modules. The display terminal displays each projection data through the display area allocated to each projection data. The display terminal can display application modules of application types that are used more frequently or have received more attention in the process of users using the display terminal in a display area that is more convenient for users to watch, so that users can use the functions of the application modules more conveniently.

[0266] As another example, the division rule indicates: according to the startup order of the application modules corresponding to the multiple projection data, the display positions of the multiple projection data on the first display screen are determined. For example, the display end allocates a display area with a smaller number to the projection data corresponding to the application module with an earlier startup order, or allocates a display area with a larger number to the projection data corresponding to the application module with an earlier startup order, based on the startup order of the application modules corresponding to the multiple projection data. The display end displays each projection data through the display area allocated to each projection data. The display end can display the latest launched application module in a display area that is more convenient for the user to watch, so that the user can use the functions of the application module more conveniently.

[0267] It should be understood that the division rules can be adjusted as needed without departing from the teachings of this application, and this application does not impose any restrictions on this.

[0268] Example 2 of Scheduling Strategy

[0269] In some embodiments of the present application, the scheduling strategy indicates: displaying the first projection data among multiple projection data on the first display screen; and displaying other projection data among the multiple projection data except the first projection data in the form of picture-in-picture.

[0270] As an example, the first projection data may be the projection data with the highest priority among the multiple projection data. The priority of the multiple projection data may be determined according to the startup order of the application modules corresponding to the multiple projection data, or the priority of the multiple projection data may be determined according to the startup order of the application modules corresponding to the multiple projection data. This application does not impose any restrictions on this.

[0271] As an example, when displaying other projection data in the form of picture-in-picture, you can refer to the division rules similar to those mentioned above, and simultaneously pop up picture-in-picture display windows for displaying other projection data in different display areas of the first display screen. You can also pop up picture-in-picture display windows for displaying other projection data in the same display area (for example, the central area of ​​the first display screen) at a certain time interval. This application does not impose any restrictions on this.

[0272] For example, projection data 1 obtained based on virtual screen 1 on the projection end, projection data 2 obtained based on virtual screen 2 on the projection end, and projection data 3 obtained based on virtual screen 3 on the projection end all indicate that they are to be displayed on the first display screen of the display end. Projection data 1 has the highest priority, and the display end displays projection data 1 on the first display screen, and displays projection data 1 and projection data 2 in a picture-in-picture manner.

[0273] Example 3 of Scheduling Strategy

[0274] In some embodiments of the present application, the scheduling information indicates: feeding back summary information of multiple projection data to the user; obtaining user selection information, and determining the projection data selected by the user based on the user selection information; and displaying the projection data selected by the user on the first display screen.

[0275] As an example, the summary information of the projection data includes at least one of the following: an identifier of the application module corresponding to the projection data (such as the name or icon of the application module); and at least one frame of image to be projected obtained based on the projection data.

[0276] It should be understood that, without departing from the teachings of this application, the summary information of the projection data may also include other information, and this application does not limit this.

[0277] For example, the projection data 1 obtained based on the virtual screen 1 of the projection end, the projection data 2 obtained based on the virtual screen 2 of the projection end, and the projection data 3 obtained based on the virtual screen 3 of the projection end are all indicated to be displayed through the first display screen of the display end. The display end can display the summary information of the projection data 1, the summary information of the projection data 2, and the summary information of the projection data 3 in the form of a list, for example. If the user needs to view the projection data 1, he clicks on the area displaying the summary information of the projection data 1 through the first display screen. After the display end detects that the area triggered by the user is the area displaying the summary information of the projection data 1, the projection data 1 is displayed on the first display screen.

[0278] It should be understood that, without departing from the teachings of this application, the scheduling strategy may also be set based on other rules, and this application does not limit this.

[0279] In some embodiments of the present application, the display end may display the projection data after receiving the projection data, or may display the projection data after receiving the projection data and satisfying the display conditions.

[0280] For example, the projection end is a mobile terminal, and the display end is a vehicle. The vehicle detects whether the mobile terminal is close to the vehicle through the distance detection module at a first frequency. After detecting that the mobile terminal is close to the vehicle, the vehicle triggers a projection request, and detects whether the mobile terminal has entered the vehicle through the distance detection module at a second frequency, and the second frequency is greater than the first frequency. After detecting that the mobile terminal has entered the vehicle, the vehicle displays the projection data transmitted by the mobile terminal. Before the mobile terminal approaches the vehicle, the vehicle detects the distance between the mobile terminal and the vehicle at a lower frequency to determine whether the mobile terminal is close to the vehicle, which can reduce the power consumption of the vehicle's distance detection module. After determining that the mobile terminal is close to the vehicle, the vehicle detects the distance between the mobile terminal and the vehicle at a higher frequency to determine whether the mobile terminal has entered the vehicle, so that the vehicle can provide projection data to the user in a more timely manner.

[0281] In some embodiments of the present application, the display screen of the display end includes at least a first display screen and a second display screen, and the projection data of the projection end is displayed at least through the first display screen. In response to the first display screen detecting the information sharing request, the display end feeds back the information sharing request, and the information sharing request indicates that the projection data of the projection end is shared to the second display screen; in response to receiving the sharing confirmation instruction, the projection data of the projection end is displayed through the first display screen and the second display screen. In other words, while the display end displays the projection data of the projection end through the first display screen, if the user needs to share the projection data to other display screens of the display end (referred to as the target display screen for short), the information sharing request can be triggered by clicking the sharing button of the first display screen, or by a sliding gesture of the instruction. The display end can feed back the information sharing request in the form of a pop-up window through the target display screen. If the target display screen detects the sharing confirmation instruction, the display end will jointly display the projection data of the projection end through the first display screen and the second display screen.

[0282] It is worth mentioning that users can initiate information sharing through the display end and share the projection data with other users, so that other users can view the projection data without moving their own position, which improves the user experience.

[0283] In some embodiments of the present application, the display screen of the display terminal includes at least a first display screen and a second display screen, the projection data includes first projection data and second projection data, the first projection data is displayed on the first display screen, and the second projection data is displayed on the second display screen. In response to detecting a screen switching instruction instructing a switch between the first and second display screens, the display terminal controls the second display screen to display the first projection data, and controls the first display screen to display the second projection data.

[0284] As an example, projection data from different sources are transmitted through different transmission links. The display end determines the display screen used to display each projection data based on the stored correspondence between the display screen and the transmission link. After detecting the switching instruction, the correspondence between the display screen and the transmission link is adjusted so that the first display screen corresponds to the second transmission link for transmitting the second projection data, and the second display screen corresponds to the first transmission link for transmitting the first projection data. When the display end projects the screen based on the adjusted correspondence between the display screen and the transmission link, the second display screen can display the first projection data, and the first display screen can display the second projection data.

[0285] It should be understood that the screen switching effect may be achieved in other ways without departing from the teachings of this application, and this application does not limit this.

[0286] To facilitate understanding, the following takes the projection end as a mobile terminal and the display end as a vehicle as an example, and combines user operations to exemplify the screen projection process between the mobile terminal and the vehicle.

[0287] The mobile terminal and the vehicle can transmit data through a USB connection or a Wi-Fi connection, and transmit the projection data of the mobile terminal to the vehicle. After the projection application is turned on, the central control display of the vehicle can first display the following Figure 4 The operation interface shown is displayed so that the user can select the application to be triggered.

[0288] User operation 1: The user clicks on the icon of application module 1 (such as a navigation module) on the central control display screen. The central control display screen detects the first operation information and sends the first operation information to the mobile terminal. The mobile terminal receives the first operation information, and based on the position of the area where the touch signal is detected on the central control display screen, analyzes that the first operation information indicates to open the application module 1, and draws the operation interface of the application module 1 on the virtual screen 1 (Vistual Display) created based on the central control display screen. The projection data 1 determined according to the virtual screen 1 is transmitted to the vehicle through the interconnection channel between the projection application and the projection application. Based on the received projection data 1, the vehicle displays the screen interface of the virtual screen 1, that is, the operation interface of the application module 1, in the first display area of ​​the central control display screen, and displays the identification information of the application module in the projection application and the identification information of the third-party application in the second display area of ​​the central control display screen.

[0289] User operation 2: The user switches the central control display screen and the passenger side display screen (for example, a three-finger swipe operation on the central control display screen, clicking a switch button, etc.). After the central control display screen detects the second operation information, the response mode may include at least the following two methods:

[0290] Response method 1

[0291] The mobile terminal detects the position of the area of ​​the touch signal in the user operation information based on the central control display screen, and determines the coordinate point touched by the user's finger and the movement distance of the user's finger; if the touch coordinate point and the movement distance indicate that the user has performed a screen switching operation, the corresponding relationship between the screen interface of the application module 1 and the vehicle's display screen is changed through the screen switching method mentioned above, so that the projection data 1 is displayed through the co-pilot display screen, that is, the co-pilot display screen displays the operation interface of the application module 1, and the central control display screen continues to display Figure 4 The specific switching method can be referred to the relevant description above and will not be repeated here.

[0292] Response method 2

[0293] The vehicle detects that the user clicks the switch button, triggering a switch instruction, and performs the screen switching operation through the display end's response to the switch instruction mentioned above.

[0294] User operation 3: The user clicks on the icon of application module 2 (such as video module) on the central control display screen. The central control display screen detects the third operation information and sends the third operation information to the mobile terminal. The mobile terminal receives the third operation information, and based on the position of the area where the touch signal is detected on the central control display screen, analyzes that the third operation information indicates to open the application module 2, draws the operation interface of the application module 2 on the virtual screen 1 (Vistual Display) created based on the central control display screen, and transmits the projection data 2 determined according to the virtual screen 1 to the vehicle through the interconnection channel between the projection application and the projection application. Based on the received projection data 2, the vehicle displays the screen interface of the virtual screen 2, that is, the operation interface of the application module 2, in the first display area of ​​the central control display screen, and displays the identification information of the application module in the projection application and the identification information of the third-party application in the second display area of ​​the central control display screen. It should be understood that if other functions besides navigation and video need to be implemented, it can be implemented by compiling an application module with this function in the projection application.

[0295] User operation 4: The user switches the central control display screen and the co-pilot display screen, and the vehicle and mobile terminal switch the central control display screen and the co-pilot display screen in a similar manner to the response to user operation 2, so that the first display area of ​​the central control display screen displays the operation interface of application module 1, the second display area of ​​the central control display screen displays the identification information of the application module in the projection application and the identification information of the third-party application, and the co-pilot display screen displays the operation interface of application module 2.

[0296] User operation 5: The user clicks on third-party application 1 in the second display area of ​​the central control display screen, and specifies that third-party application 1 be displayed on the main driver's rear screen. The central control display screen detects the third operation information and sends the third operation information to the mobile terminal. The mobile terminal receives the third operation information and, based on the position of the area where the touch signal is detected on the central control display screen, analyzes that the third operation information indicates to open third-party application 1. The projection end obtains projection data 3 of third-party application 1 through the following second projection method, and sends projection data 1, projection data 2 and projection data 3 to the vehicle, so that the vehicle displays projection data 1 through the central control display screen, projection data 2 through the co-pilot display screen, and projection data 3 through the main driver's rear screen.

[0297] It can be seen from the above content that the implementation method of the present application realizes multi-channel screen projection on the basis of realizing mobile terminal screen projection, and different projection data can be transmitted through different transmission links to realize multi-screen interaction.

[0298] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0299] An embodiment of the present application also provides an electronic device, comprising: at least one processor; a communication unit communicatively connected to the at least one processor, for performing data transmission under the control of the processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the response method of the projection end mentioned in the above embodiment, where the electronic device is the projection end; or, can execute the response method of the display end mentioned in the above embodiment, where the electronic device is the display end.

[0300] In some embodiments of the present application, the display end is a smart cockpit.

[0301] In some embodiments of the present application, the display end is a vehicle.

[0302] One embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the response method of the projection end or the response method of the display end.

[0303] Figure 8 A schematic block diagram of an electronic device 400 that can be used as a projection end or display end of an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0304] like Figure 8As shown, electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. Various programs and data required for the operation of electronic device 400 can also be stored in RAM 403. Computing unit 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0305] Multiple components in electronic device 400 are connected to I / O interface 405, including: an input unit 406, such as a touch screen, microphone, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, optical disk, etc.; and a communication unit 409, such as a network card, modem, wireless communication transceiver, etc. The communication unit 409 allows electronic device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0306] The computing unit 401 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as the response method of the projection end or the response method of the display end. For example, in some embodiments, the response method of the projection end or the response method of the display end can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the response method of the projection end or the response method of the display end described above can be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the response method of the projection end or the response method of the display end in any other appropriate manner (e.g., by means of firmware).

[0307] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0308] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0309] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0310] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor; and a keyboard and pointing device (such as a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (such as visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0311] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0312] Computer systems may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other.

[0313] The above description is merely an embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of protection involved in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the technical concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A response method of a projection terminal, characterized in that: The screen projection terminal is installed with a screen projection application and a third-party application, and the screen projection application is provided with multiple application modules; the method includes: Get user operation information; In response to the user operation information instructing to project the application module, obtaining the projected screen data of the application module through a first projecting method, wherein the first projecting method indicates: drawing a screen interface of a virtual screen created by the projecting application based on the interface information of the application module, and determining the projected screen data based on the screen interface of the virtual screen; In response to the user operation information instructing to project the screen of the third-party application, obtaining the projected screen data of the third-party application through a second projecting method; and Transmitting the projection data to a display terminal so as to display the projection data through at least one display screen of the display terminal; in, The second screen projection mode indicates: drawing the screen interface of the display screen of the projection end according to the interface information of the third-party application, and obtaining the projection data by recording the display screen of the projection end; or The second screen projection method indicates: obtaining the virtual screen created by the screen projection application through the operating system of the screen projection end, and drawing the screen interface of the virtual screen according to the interface information of the third-party application; determining the screen projection data according to the screen interface of the virtual screen through the screen projection application.

2. The method according to claim 1, wherein Drawing the screen interface of the virtual screen created by the screen projection application according to the interface information of the application module includes: Determining a virtual screen corresponding to the application module; and According to the interface information of the application module, a screen interface of the virtual screen corresponding to the application module is drawn.

3. The method according to claim 2, wherein: Determining the virtual screen corresponding to the application module includes: Determining a virtual screen of the candidate application module; and In response to the candidate virtual screen having drawn interface information of other application modules other than the application module, and the other application modules and the application module satisfy a first constraint relationship, a new virtual screen is created as the virtual screen corresponding to the application module, and the display screen of the display end corresponding to the new virtual screen is the same as the display screen of the display end corresponding to the candidate virtual screen.

4. The method according to claim 3, wherein: The determining the projection data according to the screen interface of the virtual screen includes: Determining a display screen of a display terminal corresponding to the virtual screen; Obtaining the number of virtual screens corresponding to the determined display screen of the display terminal; and In response to the acquired number of virtual screens being greater than 1, the size information of the screen interface to be projected extracted from the screen interface of the virtual screen is determined based on the number of screens and predefined cropping rules; the screen interface to be projected is extracted from the screen interface of the virtual screen based on the determined size information; and the projection data is determined based on the screen interface to be projected.

5. The method according to claim 3, wherein Determining a candidate virtual screen for the application module includes: Determining a candidate virtual screen of the application module according to the application type of the application module, the correspondence between the application type and the display screen of the display terminal, and the correspondence between the display screen of the display terminal and the virtual screen; or Determine a candidate virtual screen of the application module according to the identification information of the application module, the correspondence between the application module and the display screen of the display terminal, and the correspondence between the display screen of the display terminal and the virtual screen; or A pre-specified virtual screen is used as a candidate virtual screen for determining the application module.

6. The method according to claim 2, wherein: Determining the virtual screen corresponding to the application module includes: The pre-designated virtual screen is used as the virtual screen corresponding to the application module.

7. The method according to claim 1, wherein The method further comprises: In response to the screen switching indicated by the user operation information, the corresponding relationship between the screen interface to be switched indicated by the user operation information and the display screen of the display terminal is changed.

8. The method according to claim 1, wherein The transmitting the projection data to the display end includes: The projection data is transmitted to the display terminal through the projection application.

9. A response method of a display terminal, characterized in that: include: Displaying, through at least one display screen of the display end, identification information of an application module in a screen projection application of a screen projection end interconnected with the display end, and identification information of a third-party application with screen projection permission on the screen projection end; Acquire and transmit user operation information of at least one display screen of the display terminal to the projection terminal; as well as Obtain the projection data determined by the projection end according to the response method described in any one of claims 1 to 8, and display the projection data through at least one display screen of the display end.

10. The method according to claim 9, wherein: Displaying the projection data includes: In response to detecting that the display screens of the display end corresponding to multiple projection data obtained based on different applications are all first display screens, according to a preset scheduling strategy, the first display screen is controlled to display at least one of the multiple projection data.

11. The method according to claim 10, wherein: The scheduling strategy indicates: The multiple projection data are displayed on the first display screen in a split-screen display manner.

12. The method according to claim 11, wherein The scheduling policy also indicates: Determining display positions of the plurality of projection screen data on the first display screen according to a preset division rule; The partitioning rules indicate: Determining display positions of the plurality of projection data on the first display screen according to the priorities of the application modules corresponding to the plurality of projection data; or Determining display positions of the plurality of projection data on the first display screen according to application types to which application modules corresponding to the plurality of projection data belong; or, The display positions of the multiple projection data on the first display screen are determined according to the startup order of the application modules corresponding to the multiple projection data.

13. The method according to claim 10, wherein: The scheduling strategy indicates: Displaying the first projection data among the plurality of projection data on the first display screen; and The other projection data among the multiple projection data except the first projection data are displayed in the form of picture-in-picture.

14. The method according to claim 10, wherein: The scheduling strategy indicates: Feedback summary information of the multiple projection screen data to the user; Obtaining user selection information, and determining the projection data selected by the user based on the user selection information; and The projection data selected by the user is displayed on the first display screen.

15. The method according to claim 9, wherein The screen projection end includes a first screen projection end and a second screen projection end; the method further includes: In response to receiving an interconnection request from the first projection end, and the display end and the second projection end have established an interconnection channel, according to the preset multi-end interconnection rules, determine whether the first projection end and the display end are allowed to establish an interconnection channel for transmitting the projection data of the first projection end.

16. The method according to claim 9, wherein The display screen of the display end includes at least a first display screen and a second display screen, and the projection data of the projection end is displayed at least on the first display screen; the method further includes: In response to the first display detecting an information sharing request, feeding back an information sharing request, wherein the information sharing request indicates sharing the projection data of the projection end to the second display; and In response to receiving the sharing confirmation instruction, the projection data of the projection end is displayed through the first display screen and the second display screen.

17. An electronic device, characterized in that: include: at least one processor; a communication unit communicatively connected to the at least one processor, for performing data transmission under the control of the processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the response method of the projection end as described in any one of claims 1 to 8, where the electronic device is the projection end; or, execute the response method of the display end as described in any one of claims 9 to 16, where the electronic device is the display end.

18. The electronic device according to claim 17, wherein: The display end is a smart cockpit or a vehicle.

19. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the response method of the projection end according to any one of claims 1 to 8, or the response method of the display end according to any one of claims 9 to 16.

Images