Multi-screen interactive control method, system and electronic device

Abstract

A multi-screen interaction control method, system and electronic device, wherein the multi-screen interaction control method includes: obtaining the device identification of the newly connected screen according to the first callback message transmitted in response to the newly connected screen; controlling the graphics engine to create a corresponding graphics engine instance according to the device identification of the newly connected screen; transmitting the identification of the graphics engine instance to the created main interactive service engine instance, so that the main interactive service engine instance derives a slave interactive service engine instance corresponding to the newly connected screen; and recording the master-slave relationship between the main interactive service engine instance and the slave interactive service engine instance. The above scheme can improve the versatility of multi-screen interaction technology.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of multi-screen interaction technology, and more particularly to a multi-screen interaction control method, system and electronic device. Background Art

[0002] Now more and more cars have realized multi-screen interaction. One of the most obvious features of multi-screen interaction is that it can integrate more functions to meet the different riding needs of the main driver, the co-driver and the rear seat users. Multi-screen interaction can be divided into local multi-screen interaction and remote multi-screen interaction. Among them, local multi-screen interaction refers to the coordinated display and interactive control between multiple display screens in the same space, for example, the coordinated display and interactive control between the central control display screen, the instrument panel display screen and the display screen in the rear seat area of ​​the vehicle; remote multi-screen interaction refers to the coordinated display and interactive control between multiple display screens in different spaces. For example, the mobile terminal (mobile phone), computer terminal and car terminal can realize the coordinated display and interactive control of the display screens at different terminals through the interoperable account system.

[0003] In order to adapt to the development trend of multi-screen interaction technology and meet user needs, travel applications (such as map navigation applications) that support multi-screen interaction have begun to appear. Due to the different screen sizes and resolutions of display screens, the interaction methods are also different. If each additional interactive display screen needs to be developed in a targeted manner, there will be problems with delayed functional support. Therefore, providing a highly versatile multi-screen interaction technology is a problem that technicians in this field need to solve. Summary of the invention

[0004] In view of this, the embodiments of this specification provide a multi-screen interactive control method, system and electronic device with strong versatility.

[0005] The embodiment of this specification provides a multi-screen interactive control method, the method comprising:

[0006] Acquire a device identifier of a newly connected screen according to a first callback message transmitted in response to a newly connected screen;

[0007] According to the device identification of the newly connected screen, controlling the graphics engine to create a corresponding graphics engine instance;

[0008] Passing the identifier of the graphics engine instance to the created master interactive service engine instance, so that the master interactive service engine instance derives a slave interactive service engine instance corresponding to the newly accessed screen;

[0009] Recording the master-slave relationship between the master interactive service engine instance and the slave interactive service engine instance;

[0010] In combination with the core business data output by the core business engine, the interactive business data output by the interactive business engine and the graphic data output by the graphic engine, the interactive business engine instance corresponding to the new access screen controls the graphics engine instance corresponding to the new access screen to output the corresponding core business information, interactive business information and graphic elements to the new access screen.

[0011] Optionally, the method further comprises:

[0012] Acquire a device identifier of the disconnected screen according to a second callback message delivered in response to the screen being disconnected;

[0013] According to the device identifier of the disconnected screen, canceling the master-slave relationship between the slave interactive service engine instance corresponding to the disconnected screen and the master interactive service engine instance;

[0014] The master interactive service engine instance destroys the slave interactive service engine instance corresponding to the disconnected screen;

[0015] Destroy the graphics engine instance corresponding to the disconnected screen.

[0016] Optionally, the method further includes: determining whether to deliver the message to all interactive service engine instances or some interactive service engine instances according to the type of the received message, including:

[0017] The received global message is delivered to the master interactive service engine instance, and synchronized to the slave interactive service engine instance, so that the master interactive service engine instance and the slave interactive service engine instance respectively perform corresponding processing based on the global message;

[0018] The received non-global message is delivered to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance performs corresponding processing based on the non-global message.

[0019] Optionally, delivering the received global message to the master interactive service engine instance and synchronizing it to the slave interactive service engine instance, so that the master interactive service engine instance and the slave interactive service engine instance respectively perform corresponding processing based on the global message, includes:

[0020] The core business update information transmitted by the core business engine is respectively transmitted to the main interactive business engine instance and the slave interactive business engine instance, so that the main interactive business engine instance and the slave interactive business engine instance respectively control the corresponding graphics engine instance based on the received core business update information to update the corresponding graphic elements in the screen according to the core business update information.

[0021] Optionally, delivering the received non-global message to a corresponding interactive service engine instance so that the corresponding interactive service engine instance performs corresponding processing based on the non-global message includes:

[0022] acquiring an instruction for switching the screen perspective transmitted in response to detecting that the screen perspective is switched;

[0023] The instruction for switching the screen perspective is passed to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance follows the instruction for switching the screen perspective and controls the corresponding interactive service engine instance to output the corresponding graphic elements according to the corresponding graphic engine instance parameters after switching the screen perspective.

[0024] Optionally, delivering the received non-global message to a corresponding interactive service engine instance so that the corresponding service engine instance performs corresponding processing based on the non-global message includes:

[0025] receiving an interactive service engine control instruction or a core service engine control instruction transmitted in response to an interactive control operation;

[0026] The interactive service engine control instruction or the core service engine control instruction is transmitted to the main interactive service engine instance, and the main interactive service engine instance transmits the instruction to the interactive service engine or the core service engine accordingly;

[0027] The status information transmitted by the interactive service engine or the core service engine is transmitted to the corresponding interactive service engine instance respectively, and each interactive service engine instance controls the corresponding graphic engine instance to execute the processing operation corresponding to the interactive service control instruction or the core service engine control instruction.

[0028] Optionally, when the master interactive service engine instance derives a slave interactive service engine instance corresponding to the new access screen, it further includes:

[0029] The master interactive service engine instance sets parameters of the slave interactive service engine instance.

[0030] Optionally, the graphics engine comprises: a map engine adapted to control a corresponding map engine instance to output map elements based on map data;

[0031] The core service engine includes a positioning engine, which is adapted to comprehensively calculate the current coordinates of the user based on the measurement data from the positioning sensor and the acquired route planning data to obtain positioning data;

[0032] The interactive service engine includes a navigation engine, and the navigation engine is adapted to output user navigation guidance information according to the positioning data and route planning data output by the positioning engine;

[0033] The interactive service engine instance includes a travel engine instance, and the travel engine instance is suitable for combining positioning data and user navigation guidance information to control the corresponding map engine instance to output the corresponding map elements and user navigation guidance information to the screen.

[0034] The embodiment of this specification also provides a multi-screen interactive control system, the system comprising:

[0035] An interactive interface, adapted to receive a first callback message transmitted in response to a new access to a screen;

[0036] The central processing unit is adapted to obtain a device identifier of the newly accessed screen based on the first callback message, control the graphics engine to create a corresponding graphics engine instance, and pass the graphics engine instance to the created master interactive service engine instance, so that the master interactive service engine instance derives a slave interactive service engine instance corresponding to the newly accessed screen and records the master-slave relationship between the master interactive service engine instance and the slave interactive service engine instance;

[0037] The slave interactive service engine instance corresponding to the new access screen is suitable for combining the core business data output by the core business engine, the interactive service data output by the interactive service engine and the graphic data output by the graphic engine, and controlling the graphic engine instance corresponding to the new access screen to output the corresponding core business information, interactive service information and graphic elements to the new access screen through the interactive interface of the new screen.

[0038] The embodiment of this specification also provides another multi-screen interactive control system, the system comprising:

[0039] A central processing unit, adapted to create a graphics engine, a core business engine, an interactive business engine, a message center and a user interaction interface when a business application is started, and to create a corresponding main interactive business engine instance based on the interactive business engine when an interactive business start instruction is obtained through the user interaction interface of the main screen; to obtain a device identifier of the newly connected screen according to a first callback message transmitted when a screen is newly connected, to control the graphics engine to create a corresponding graphics engine instance according to the device identifier of the newly connected screen, and to transmit the identifier of the graphics engine instance to the main interactive business engine instance;

[0040] Core business engine, suitable for outputting core business data;

[0041] An interactive service engine, suitable for outputting interactive service data;

[0042] A graphics engine, suitable for outputting graphics data;

[0043] The interactive service engine instance corresponds to the screen one by one, and is adapted to control the graphics engine instance of the corresponding screen to output the corresponding core service information, interactive service information and graphic elements to the corresponding screen in combination with the core service data output by the core service engine, the interactive service data output by the interactive service engine and the graphic data output by the graphic engine; the interactive service engine instance comprises a main interactive service engine instance and a slave interactive service engine instance, wherein the main interactive service engine instance is adapted to derive a slave interactive service engine instance corresponding to the newly accessed screen according to the received identifier of the graphics engine instance corresponding to the newly accessed screen;

[0044] The message center is suitable for communicating with the central processing unit, the core business engine, the interactive business engine and each interactive business engine instance by using a message distribution and subscription method, and recording the master-slave relationship between the master interactive business engine instance and the slave interactive business engine instance.

[0045] Optionally, the central processing unit is further adapted to obtain a device identifier of the disconnected screen according to a second callback message transmitted in response to the screen being disconnected, and instruct the message center to cancel the master-slave relationship between the slave interactive service engine instance corresponding to the disconnected screen and the master interactive service engine instance, instruct the master interactive service engine instance to destroy the slave interactive service engine instance corresponding to the disconnected screen, and destroy the graphics engine instance corresponding to the disconnected screen;

[0046] The message center is adapted to cancel the master-slave relationship between the slave interactive service engine instance and the master interactive service engine instance corresponding to the disconnected screen according to the device identifier of the disconnected screen.

[0047] Optionally, the message center is also suitable for determining whether to deliver the message to all interactive service engine instances or some interactive service engine instances according to the type of received message, including: delivering the received global message to the main interactive service engine instance, and synchronizing it to the slave interactive service engine instance, so that the main interactive service engine instance and the slave interactive service engine instance respectively perform corresponding processing based on the global message, and delivering the received non-global message to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance performs corresponding processing based on the non-global message.

[0048] Optionally, the message center is suitable for transmitting the core business update information transmitted by the core business engine to the main interactive business engine instance and the slave interactive business engine instance respectively, so that the main interactive business engine instance and the slave interactive business engine instance update the corresponding graphic elements in the screen according to the core business update information.

[0049] Optionally, the central processing unit is adapted to obtain an instruction for switching the screen perspective transmitted in response to detecting the switching of the screen perspective, and transmit the instruction to the message center;

[0050] The message center is also suitable for transmitting the instruction of switching the screen perspective to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance controls the corresponding graphics engine instance to output graphic elements according to the corresponding graphics engine instance parameters after the screen perspective is switched according to the instruction of switching the screen perspective.

[0051] Optionally, the central processing unit is adapted to receive an interactive service engine control instruction or a core service engine control instruction transmitted in response to an interactive control operation and transmit the instruction to the message center;

[0052] The message center is suitable for transmitting the interactive service engine control instruction or the core service engine control instruction to the main interactive service engine instance, and the main interactive service engine instance transmits it to the corresponding slave interactive service engine or core service engine; and transmitting the status information transmitted by the interactive service engine or the core service engine to the corresponding interactive service engine instance respectively, and each interactive service engine instance controls the corresponding graphics engine instance to execute the processing operation corresponding to the interactive service control instruction or the core service engine control instruction.

[0053] Optionally, the master interactive service engine instance also sets parameters of the slave interactive service engine instance when deriving a slave interactive service engine instance corresponding to the new access screen.

[0054] Optionally, the graphics engine comprises: a map engine adapted to control a corresponding map engine instance to output map elements based on map data;

[0055] The core service engine includes a positioning engine, which is adapted to comprehensively calculate the current coordinates of the user based on the measurement data from the positioning sensor and the acquired route planning data to obtain positioning data;

[0056] The interactive service engine includes a navigation engine, and the navigation engine is adapted to output user navigation guidance information according to the positioning data and route planning data output by the positioning engine;

[0057] The interactive service engine instance includes a travel engine instance, and the travel engine instance is suitable for combining positioning data and user navigation guidance information to control the corresponding map engine instance to output the corresponding map elements and user navigation guidance information to the screen.

[0058] An embodiment of the present specification also provides an electronic device, which includes: a memory, a processor and an expansion interface, wherein the expansion interface is suitable for coupling to an external screen, the memory stores computer instructions that can be executed on the processor, and the processor is suitable for executing the multi-screen interactive control method described in any of the aforementioned embodiments when running the computer instructions.

[0059] Optionally, the extended interface includes at least one of the following:

[0060] Wired interface;

[0061] Wireless interface.

[0062] By adopting the multi-screen interactive control scheme of the embodiment of the present specification, when a new screen is detected to be connected, a corresponding graphics engine instance is generated, and a slave interactive service engine instance can be automatically derived through the created main interactive service instance, and then the derived slave interactive service engine instance controls the graphics engine instance corresponding to the newly connected screen to output the corresponding core business information, interactive business information and graphic elements to the new screen. Therefore, there is no need to develop pages separately for different display screens. Therefore, it has strong versatility and reusability, which is conducive to realizing interaction between different types of screens, thereby improving development efficiency.

[0063] Furthermore, the message center communicates with the central processing unit, the core business engine, the interactive business engine and each interactive business engine instance, and performs message interaction through message distribution and subscription. The message center records the master-slave relationship between the main interactive business engine instance and the slave interactive business engine instance. This can reduce the coupling of message interaction, thereby improving the versatility and reusability of the multi-screen interactive control solution, and further improving development efficiency.

[0064] Furthermore, based on the device identification of the corresponding screen obtained when the screen is connected or disconnected, the corresponding map instance is created or destroyed, and the main interactive business engine accordingly creates or destroys the corresponding slave interactive business engine instance, thereby realizing hot plugging of the screen during the business execution of the business application. The creation and destruction of the slave interactive business engine instance and the slave graphics engine instance during the business execution will not affect the operation of the main interactive business engine instance and the main graphics engine instance, thereby reducing the complexity of multi-screen interaction and avoiding user distraction.

[0065] Furthermore, the message center determines whether to pass the message to all interactive service engines or some corresponding interactive service engine instances based on the received message type. On the one hand, each screen can meet the general requirements of the operation of the core business of the application. On the other hand, it can meet the personalized needs of multi-screen interactive users according to the different characteristics of each screen or the different characteristics of the corresponding users.

[0066] Furthermore, by respectively transmitting the core business update information to the main interactive business engine instance and the slave interactive business engine instance, and the main interactive business engine instance and the slave interactive business engine instance respectively control the corresponding graphics engine instance based on the received core business update message to update the corresponding graphic elements in the screen according to the core business update information, it is possible to achieve synchronous update of information between multiple screens and achieve synchronization and consistency of core business information between each screen.

[0067] Furthermore, when an instruction to switch the screen perspective is obtained, the instruction to switch the screen perspective is passed to the corresponding interactive service engine instance, and the corresponding interactive service engine instance is controlled by the corresponding interactive service engine instance to output corresponding graphic elements according to the corresponding graphic engine instance parameters after switching the perspective in accordance with the instruction to switch the screen perspective, thereby enabling users corresponding to each screen to set a personalized screen perspective and improving the user experience.

[0068] Furthermore, the interactive business engine control instructions or the core business engine control instructions are only passed to the main interactive business engine instance, which in turn passes them to the interactive business engine or the core business engine. The status message of the interactive business engine passed by the interactive business engine or the status information of the core business engine passed by the core business engine are respectively passed to each interactive business engine instance, and each interactive business engine instance controls the corresponding map engine instance to execute the processing operation corresponding to the interactive business engine control instruction or the core business engine control instruction. That is, only the main interactive business engine instance can control the core business engine and the interactive business engine, which can avoid control conflicts or repeated issuance of control instructions among the interactive business engine instances, and further improve the consistency of business control of each screen.

[0069] Furthermore, when the main interactive service engine instance derives a slave interactive service engine instance corresponding to the newly accessed screen, by setting the parameters of the slave interactive service engine instance, a slave screen service operation graphical interface different from the main screen service operation can be implemented, thereby further enhancing the personalization of multi-screen interactive control.

[0070] Furthermore, by adopting the multi-screen interactive control scheme of the embodiment of this specification, through the cooperation of the map engine, positioning engine and navigation engine of the multi-screen interactive control system or device, and the creation of a master-slave relationship between the main travel engine instance and the slave travel engine instance, multi-screen interaction of the navigation application can be realized to meet the multi-screen navigation interaction needs of different users in the same vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1A schematic diagram of a multi-screen interaction scenario in an embodiment of this specification is shown;

[0072] Figure 2 A schematic diagram of the structure of a multi-screen interactive control system in an embodiment of this specification is shown;

[0073] Figure 3 A schematic diagram of the structure of another multi-screen interactive control system in an embodiment of this specification is shown;

[0074] Figure 4 A flowchart of a multi-screen interactive control method in an embodiment of this specification is shown;

[0075] Figure 5 A flowchart of another multi-screen interactive control method in an embodiment of this specification is shown;

[0076] Figure 6 A schematic diagram of the structure of a multi-screen interactive control system applied to a map navigation scenario in an embodiment of this specification is shown;

[0077] Figure 7 A schematic diagram of a processing flow of a multi-screen interactive control system applied to a map navigation scenario when a new screen is connected in an embodiment of this specification is shown;

[0078] Figure 8 A schematic diagram of a processing flow of a multi-screen interactive control system applied to a map navigation scenario when the screens are disconnected in an embodiment of this specification is shown;

[0079] Fig. 9 A schematic diagram of the processing flow of a multi-screen interactive control system applied to a map navigation scenario in an embodiment of this specification when updating positioning information is shown;

[0080] Fig.10 A schematic diagram of the processing flow of a multi-screen interactive control system applied to a map navigation scenario in an embodiment of this specification when setting a personalized navigation perspective is shown;

[0081] Fig.11 A schematic diagram of the processing flow of a multi-screen interactive control system applied to a map navigation scenario when exiting navigation in an embodiment of this specification is shown;

[0082] Fig.12 A specific interactive scenario processing schematic diagram of a multi-screen interactive control system applied to a map navigation scenario in an embodiment of this specification is shown.

[0083] Fig.13 A schematic diagram of the structure of another multi-screen interactive control system in an embodiment of this specification is shown. DETAILED DESCRIPTION

[0084] As mentioned earlier, the current multi-screen interaction solutions have different interaction methods due to the different screen sizes and resolutions of the displays. If each additional interactive display screen is required, targeted development is required, and applications (such as map navigation applications) need to be adapted separately for different screens, resulting in low development efficiency.

[0085] Reference Figure 1 In the schematic diagram of the multi-screen interactive scene shown, the electronic device 1 is configured with a memory 11, a processor 12, and can be optionally configured with a display screen 13, and one or more expansion interfaces 14, through which one or more display screens such as display screen 2 and display screen 3 can be connected. Among them, the screen sizes and resolutions of multiple display screens such as display screen 13, display screen 2 and display screen 3 may be different.

[0086] In a specific implementation, the multi-screen interactive control system in the embodiment of this specification can be installed on the memory 11 of the electronic device 1 and run on the processor 12, and interactive control of the multi-screen interactive control system can be achieved through the display screen 13 and the display screen 2, the display screen 3, etc.

[0087] The electronic device may be a fixed electronic device or a mobile device, such as a personal computer, a set-top box or a vehicle-mounted device. Accordingly, the selectively configured display screen may be a computer display screen, a television screen or a vehicle-mounted central control screen, etc. The external display screen may be a mobile terminal screen, a tablet screen, a projection screen, or a screen disposed inside the vehicle (such as a screen disposed on the back of a seat, the inner wall of a window, or a screen hung inside the vehicle). The screens of each display screen may be a liquid crystal display (LCD) screen, an organic light-emitting diode (OLED) screen, etc., and the specific material, process, model, size, resolution, etc. of the screen are not limited.

[0088] The one or more expansion interfaces 14 may be wired interfaces or wireless interfaces, or some interfaces may be wired interfaces and some interfaces may be wireless interfaces, wherein the wired interface may be a Universal Serial Bus (USB) interface, a screen projection interface, etc., and the wireless interface may be a wireless Internet access (WIFI) wireless screen projection interface, etc.

[0089] In order to improve the versatility of multi-screen interaction, the multi-screen interaction control scheme of the embodiment of this specification can be adopted, wherein, when a new screen is detected to be connected, a corresponding graphics engine instance can be generated, and a slave interactive service engine instance can be derived from the created main interactive service instance, so that the derived slave interactive service engine instance controls the graphics engine instance corresponding to the newly connected screen to output the corresponding core business information, interactive business information and graphic elements to the new screen.

[0090] In order to enable those skilled in the art to better understand and implement the embodiments of this specification, the following is a detailed description through specific embodiments and in combination with specific application scenarios with reference to the accompanying drawings.

[0091] In order to facilitate understanding of the technical concept of the embodiments of this specification, the structure of a multi-screen interactive control system in the embodiments of this specification is first introduced. Figure 2 The structural diagram of a multi-screen interactive control system shown in the figure, the multi-screen interactive control system 20 may include a central processing unit 21, a graphics engine 22, a core business engine 23, an interactive business engine 24 and an interactive interface 25. The interactive interface 25 may be set to two or more, and the interactive interface 25 may include a screen interface 251 and a user interactive interface 252 for interaction through the screen, and the user interactive interface 252 may include an input interface (not shown) and an output interface (not shown). In a specific implementation, one of the interactive interfaces 25 may include a screen interface (such as screen interface 1) of the screen of the electronic device running the multi-screen interactive system 20, and a user interactive interface (such as user interactive interface 1) of the screen connected through the screen interface 1; the other interactive interfaces 25 may include a screen interface (such as screen interface 2) of the external screen of the electronic device running the multi-screen interactive system, and a user interactive interface (such as user interactive interface 2) for interactive control through the screen interface of the external screen.

[0092] In a specific implementation, the central processing unit 21 may create a graphic engine 22, a core business engine 23, an interactive business engine 24 and a user interactive interface 252 when a business application is started.

[0093] The graphic engine 22 is suitable for outputting graphic data. The business application program divides the business engine into the core business engine 23 and the interactive business engine 24 according to the business logic, wherein the core business engine 23 executes the core business logic of the business application irrelevant to the interaction and is suitable for outputting the core business data; while the interactive business engine 24 executes the business logic related to the interactive business and is suitable for outputting the interactive business data.

[0094] In a specific implementation, the interactive service engine instance 26 corresponds to the screen one by one, and is suitable for combining the core service data output by the core service engine 23, the interactive service data output by the interactive service engine 24, and the graphic data output by the graphic engine 22, and controlling the graphic engine instance of the corresponding screen to output the corresponding core service information, interactive service information, and graphic elements to the corresponding screen. The interactive service engine instance 26 may include a main interactive service engine instance and a slave interactive service engine instance, wherein the central processing unit 21 may create a corresponding main interactive service engine instance (such as interactive service engine instance 1) based on the interactive service engine 24, and create a corresponding graphic engine instance (such as graphic engine instance 1) based on the graphic engine 22 when obtaining an instruction to start the interactive service through the user interaction interface (such as user interaction interface 1) of the main screen.

[0095] When a new screen is connected, the multi-screen interactive control system in the embodiment of this specification can derive a slave interactive service engine instance from the master interactive service engine instance for the newly connected screen. Figure 2 If a screen is connected to the multi-screen interactive control system 20 through the screen interface 2, the central processing unit 21 can obtain the device identification of the newly connected screen based on the first callback message transmitted in response to the new connection of the screen, and control the graphics engine 22 to create a corresponding graphics engine instance (such as graphics engine instance 2) according to the device identification of the newly connected screen, and pass the identification of the graphics engine instance to the main interactive service engine instance (such as interactive service engine instance 1), and the main interactive service engine instance is suitable for deriving a slave interactive service engine instance corresponding to the newly connected screen based on the received identification of the graphics engine instance corresponding to the newly connected screen.

[0096] The master interactive service engine instance may derive a slave interactive service engine instance corresponding to the newly accessed screen in the following manner: create a slave interactive service engine instance and synchronize its own state information to the newly created slave interactive service engine instance.

[0097] In a specific implementation, the multi-screen interactive service system may also include a message center, such as Figure 3 Another structural diagram of a multi-screen interactive system is shown in FIG. Figure 2The difference is that the multi-screen interactive control system 30 can also include a message center 27, and the message center 27 is suitable for communicating with the central processing unit 21, the core business engine 23, the interactive business engine 24 and each interactive business engine instance 26 in a message distribution and subscription manner, and can record the master-slave relationship between the main interactive business engine instance and the slave interactive business engine instance. By adopting the above embodiment, message interaction is carried out in a message distribution and subscription manner, and the message center records the master-slave relationship between the main interactive business engine instance and the slave interactive business engine instance, which can reduce the coupling of information interaction, thereby further improving the versatility and reusability of the multi-screen interactive control solution, and further improving development efficiency.

[0098] It is understandable that, in a specific implementation, the central processing unit 21 or other functional units may also record the master-slave relationship between the master interactive service engine instance and the slave interactive service engine instance.

[0099] The following describes the multi-screen interaction principle in the embodiments of this specification through the specific interaction process of multi-screen interactive control.

[0100] Reference Figure 4 The flowchart of a multi-screen interaction control method shown in FIG. 1 is a flowchart of a multi-screen interaction control method shown in FIG. 1 , and the multi-screen interaction can be performed by using the following steps:

[0101] S41 obtains a device identifier of a newly connected screen according to a first callback message transmitted in response to a newly connected screen.

[0102] In a specific implementation, when the operating system detects that a new screen is connected, it can transmit the first callback message to the multi-screen interactive control system through the interactive interface, and the device identifier of the newly connected screen can be transmitted to the central processing unit through the first callback message. Figure 2 When the screen interface 2 detects that a new screen is connected, the operating system can transmit the first callback message to the central processing unit 21.

[0103] S42: Control the graphics engine to create a corresponding graphics engine instance according to the device identifier of the newly connected screen.

[0104] Continue to refer to Figure 2 The central processing unit 21 can control the graphics engine 22 to create a corresponding graphics engine instance, such as graphics engine instance 2, according to the device identifier of the newly connected screen.

[0105] S43: passing the identifier of the graphic engine instance to the created main interactive service engine instance, so that the main interactive service engine instance derives a slave interactive service engine instance corresponding to the new access screen.

[0106] In the specific implementation, refer to Figure 2 and Figure 3 The central processing unit 21 can directly pass the identifier of the graphic engine instance (such as the identifier of the graphic engine instance 2) to the created main interactive service engine instance (such as the interactive service engine instance 1), or pass it to the created main interactive service engine instance (such as the interactive service engine instance 1) through the message center 27. The main interactive service engine instance (interactive service engine instance 1) can derive a slave interactive service engine instance (such as the interactive service engine instance 2).

[0107] When the main interactive service engine instance derives a slave interactive service engine instance corresponding to the newly accessed screen, as an optional embodiment, the parameters of the slave interactive service engine instance can be set. By setting the parameters of the slave interactive service engine instance, a slave screen service operation graphical interface different from the main screen service operation can be implemented, further enhancing the personalization of multi-screen interactive control.

[0108] S44, recording the master-slave relationship between the master interactive service engine instance and the slave interactive service engine instance.

[0109] In the specific implementation, refer to Figure 2 and Figure 3 The central processing unit 21 or the message center 27 may record the master-slave relationship between the master interactive service engine instance and the slave interactive service engine instance. If there are multiple access screens, the master-slave relationship between one master interactive service engine instance and multiple slave interactive service engine instances is recorded.

[0110] S45, combining the core business data output by the core business engine, the interactive business data output by the interactive business engine and the graphic data output by the graphic engine, the interactive business engine instance corresponding to the new access screen controls the graphic engine instance corresponding to the new access screen to output the corresponding core business information, interactive business information and graphic elements to the new access screen.

[0111] Continue to refer to Figure 2 and Figure 3 The interactive service engine instance (such as interactive service engine instance 2) corresponding to the new access screen can combine the core service data output by the core service engine 23, the interactive service data output by the interactive service engine 24 and the graphic data output by the graphic engine 22 to control the graphic engine instance (such as graphic engine instance 2) corresponding to the new access screen to output the corresponding core service information, interactive service information and graphic elements to the new access screen, and display them on the display interface of the new access screen.

[0112] Accordingly, using the multi-screen interaction solution of the embodiment of this specification, when the connected screen is disconnected, the interactive service engine instance and map engine instance corresponding to the disconnected screen can be destroyed. The specific process of the multi-screen interaction control method can be the opposite of the process when the screen is newly connected. The following is a specific example, referring to Figure 5 The flowchart of another multi-screen interactive control method in the embodiment of this specification is shown. When the connected screen is disconnected, the following steps can be used to perform multi-screen interactive control:

[0113] S51, acquiring a device identifier of the disconnected screen according to a second callback message transmitted in response to the screen being disconnected.

[0114] In a specific implementation, when the operating system detects that a new screen is connected, it can transmit the second callback message to the multi-screen interactive control system through the interactive interface, and the device identifier of the disconnected screen can be transmitted to the central processing unit or the message center through the second callback message. Figure 2 When the screen interface 2 detects that the external screen is disconnected, the operating system may transmit the second callback message to the central processing unit 21 or the message center 27 .

[0115] S52: canceling the master-slave relationship between the slave interactive service engine instance and the master interactive service engine instance corresponding to the disconnected screen according to the device identifier of the disconnected screen.

[0116] In a specific implementation, the central processing unit 21 or the message center 27 can cancel the master-slave relationship between the slave interactive service engine instance and the master interactive service engine instance corresponding to the disconnected screen according to the device identifier of the disconnected screen, for example, cancel the master-slave relationship between the interactive service engine instance 2 and the main interactive service engine instance 1.

[0117] S53: The master interactive service engine instance destroys the slave interactive service engine instance corresponding to the disconnected screen.

[0118] Continue to refer to Figure 2 and Figure 3 After the central processing unit 21 or the message center 27 cancels the master-slave relationship between the interactive service engine instance 2 and the interactive service engine instance 1, the interactive service engine instance 1 can destroy the interactive service engine instance 2 corresponding to the disconnected screen.

[0119] S54, destroying the graphics engine instance corresponding to the disconnected screen.

[0120] Continue to refer to Figure 2 or Figure 3 , the central processing unit 21 can destroy the graphics engine instance 2 corresponding to the disconnected screen.

[0121] By adopting the above-mentioned embodiment of the present description, the corresponding map instance is created or destroyed based on the device identification of the corresponding screen obtained when the screen is connected or disconnected, and the corresponding slave interactive business engine instance is correspondingly created or destroyed by the main interactive business engine, so that the screen can be hot-swapped during the business execution process of the business application. The creation and destruction of the slave interactive business engine instance and the slave graphic engine instance during the business execution process will not affect the operation of the main interactive business engine instance and the main graphic engine instance, thereby reducing the complexity of multi-screen interaction and avoiding user distraction. When the multi-screen interactive control system runs on the vehicle-mounted device, by realizing the hot-swappable screen, the user can avoid performing complex operations for multi-screen interaction, thereby avoiding distraction of user attention and improving driving safety.

[0122] In the specific implementation process of the multi-screen interactive control system of the embodiment of this specification, it can be determined whether to pass the message to all interactive service engine instances or some interactive service engine instances according to the type of the received message. Specifically, the received global message can be passed to the main interactive service engine instance and synchronized to the slave interactive service engine instance, so that the main interactive service engine instance and the slave interactive service engine instance respectively perform corresponding processing based on the global message; the received non-global message is passed to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance performs corresponding processing based on the non-global message. Reference Figure 2 and Figure 3 Specifically, the central processing unit 21 or the message center may determine whether to transmit the message to all interactive service engine instances or some interactive service engine instances.

[0123] By adopting the above embodiment, it is determined whether to pass the message to all interactive service engines or some corresponding interactive service engine instances based on the received message type. On the one hand, each screen can meet the general requirements of the operation of the core business of the application. On the other hand, it can meet the personalized needs of multi-screen interactive users according to the different characteristics of each screen or the different characteristics of the corresponding users.

[0124] In order to enable those skilled in the art to better understand and implement, some examples that can be transmitted as global messages or non-global messages are given below.

[0125] Example 1: The core business update information transmitted by the core business engine can be respectively transmitted to the main interactive business engine instance and the slave interactive business engine instance, so that the main interactive business engine instance and the slave interactive business engine instance respectively control the corresponding graphics engine instance based on the received core business update information to update the corresponding graphic elements in the screen according to the core business update information.

[0126] By transmitting the core business engine update information transmitted by the core business engine as a global message, it is possible to synchronize update information between multiple screens and achieve synchronization and consistency of core business information between each screen.

[0127] In some embodiments of the present specification, the information transmitted by the core business engine can be transmitted as a global message to all interactive business engine instances to achieve synchronization of the core business information of each screen.

[0128] Example 2: Obtaining an instruction for switching the screen perspective transmitted in response to detecting the switching of the screen perspective; transmitting the instruction for switching the screen perspective to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance follows the instruction for switching the screen perspective and controls the corresponding interactive service engine instance to output corresponding graphic elements according to the corresponding graphic engine instance parameters after switching the screen perspective.

[0129] In addition to the core business data based on the core business engine, the settings including the user interaction interface display interface can be used as personalized settings for each screen. For example, different users may have different requirements for screen viewing angles, and some users may have different requirements for screen viewing angles. Therefore, when the instruction to switch the screen viewing angle is obtained, the instruction to switch the screen viewing angle is passed to the corresponding interactive business engine instance, and the corresponding interactive business engine instance is controlled to output the corresponding graphic elements according to the corresponding graphic engine instance parameters after the viewing angle is switched according to the instruction to switch the screen viewing angle, so that the users corresponding to each screen can set personalized screen viewing angles, thereby improving the user experience.

[0130] Example three: receiving an interactive service engine control instruction or a core service engine control instruction transmitted in response to an interactive control operation; transmitting the interactive service engine control instruction or the core service engine control instruction to the main interactive service engine instance, which in turn transmits the instruction to the interactive service engine or the core service engine; transmitting the status information transmitted by the interactive service engine or the core service engine to the corresponding interactive service engine instances respectively, and each interactive service engine instance controls the corresponding graphics engine instance to execute the processing operation corresponding to the interactive service control instruction or the core service engine control instruction.

[0131] In some embodiments of the present specification, it is set that only the main interactive service engine instance can control the interactive service engine or the core service engine. For example, the exit of the interactive service engine can be completed by the main interactive service engine instance. Therefore, the control instruction of the interactive service engine or the core service engine can be used as a non-global message and only transmitted to the main interactive service engine instance.

[0132] As another example, the interactive service engine can reset the interactive interface of the main screen without any impact on the connected slave screen. For example, based on the detected geographic location information, the interactive service engine can send a notification to the main interactive service engine instance to display a large image, and the main interactive service engine instance controls the main graphics engine to output the corresponding graphics engine instance, so that the enlarged image can be displayed on the interface of the main screen without any impact on the slave interactive service engine instance. The view angle and view ratio displayed on the slave screen remain unchanged. Here, the notification message sent by the interactive service engine is processed as a non-global message.

[0133] By passing the interactive business engine control instruction or the core business engine control instruction only to the main interactive business engine instance, and the main interactive business engine instance passes it to the interactive business engine or the core business engine accordingly, and the status message of the interactive business engine passed by the interactive business engine or the status information of the core business engine passed by the core business engine is passed to each interactive business engine instance respectively, each interactive business engine instance controls the corresponding map engine instance to execute the processing operation corresponding to the interactive business engine control instruction or the core business engine control instruction, that is, only the main interactive business engine instance can control the core business engine and the interactive business engine, which can avoid control conflicts or repeated issuance of control instructions of each interactive business engine instance, and further improve the consistency of business control of each screen.

[0134] The multi-screen interactive control solution of the embodiment of this specification can be applied to a multi-screen interactive control system in various scenarios, for example, it can be applied to different types of business applications such as map navigation applications, music playback applications, and video playback applications. Different business applications can determine the corresponding core business engine and interactive business engine according to their own business characteristics and business logic.

[0135] For example, for a music playback application, the music playback engine can be used as the core business engine, and the engines corresponding to other interactive services not related to music playback can be used as interactive business engines. For example, the display of playlists and lyrics can be used as interactive services, and the corresponding engines can be used as interactive business engines.

[0136] For another example, for a video playback application, the video playback engine may be used as the core service engine, and engines corresponding to other interactive services unrelated to video playback, such as display of a video-on-demand list, may be used as interactive service engines.

[0137] For another example, for a map navigation application, the graphics engine may include a map engine, which is suitable for controlling the corresponding map engine instance to output map elements based on map data; the core business engine may include a positioning engine, which is suitable for comprehensively calculating the user's current coordinates based on measurement data from a positioning sensor and the acquired route planning data to obtain positioning data; the interactive business engine may include a navigation engine, which may output user navigation guidance information based on the positioning data and route planning data output by the positioning engine; accordingly, the interactive business engine instance may include a travel engine instance, which may control the corresponding map engine instance in combination with the positioning data and user navigation guidance information to output the corresponding map elements and user navigation guidance information to the screen.

[0138] In order to enable those skilled in the art to better understand and implement, the specific application of the map navigation application scenario is described in detail below.

[0139] Figure 6 The structure diagram of a multi-screen interactive control system applied to a map navigation scenario in an embodiment of this specification is shown. Figure 6 The multi-screen interactive control system 60 may include a central processing unit 61, a map engine 62, a positioning engine 63, a navigation engine 64, a travel engine instance 65, a message center 66 and an interactive interface 67, wherein the interactive interface 67 includes a screen interface 671 and a user interactive interface 672, wherein:

[0140] The central processing unit 61 is adapted to create a positioning engine 63, a navigation engine 64, a message center 66 and a user interaction interface 672-1 (i.e., user interaction interface 1) of the main screen corresponding to the main screen when the map navigation application is started, and to create a corresponding main travel engine instance 65-1 (i.e., travel engine instance 1) when a navigation start instruction is obtained through the user interaction interface 672-1 of the main screen, and to create a corresponding map engine instance 1 in the corresponding user interaction interface 672-1 when a device identifier of a newly connected screen is obtained, and to pass the device identifier of the newly connected screen to the main travel engine instance 65-1;

[0141] A map engine 62, adapted to control a corresponding map engine instance to output map elements based on map data;

[0142] A positioning engine 63, adapted to comprehensively calculate the user's current coordinates based on the measurement data from the positioning sensor and the acquired route planning data to obtain positioning data;

[0143] A navigation engine 64, adapted to output user navigation guidance information according to the positioning data and route planning data output by the positioning engine 63;

[0144] The travel engine instance 65 is adapted to combine the positioning data, navigation guidance information and user settings to output the map and navigation guidance information to the corresponding user interaction interface, including outputting the map information to the corresponding map engine instance, and controlling the map engine instance of the corresponding screen to output the corresponding positioning data, navigation guidance information and map elements to the corresponding screen; the travel engine instance 65 may include a main travel engine instance 65-1 and a slave travel engine instance 65-2, and the main travel engine instance 65-1 is adapted to derive a slave travel engine instance 65-2 when obtaining the device identifier of the newly connected screen, specifically by creating a slave travel engine instance 65-2 and synchronizing its own status information to the slave travel engine instance 65-2;

[0145] The user interaction interface 672 corresponds to the screen one by one, and may include an input interface (not shown) and an output interface (not shown), wherein the input interface is suitable for receiving user operation instructions; the output interface is suitable for outputting a map interface and navigation information, and the output interface may include a map engine instance, and is suitable for outputting a map display interface based on the map setting information output by the corresponding travel engine instance, and outputting the navigation information transmitted by the corresponding travel engine instance;

[0146] The message center 66 communicates with the central processing unit 61, the positioning engine 63, the navigation engine 64 and each travel engine instance 65 respectively, is suitable for interacting with the central processing unit 61, the positioning engine 63, the navigation engine 64 and each travel engine instance 65 in a message distribution and subscription manner, and is suitable for recording the master-slave relationship between the master travel engine instance 65-1 and the slave travel engine instance 65-2.

[0147] In a specific implementation, the multi-screen map navigation control device can be run on an electronic device as a business application (APP), a client, a client applet or a browser. Figure 1 In the schematic diagram of the multi-screen interactive scene shown in the figure, the electronic device 1 may be provided with a screen 13, and may also be provided with an expansion interface 14, and the expansion interface 14 is suitable for accessing display screens 2 and 3, etc. The electronic device may be a vehicle-mounted device, a handheld device, a computer device, a home data center, a router, a set-top box, a projector, etc., wherein the vehicle-mounted device may be a central control terminal, the handheld device may be a mobile phone, and the display screen may include a central control display screen, a dashboard display screen, a rear seat display screen, a window display screen, a mobile phone display screen, a computer display screen, a TV display screen, a projection screen, etc. The embodiments of this specification do not impose any restrictions on the specific display screen, including the size, resolution, model, structure, material, etc. of the display screen.

[0148] Understandably, Figure 1 and Figure 6This is just an example. In a specific implementation, there may be multiple display screens connected at the same time, so there are correspondingly multiple slave travel engines and slave screen user interaction interfaces, and correspondingly multiple map engine instances. It is understandable that some user interaction interfaces may only have output interfaces but no input interfaces.

[0149] Reference Figure 7 The figure shows a processing flow diagram of a multi-screen interactive control system applied to a navigation scenario when a new screen is connected. The processing flow of the multi-screen interactive control system in the embodiment of this specification is described in detail through specific steps below.

[0150] S71, when the user connects a new screen to a mobile phone or a car device, the operating system will detect the new screen connection and can send a callback message to the central processing unit in the multi-screen interactive control system to call back the device identifier (deviceId) of the newly connected screen to the central processing unit.

[0151] In a specific implementation, when the screen is newly connected to an electronic device running a multi-screen interactive control system (such as a map navigation application) according to an embodiment of the present specification, if the map navigation application has been started, a positioning engine, a navigation engine, a message center, a user interaction interface of the main screen corresponding to the main screen, and a main travel engine instance will be created accordingly.

[0152] In a specific implementation, if a screen outside the electronic device is connected before the map navigation application is started, a map engine instance and a main travel engine instance corresponding to the main screen are first created during startup. After the startup is completed, the operating system can immediately detect the message that the external display screen has been connected, and call back the device identifier of the external connected screen to the central processing unit.

[0153] S72, the central processing unit creates a slave screen map engine instance according to the device identifier (deviceId) of the newly connected screen and obtains the engine identifier (engineId) of the map engine instance, and controls the map engine slave screen map engine instance to output the map to the newly connected screen.

[0154] S73, a message indicating that the map engine instance is successfully created is fed back from the screen map engine instance to the central processing unit.

[0155] S74, the central processing unit sends a message to the main travel engine instance to instruct the creation of a slave travel engine instance, which may include the engine identifier of the slave screen map engine instance, and passes the engine identifier of the slave screen map engine instance to the main travel engine instance.

[0156] S75, the master travel engine instance creates a slave travel engine instance corresponding to the newly connected screen based on the instruction of the central processing unit.

[0157] In a specific implementation, the master travel engine instance may create the slave travel engine instance by replicating itself.

[0158] S76, the master travel engine instance synchronizes its own state to the newly created slave travel engine instance.

[0159] In a specific implementation, the master travel engine instance may also set the created slave travel engine instance to be different from itself. Specifically, the master travel engine instance may set the parameters of the slave travel engine instance when creating the slave travel engine instance. For example, the main screen corresponding to the master travel engine instance may display three routes, including one main route and two alternative routes. The master travel engine instance may set the screen of the external display screen to display only one main route.

[0160] S77, the main travel engine instance feeds back a message of successful creation from the travel engine to the central processing unit.

[0161] S78, the master travel engine instance sends the master-slave relationship information between the master travel engine instance and the slave travel engine instance to the message center, and the message center records the master-slave relationship between the master travel engine instance and the slave travel engine instance.

[0162] In a specific implementation, it may happen that the user unplugs the external display screen during map navigation. For this application scenario, the multi-screen interactive control system of the embodiment of this specification can use the opposite process to that when one screen is connected for corresponding processing.

[0163] Specifically, refer to Figure 6 The central processing unit 61 is adapted to obtain the device identification of the disconnected screen according to the second callback message transmitted in response to the screen disconnection, and can transmit the screen disconnection message to the message center 66, instructing the message center to cancel the disconnected screen;

[0164] The message center 66 is further adapted to cancel the master-slave relationship between the slave travel engine instance 65-2 and the master travel engine instance 65-1 corresponding to the disconnected screen according to the device identifier of the disconnected screen;

[0165] The master travel engine instance 65-1 is further adapted to destroy the slave travel engine instance 65-2 corresponding to the disconnected screen when the message center 66 cancels the master-slave relationship between the slave travel engine instance 65-2 corresponding to the disconnected screen and the master travel engine instance 65-1;

[0166] The central processing unit 61 is further adapted to destroy the map engine instance (map engine instance 2) corresponding to the disconnected screen after the master travel engine instance 65-1 destroys the slave travel engine instance 65-2 corresponding to the disconnected screen.

[0167] The following reference Figure 8 , the specific processing flow of the multi-screen map navigation control device when an external display screen is disconnected in the embodiment of this specification is described in detail.

[0168] S81, when the operating system detects that the screen is disconnected, it transmits a second callback message to the central processing unit, and calls back the device identifier (deviceId) of the disconnected screen to the central processing unit.

[0169] S82, the central processing unit sends a message to the message center instructing to cancel the master-slave relationship corresponding to the disconnected screen, the message including the device identifier of the disconnected screen, and the message center cancels the master-slave relationship between the slave screen map engine instance and the master screen map engine instance corresponding to the disconnected screen.

[0170] S83: The central processing unit sends a message to the master travel engine instance to instruct the slave travel engine instance to be destroyed.

[0171] S84: The master travel engine instance destroys the slave travel engine instance corresponding to the disconnected screen based on the received message instructing to destroy the slave travel engine instance.

[0172] S85, the main travel engine feeds back to the central processing unit a message that the slave travel engine instance has been destroyed.

[0173] S86: The central processing unit destroys the slave screen map engine instance corresponding to the disconnected screen.

[0174] It can be seen from the above-mentioned screen connection and disconnection processing flow that the multi-screen interactive control system using the embodiment of this specification can support hot plugging of external screens during map navigation. The creation and destruction of slave travel engine instances and slave screen map instances during navigation will not affect the operation of the main travel engine instance and the main screen map instance, thereby reducing the complexity of multi-screen map navigation interaction and avoiding user distraction, thereby improving driving safety.

[0175] Reference Figure 6In a specific implementation, the message center 66 is suitable for determining whether to pass the message to all travel engine instances or synchronize it to only some travel engine instances according to the type of the received message. Specifically, the received global message can be passed to the main travel engine instance and synchronized to all slave travel engine instances, so that the main travel engine instance and all slave travel engine instances perform corresponding processing based on the global message respectively; and the received non-global message can be passed to the corresponding travel engine instance, so that the corresponding travel engine instance performs corresponding processing based on the non-global message.

[0176] Take the application scenario of location information update as an example, refer to Figure 6 The message center 66 is suitable for transmitting the positioning update information transmitted by the positioning engine 63 to the main travel engine instance 65-1 and the slave travel engine instance 65-2 respectively; the main travel engine instance 65-1 and the slave travel engine instance 65-2 respectively send information for resetting the screen coordinate center to the corresponding map engine instance 1 and map engine instance 2 based on the received positioning update information; when the map engine instance 1 and the map engine instance 2 receive the message for resetting the screen coordinate center, they update the screen coordinate center corresponding to the map display interface.

[0177] In the embodiments of this specification, positioning information, navigation information, etc. can be synchronized as global messages to the processing modules corresponding to each display screen, including travel engine instances and user interaction interfaces (map engine instances), etc.

[0178] Reference Fig. 9 The processing flow diagram of a multi-screen interactive control system applied to a map navigation scenario when updating positioning information is shown. The processing flow adopted by the multi-screen interactive control system of an embodiment of this specification when updating positioning information is introduced through specific steps below:

[0179] S91, the positioning engine sends a GPS coordinate update message to the message center.

[0180] S92, the message center sends a GPS coordinate update message to the main travel engine instance.

[0181] S93, the main travel engine sets the screen center coordinates of the main screen map engine instance based on the received arrival GPS coordinate update message, and updates the map center point.

[0182] S94, the message center sends a GPS coordinate update message to the travel engine instance.

[0183] S95, the slave travel engine instance sets the screen center coordinates of the corresponding slave screen map engine instance based on the received arrival GPS coordinate update message, and updates the map center point.

[0184] It should be noted that, in a specific implementation, the message center can synchronously execute steps S92 and S94, that is, synchronously transmit the GPS coordinate update message to the main travel engine instance and the slave travel engine instance, so that each travel engine instance resets its corresponding map center point to achieve synchronous update of the map on each screen.

[0185] Using the multi-screen interactive control system of the embodiment of this specification, in certain scenarios, the message center can only pass the message to the corresponding travel engine instance based on the type of the received message, rather than global synchronization, thereby meeting the different needs of users of each screen and realizing personalized settings for each screen.

[0186] In a specific implementation, setting information related to a specific display mode or screen, or personalized setting information that will not affect other users, can be transmitted only by the message center to the corresponding travel engine instance, thereby achieving differentiated display of each screen, thereby adapting to the characteristics of different screens and meeting the personalized needs of users in different locations or users with different preferences.

[0187] For example, continue to refer to Figure 6 The central processing unit 61 is suitable for transmitting the instruction of switching screen perspective transmitted by the input interface to the message center 66, and the message center 66 is suitable for transmitting the instruction of switching screen perspective to the corresponding travel engine instance 65 when receiving the instruction of switching screen perspective transmitted by the central processing unit 61; the travel engine instance 65 is suitable for updating the corresponding map engine instance to the map engine instance parameters corresponding to the switched screen perspective according to the instruction of switching screen perspective.

[0188] In order to enable those skilled in the art to better understand and implement the embodiments of this specification, the following reference is made to Fig.10 The processing flow diagram of the multi-screen map navigation control device of the embodiment of this specification when setting the personalized viewing angle is shown, and the processing flow of the multi-screen map navigation control device of the embodiment of this specification setting the personalized navigation viewing angle is described in detail through specific steps:

[0189] S101, a user clicks a screen viewing angle button displayed on an external screen displayed on a user interaction interface.

[0190] In a specific implementation, the user interaction interface may include an input interface and an output interface, wherein the input interface may include control buttons, input panels, etc. presented on the input interface. It can be understood that in a specific implementation, the input interface may also be a voice input interface, a touch panel, etc., and a corresponding voice trigger signal or trigger hardware for switching the screen perspective may be set based on the corresponding input form. The embodiments of this specification do not impose any restrictions on the specific instructions for switching the screen perspective, as long as an instruction for switching the screen perspective of the connected screen can be generated based on the user's corresponding trigger operation.

[0191] S102: The user interaction interface generates an instruction for switching the screen perspective of the access screen based on the operation of the user clicking the screen perspective button displayed on the external screen, and transmits the instruction to the central processing unit.

[0192] S103, the central processing unit transmits an instruction for switching the screen viewing angle of the access screen to the message center.

[0193] S104, the message center transmits the instruction of switching the screen perspective of the access screen to the corresponding slave travel engine instance.

[0194] S105, the slave travel engine instance sets the corresponding slave screen map engine instance to adopt map engine instance parameters corresponding to the switched screen viewing angle.

[0195] In a specific implementation, the corresponding map engine instance parameters may include one or more of the map center point (coordinates: longitude and latitude), vehicle position (coordinates: longitude and latitude), vehicle rotation angle, vehicle pitch angle, map rotation angle, map pitch angle, map level, etc., wherein the icon of the parking space presented on the map during the vehicle's navigation process.

[0196] In the above embodiment, the user clicks on the screen perspective of the access screen. It can be understood that if a user clicks on the screen perspective button presented on the main screen, the message center, after receiving the screen perspective switching instruction transmitted by the central processing unit, transmits the screen perspective switching instruction to the main travel engine instance. Accordingly, the main travel engine instance sets the corresponding main screen to adopt the map engine instance parameters corresponding to the switched screen perspective.

[0197] It can be seen from the above embodiment that when the message center receives the instruction to switch the screen perspective transmitted by the central processing unit, it transmits the instruction to switch the screen perspective to the corresponding travel engine instance, and the travel engine instance updates the corresponding map engine instance parameters after the screen perspective is switched according to the instruction to switch the screen perspective, so that users corresponding to each screen can set a personalized navigation perspective.

[0198] In the embodiment of this specification, in order to avoid control conflicts or repeated issuance of control instructions by each travel engine instance, only the main travel engine instance can control the positioning engine and the navigation engine. Figure 6 The central processing unit 61 is suitable for transmitting the navigation control instructions or positioning control instructions transmitted by the input interface to the message center 66. The message center 66 is suitable for transmitting the navigation control instructions or positioning control instructions to the main travel engine instance 65-1 when receiving the navigation control instructions or positioning control instructions transmitted by the central processing unit 61. The main travel engine instance 65-1 correspondingly transmits them to the navigation engine 64 or the positioning engine 63, and transmits the status message of the navigation engine 64 transmitted by the navigation engine 64 or the status information of the positioning engine 63 transmitted by the positioning engine 63 to each travel engine instance respectively, and each travel engine instance controls the corresponding map engine instance to execute the processing operation corresponding to the navigation control instruction or the positioning control instruction.

[0199] For example, when exiting navigation, in the multi-screen map control device of the embodiment of this specification, Figure 6 As shown, the central processing unit 61 is suitable for transmitting the exit navigation instruction transmitted by the input interface to the message center 66. When receiving the exit navigation instruction transmitted by the central processing unit 61, the message center 66 is suitable for transmitting the exit navigation instruction to the main travel engine instance 65-1, which is correspondingly transmitted to the navigation engine 64, and the status message of the navigation engine 64 received from the navigation engine 64 is transmitted to each travel engine instance (travel engine instance 65-2), and each travel engine instance (travel engine instances 65-1, 65-2) controls the corresponding map engine instance to perform the processing operation of exiting navigation.

[0200] Specifically, the message center 66 is suitable for transmitting messages of exiting navigation and exiting navigation to each travel engine instance (travel engine instances 65-1, 65-2) in sequence. When receiving the message of exiting navigation, each travel engine instance (travel engine instances 65-1, 65-2) sends a prompt instruction of displaying exiting navigation to the corresponding map engine instance, and when receiving the message of exiting navigation, instructs the corresponding map engine instance to clear the navigation elements on the map.

[0201] In order to enable those skilled in the art to better understand and implement the embodiments of this specification, the following reference is made to Fig.11 , through specific steps, the processing flow of a multi-screen interactive control system applied to map navigation scenarios when exiting navigation is described in detail:

[0202] S1101, the user clicks the exit navigation button of the user interaction interface.

[0203] S1102-S1103, the user interaction interface generates an exit navigation instruction based on the exit navigation button clicked by the user and transmits it to the message center through the central processing unit.

[0204] S1104: The message center continues to transmit the exit navigation instruction to the main travel engine instance.

[0205] S1105: The main travel engine instance transmits the instruction to exit navigation to the navigation engine.

[0206] S1106: After receiving the instruction to exit navigation, the navigation engine executes the operation to exit navigation and sends a status message of exiting navigation to the message center.

[0207] S1107: The message center transmits the state information of exiting navigation to the main travel engine instance.

[0208] S1108, the main travel engine instance instructs the corresponding main screen map engine instance to display a prompt message that the navigation is being exited.

[0209] S1109: The message center transmits the status information of exiting navigation to the slave travel engine instance.

[0210] S1110, the slave travel engine instance instructs the corresponding slave screen map engine instance to display a prompt message that it is exiting navigation.

[0211] In a specific implementation, after receiving the status information of exiting navigation from the navigation engine, the message center can simultaneously transmit the indication information of exiting navigation to each travel engine instance, and each travel engine instance instructs the corresponding map engine instance to display the prompt information of exiting navigation.

[0212] S1111, after exiting navigation, the navigation engine feeds back to the message center an indication of exiting navigation.

[0213] S1112: The message center transmits an indication message indicating that the navigation has been exited to the main travel engine instance.

[0214] S1113: The main travel engine instance controls the corresponding main screen map engine instance to clear the navigation elements on the map based on the received indication information of exiting the navigation.

[0215] S1114, the message center transmits an indication message indicating that the navigation has been exited to the travel engine instance.

[0216] S1115, the slave travel engine instance instructs the corresponding slave screen map engine instance to clear the navigation elements on the map based on the received indication information of exiting the navigation.

[0217] Likewise, step S1112 and step S1114 may be executed synchronously.

[0218] In order to enable those skilled in the art to better understand and implement the embodiments of this specification, the following reference is made to Fig.12 The specific interaction scenario of a multi-screen interactive control system applied to a map navigation scenario is described in detail. In a specific implementation, as mentioned above, the user interaction interface may include an input interface and an output interface, wherein the input interface may include a button (Button), a panel (Panel), etc., and the output interface may include a map engine instance, and may also include a label (Lable), etc.

[0219] In this way, the positioning engine can update the GPS coordinates in real time and pass them to the message center. The message center can synchronously pass the GPS coordinate update message to the main travel engine instance and the slave travel engine instance. The main travel engine instance and the slave travel engine instance can send the setting information of setting the map center point to the corresponding map engine instance, so that the map engine instance of the main screen user interaction interface and the map engine instance of the slave screen user interaction interface can both update the map center point in real time, so that the map keeps scrolling forward along the driving path. In addition, the main travel instance and the slave travel instance can also estimate the arrival time of the corresponding car from the destination based on the updated map engine instance, and can be displayed through the label of the user interaction interface. The label can be in text form or digital form.

[0220] When the user clicks the exit navigation button on the user interaction interface of a certain screen, the exit navigation instruction can be transmitted to the message center through the central processing center, and then transmitted to the navigation engine by the message center, so that the navigation engine exits the navigation, and the message center can synchronize the state information of the exit navigation process to each travel engine instance. The specific processing flow can be referred to Fig.11 The corresponding specific processing steps will not be repeated here.

[0221] As mentioned above, the input interface may be a button control interface, a text input interface, a voice input interface, etc., in addition to being a button control interface, to facilitate user interaction during driving.

[0222] The present specification also provides an electronic device, referring to Figure 1As shown in the structural schematic diagram of the electronic device in the multi-screen interaction scenario, the electronic device 1 may include: a memory 11, a processor 12 and an expansion interface and an expansion interface 14, the expansion interface 14 is suitable for coupling an external screen, the processor 11 stores computer instructions that can be executed on the processor 12, and the processor 12 can execute the multi-screen interaction control method described in any of the aforementioned embodiments when running the computer instructions. For details, please refer to the description of the aforementioned multi-screen interaction control method embodiment, which will not be repeated here.

[0223] In a specific implementation, the electronic device 1 may include multiple expansion interfaces 14 at the same time.

[0224] In a specific implementation, the expansion interface 14 can be a wired interface or a wireless interface, or some interfaces can be wired interfaces and some interfaces can be wireless interfaces, where the wired interface can be a USB interface, a screen projection interface, etc., and the wireless interface can be a WIFI wireless screen projection interface, etc.

[0225] The embodiment of this specification also provides another multi-screen interactive control system, such as Fig.13 As shown, the multi-screen interactive control system 130 may include: an interactive interface 131, a central processing unit 132, wherein:

[0226] The interactive interface 131 is adapted to receive a first callback message transmitted in response to a new screen access;

[0227] The central processing unit 132 is adapted to obtain the device identifier of the newly accessed screen based on the first callback message, control the graphics engine to create a corresponding graphics engine instance, and pass the graphics engine instance to the created main interactive service engine instance, so that the main interactive service engine instance derives a slave interactive service engine instance corresponding to the newly accessed screen and records the master-slave relationship between the main interactive service engine instance and the slave interactive service engine instance;

[0228] The slave interactive service engine instance corresponding to the new access screen is suitable for combining the core business data output by the core business engine, the interactive service data output by the interactive service engine and the graphic data output by the graphic engine, and controlling the graphic engine instance corresponding to the new access screen to output the corresponding core business information, interactive service information and graphic elements to the new access screen through the interactive interface of the new screen.

[0229] In a specific implementation, the above-mentioned multi-screen interactive system can be further expanded. The specific expansion method can be found in the introduction of the above-mentioned multi-screen interactive control system, which will not be repeated here.

[0230] Although the embodiments of this specification are disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, so the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A multi-screen interactive control method, wherein: include: According to the first callback message delivered in response to the newly connected screen, the device identifier v of the newly connected screen is obtained. According to the device identification of the newly connected screen, controlling the graphics engine to create a corresponding graphics engine instance; Passing the identifier of the graphics engine instance to the created master interactive service engine instance, so that the master interactive service engine instance derives a slave interactive service engine instance corresponding to the newly accessed screen; Recording the master-slave relationship between the master interactive service engine instance and the slave interactive service engine instance; In combination with the core business data output by the core business engine, the interactive business data output by the interactive business engine and the graphic data output by the graphic engine, the interactive business engine instance corresponding to the new access screen controls the graphic engine instance corresponding to the new access screen to output the corresponding core business information, interactive business information and graphic elements to the new access screen; Among them, the core business engine executes the core business logic of business applications unrelated to interaction and is suitable for outputting core business data; the interactive business engine executes the business logic related to interactive business and is suitable for outputting interactive business data; the graphics engine is suitable for outputting graphics data; and the main interactive business engine instance is created by the main interactive business engine.

2. The multi-screen interactive control method according to claim 1, wherein: Also includes: Acquire a device identifier of the disconnected screen according to a second callback message delivered in response to the screen being disconnected; According to the device identifier of the disconnected screen, canceling the master-slave relationship between the slave interactive service engine instance corresponding to the disconnected screen and the master interactive service engine instance; The master interactive service engine instance destroys the slave interactive service engine instance corresponding to the disconnected screen; Destroy the graphics engine instance corresponding to the disconnected screen.

3. The multi-screen interactive control method according to claim 1 or 2, wherein: Also includes: Determine whether to deliver the message to all or some interactive service engine instances based on the type of the received message, including: The received global message is delivered to the master interactive service engine instance, and synchronized to the slave interactive service engine instance, so that the master interactive service engine instance and the slave interactive service engine instance respectively perform corresponding processing based on the global message; The received non-global message is delivered to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance performs corresponding processing based on the non-global message.

4. The multi-screen interactive control method according to claim 3, wherein: The receiving global message is transmitted to the master interactive service engine instance and synchronized to the slave interactive service engine instance, so that the master interactive service engine instance and the slave interactive service engine instance respectively perform corresponding processing based on the global message, including: The core business update information transmitted by the core business engine is respectively transmitted to the main interactive business engine instance and the slave interactive business engine instance, so that the main interactive business engine instance and the slave interactive business engine instance respectively control the corresponding graphics engine instance based on the received core business update information to update the corresponding graphic elements in the screen according to the core business update information.

5. The multi-screen interactive control method according to claim 3, wherein: The step of delivering the received non-global message to the corresponding interactive service engine instance so that the corresponding interactive service engine instance performs corresponding processing based on the non-global message includes: acquiring an instruction for switching the screen perspective transmitted in response to detecting that the screen perspective is switched; The instruction for switching the screen perspective is passed to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance controls the corresponding interactive service engine instance to output the corresponding graphic elements according to the corresponding graphic engine instance parameters after switching the screen perspective according to the instruction for switching the screen perspective.

6. The multi-screen interactive control method according to claim 3, wherein: The step of delivering the received non-global message to the corresponding interactive service engine instance so that the corresponding service engine instance performs corresponding processing based on the non-global message includes: receiving an interactive service engine control instruction or a core service engine control instruction transmitted in response to an interactive control operation; The interactive service engine control instruction or the core service engine control instruction is transmitted to the main interactive service engine instance, and the main interactive service engine instance transmits the instruction to the interactive service engine or the core service engine accordingly; The status information transmitted by the interactive service engine or the core service engine is transmitted to the corresponding interactive service engine instance respectively, and each interactive service engine instance controls the corresponding graphic engine instance to execute the processing operation corresponding to the interactive service control instruction or the core service engine control instruction.

7. The multi-screen interactive control method according to claim 1, wherein: When the master interactive service engine instance derives a slave interactive service engine instance corresponding to the new access screen, it also includes: The master interactive service engine instance sets parameters of the slave interactive service engine instance.

8. The multi-screen interactive control method according to any one of claims 1, 2, or 4 to 7, wherein: The graphics engine includes: a map engine adapted to control a corresponding map engine instance to output map elements based on map data; The core service engine includes a positioning engine, which is adapted to comprehensively calculate the current coordinates of the user based on the measurement data from the positioning sensor and the acquired route planning data to obtain positioning data; The interactive service engine includes a navigation engine, and the navigation engine is adapted to output user navigation guidance information according to the positioning data and route planning data output by the positioning engine; The interactive service engine instance includes a travel engine instance, and the travel engine instance is suitable for combining positioning data and user navigation guidance information to control the corresponding map engine instance to output the corresponding map elements and user navigation guidance information to the screen.

9. A multi-screen interactive control system, wherein: include: An interactive interface, adapted to receive a first callback message transmitted in response to a new access to a screen; The central processing unit is adapted to obtain a device identifier of the newly accessed screen based on the first callback message, control the graphics engine to create a corresponding graphics engine instance, and pass the graphics engine instance to the created master interactive service engine instance, so that the master interactive service engine instance derives a slave interactive service engine instance corresponding to the newly accessed screen and records the master-slave relationship between the master interactive service engine instance and the slave interactive service engine instance; The slave interactive service engine instance corresponding to the new access screen is adapted to combine the core service data output by the core service engine, the interactive service data output by the interactive service engine and the graphic data output by the graphic engine, and control the graphic engine instance corresponding to the new access screen to output the corresponding core service information, interactive service information and graphic elements to the new access screen through the interactive interface of the new screen; Among them, the core business engine executes the core business logic of business applications unrelated to interaction and is suitable for outputting core business data; the interactive business engine executes the business logic related to interactive business and is suitable for outputting interactive business data; the graphics engine is suitable for outputting graphics data; and the main interactive business engine instance is created by the main interactive business engine.

10. A multi-screen interactive control system, wherein: include: A central processing unit, adapted to create a graphics engine, a core business engine, an interactive business engine, a message center and a user interaction interface when a business application is started, and to create a corresponding main interactive business engine instance based on the interactive business engine when an interactive business start instruction is obtained through the user interaction interface of the main screen; According to the first callback message transmitted in response to the newly connected screen, the device identification of the newly connected screen is obtained, and according to the device identification of the newly connected screen, the graphics engine is controlled to create a corresponding graphics engine instance, and the identification of the graphics engine instance is transmitted to the main interactive service engine instance; The core business engine executes the core business logic of business applications that are not related to interaction and is suitable for outputting core business data; The interactive service engine executes the service logic related to the interactive service and is suitable for outputting the interactive service data; A graphics engine, suitable for outputting graphics data; The interactive service engine instance corresponds to the screen one by one, and is adapted to combine the core service data output by the core service engine, the interactive service data output by the interactive service engine, and the graphic data output by the graphic engine, and control the graphic engine instance of the corresponding screen to output the corresponding core service information, interactive service information, and graphic elements to the corresponding screen; The interactive service engine instance includes a main interactive service engine instance and a slave interactive service engine instance, wherein the main interactive service engine instance is adapted to derive a slave interactive service engine instance corresponding to the newly accessed screen according to an identifier of the received graphics engine instance corresponding to the newly accessed screen; The message center is suitable for communicating with the central processing unit, the core business engine, the interactive business engine and each interactive business engine instance by using a message distribution and subscription method, and recording the master-slave relationship between the master interactive business engine instance and the slave interactive business engine instance.

11. The multi-screen interactive control system according to claim 10, wherein: The central processing unit is further adapted to obtain a device identifier of the disconnected screen according to a second callback message transmitted in response to the screen being disconnected, and instruct the message center to cancel the master-slave relationship between the slave interactive service engine instance corresponding to the disconnected screen and the master interactive service engine instance, instruct the master interactive service engine instance to destroy the slave interactive service engine instance corresponding to the disconnected screen, and destroy the graphics engine instance corresponding to the disconnected screen; The message center is adapted to cancel the master-slave relationship between the slave interactive service engine instance and the master interactive service engine instance corresponding to the disconnected screen according to the device identifier of the disconnected screen.

12. The multi-screen interactive control system according to claim 10 or 11, wherein: The message center is also suitable for determining whether to pass the message to all interactive service engine instances or some interactive service engine instances according to the received message type, including: passing the received global message to the main interactive service engine instance, and synchronizing it to the slave interactive service engine instance, so that the main interactive service engine instance and the slave interactive service engine instance respectively perform corresponding processing based on the global message, and passing the received non-global message to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance performs corresponding processing based on the non-global message.

13. The multi-screen interactive control system according to claim 12, wherein: The message center is suitable for transmitting the core business update information transmitted by the core business engine to the main interactive business engine instance and the slave interactive business engine instance respectively, so that the main interactive business engine instance and the slave interactive business engine instance update the corresponding graphic elements in the screen according to the core business update information.

14. The multi-screen interactive control system according to claim 12, wherein: The central processing unit is adapted to obtain an instruction for switching the screen viewing angle transmitted in response to detecting the switching of the screen viewing angle, and transmit the instruction to the message center; The message center is also suitable for transmitting the instruction of switching the screen perspective to the corresponding interactive service engine instance, so that the corresponding interactive service engine instance controls the corresponding graphics engine instance to output graphic elements according to the corresponding graphics engine instance parameters after the screen perspective is switched according to the instruction of switching the screen perspective.

15. The multi-screen interactive control system according to claim 12, wherein: The central processing unit is adapted to receive an interactive service engine control instruction or a core service engine control instruction transmitted in response to an interactive control operation and transmit the instruction to the message center; The message center is adapted to transmit the interactive service engine control instruction or the core service engine control instruction to the master interactive service engine instance, and the master interactive service engine instance transmits the instruction to the corresponding slave interactive service engine or core service engine; And the status information transmitted by the interactive service engine or the core service engine is respectively transmitted to the corresponding interactive service engine instance, and each interactive service engine instance controls the corresponding graphics engine instance to execute the processing operation corresponding to the interactive service control instruction or the core service engine control instruction.

16. The multi-screen interactive control system according to claim 10, wherein: The master interactive service engine instance also sets the parameters of the slave interactive service engine instance when deriving a slave interactive service engine instance corresponding to the new access screen.

17. The multi-screen interactive control system according to claim 10, 11 or any one of claims 13 to 16, wherein: The graphics engine comprises: A map engine, adapted to control a corresponding map engine instance to output map elements based on map data; The core service engine includes a positioning engine, which is adapted to comprehensively calculate the current coordinates of the user based on the measurement data from the positioning sensor and the acquired route planning data to obtain positioning data; The interactive service engine includes a navigation engine, and the navigation engine is adapted to output user navigation guidance information according to the positioning data and route planning data output by the positioning engine; The interactive service engine instance includes a travel engine instance, and the travel engine instance is suitable for combining positioning data and user navigation guidance information to control the corresponding map engine instance to output the corresponding map elements and user navigation guidance information to the screen.

18. An electronic device, wherein: include: A memory, a processor and an expansion interface, wherein the expansion interface is suitable for coupling an external screen, the memory stores computer instructions that can be executed on the processor, and the processor is suitable for executing the multi-screen interactive control method described in any one of claims 1 to 7 when running the computer instructions.

19. The electronic device according to claim 18, wherein: The extended interface includes at least one of the following: Wired interface; wireless interface.

Images