Screen display window circulation method and system, vehicle, equipment and medium

Abstract

The invention relates to a screen display window circulation method and system, a vehicle, equipment and a medium, and the method comprises the steps: a data channel corresponding to screen projection data is arranged between circulation receiving equipment and screen projection source equipment; a data channel corresponding to the projection screen data is managed and controlled by the circulation control equipment, and any circulation receiving equipment does not directly establish a data channel with the projection screen source equipment but is guided to be established by the circulation control equipment. The circulation control equipment obtains connection information of a data channel corresponding to the screen projection data from the screen projection source equipment; and controlling whether a data channel is established between the circulation receiving equipment and the projection screen source equipment or not by controlling the connection information and performing directional sending control on the receiving object. Flexible circulation of the screen display window can be realized in a cross-device manner, the applicability is wide, and the expandability is high; and the computing resource consumption of the circulation control equipment is effectively reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle and multi-screen interaction technology, and in particular to a method, system, vehicle, device and medium for screen display window flow. Background Art

[0002] With the development of screen projection technology and cross-screen display technology, many demands and scenarios for multi-screen interaction have emerged. How to achieve stable transmission across devices and different operating systems, controlled window transfer and other multi-screen interactions has also become a technical problem faced by smart home devices, vehicles, etc.

[0003] Taking a usage scenario of window flow and multi-screen interaction in a vehicle as an example, display screens are set up in the main driver's seat, the co-pilot seat and the back row of the vehicle, among which the display screen at the main driver's seat is the main screen, and the display screens of the co-pilot and the rear license plate are both sub-screens; the terminal device used by the co-pilot user in the vehicle projects a front-end display window to the main screen and hopes to direct the flow of this content to the display screen in the back row without displaying it on the display screen at the co-pilot seat; because for the vehicle, the display screens at the co-pilot seat and the back row of the vehicle are both sub-screens, then when the screen display window flows, the problem of how to stably realize multi-screen interaction and controlled window flow is faced.

[0004] In the related technologies, in the solutions for multi-screen interaction and display window transfer, most of the multiple devices that can realize the transfer of projection display windows belong to the same brand system, that is, these devices need to have a common account management system, the same or display-compatible operating system, etc., in order to achieve the synchronization of projection data and support cross-screen display data, which has limitations. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a method, system, vehicle, device and medium for screen display window flow.

[0006] According to the first aspect of the embodiments of the present disclosure, a method for screen display window flow is provided. The method includes: there is a data channel corresponding to the screen projection data between the flow receiving device and the screen projection source device; wherein the data channel corresponding to the screen projection data is controlled by the flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the flow control device.

[0007] In some embodiments, the flow control device guides the establishment of a data channel in the following manner: after the flow control device receives the flow control instruction for the above-mentioned screen projection data, it generates a screen projection flow instruction according to the above-mentioned flow control instruction and sends it to the corresponding flow receiving device in a directed manner; wherein the above-mentioned screen projection flow instruction carries the connection information of the data channel corresponding to the screen projection data; the above-mentioned connection information is held by the above-mentioned flow control device and performs directed sending control on the receiving object; the flow receiving device that receives the above-mentioned screen projection flow instruction establishes a data channel with the corresponding screen projection source device.

[0008] In some embodiments, the connection information includes: the target projection data subject information corresponding to the projection data to be transferred. The transfer receiving device that receives the projection transfer instruction determines the transmission channel of the projection data to be transferred according to the target projection data subject information, and subscribes to the corresponding subject by connecting to the transmission channel to establish a data channel with the corresponding projection source device.

[0009] In some embodiments, the data channel transmits the projection data based on a publish-subscribe mechanism. The projection source device acts as the publisher of the topic corresponding to the projection data. After the flow receiving device is guided by the flow control device to establish a data channel with the corresponding projection source device, the flow receiving device acts as the subscriber of the topic corresponding to the projection data to receive the projection data from the projection source device. The flow control device obtains the connection information of the data channel corresponding to the projection data from the projection source device, and the acquisition method includes: the projection source device acts as the publisher of the connection information, and the flow control device acts as the subscriber of the connection information to obtain the connection information.

[0010] In some embodiments, the flow control device obtains connection information of a data channel corresponding to the screen projection data from the screen projection source device; controls whether a data channel is established between the flow receiving device and the screen projection source device by controlling the above connection information and performing directional sending control on the receiving object;

[0011] The above method also includes one or more of the following:

[0012] The streaming receiving device receives the screen projection data based on the data channel; or,

[0013] The above-mentioned streaming receiving device receives the above-mentioned screen projection data based on the above-mentioned data channel, and displays the above-mentioned screen projection data on the display interface of the above-mentioned streaming receiving device; or,

[0014] The above-mentioned flow control device establishes a data channel with the screen projection source device according to the above-mentioned connection information; or,

[0015] The flow control device establishes a data channel with the screen projection source device according to the connection information, and receives the screen projection data of the screen projection source device based on the data channel; or,

[0016] The above-mentioned flow control device establishes a data channel with the screen projection source device according to the above-mentioned connection information, receives the screen projection data of the above-mentioned screen projection source device based on the above-mentioned data channel, and displays the above-mentioned screen projection data on the display interface of the above-mentioned flow control device.

[0017] In some embodiments, the screen projection data is one or more; displaying the screen projection data on the display interface of the stream receiving device includes: displaying the one or more screen projection data on the display interface of the stream receiving device based on first preset display configuration information; the first preset display configuration information is information pre-configured by the stream receiving device, including: a first window layout and a first display style of one or more display windows corresponding to the number of the screen projection data;

[0018] Displaying the above-mentioned screen projection data on the display interface of the above-mentioned flow control device includes:

[0019] For one or more projection data, the display interface of the above-mentioned flow control device is displayed based on the second preset display configuration information; the above-mentioned second preset display configuration information is information pre-configured by the above-mentioned flow control device, including: a second window layout and a second display style of one or more display windows corresponding to the number of the above-mentioned projection data.

[0020] In some embodiments, the above-mentioned flow control instructions include instructions in at least one of the following forms: voice instructions, touch screen control instructions, and human motion visual perception instructions; among them, human motion visual perception instructions include perception of at least one of the following types of actions: gestures, head movements, eye movements, and facial expressions.

[0021] In some embodiments, a screen projection flow instruction is generated according to the above-mentioned flow control instruction and sent to the corresponding flow receiving device in a directed manner, including: in response to detecting a target flow control instruction for controlling the flow of projection data corresponding to the current window in the target window of the above-mentioned flow control device, determining the target flow direction matching the above-mentioned target flow control instruction according to the mapping relationship between the preset operation control instruction and the flow direction; determining the target claiming role corresponding to the above-mentioned target flow direction according to the pre-configured claiming roles corresponding to each flow direction; determining the electronic device under the above-mentioned target claiming role as the target flow receiving device; determining the screen projection source device corresponding to the projection data of the above-mentioned target window and the target projection data subject information of the above-mentioned screen projection source device for the projection data to be circulated during the data distribution process; generating the screen projection flow instruction corresponding to the above-mentioned target flow control instruction according to the above-mentioned target projection data subject information; and sending the above-mentioned screen projection flow instruction in a directed manner to the above-mentioned target flow receiving device.

[0022] In some embodiments, the above-mentioned target flow control instruction is at least one of the following instructions: a first control instruction in the first target window used to control the projection interface in the current window to perform projection flow; or, a second control instruction in the second target window used to control the built-in application interface in the current window to perform projection flow; or, a third control instruction in the third target window used to control the flow of projection data received corresponding to the current window; or, a fourth control instruction in the fourth target window used to control the flow of projection data corresponding to the data channel connected to the current window.

[0023] In some embodiments, the method of pre-configuring the claiming role corresponding to each flow direction is as follows: according to the spatial positions of multiple electronic devices of the role to be claimed, each flow direction is automatically matched with the above spatial positions to obtain a matching result; according to the above matching result, a claiming role corresponding to the matched flow direction is configured for each electronic device; or, user claiming information for each flow direction of multiple electronic devices of the role to be claimed is received; according to the above user claiming information, a claiming role corresponding to the claimed flow direction is configured for each electronic device.

[0024] In some embodiments, after the screen projection flow instruction is sent to the corresponding flow receiving device, the method further includes: the flow control device receives an interface control instruction for minimizing or closing the screen projection interface in the first window; wherein the screen projection interface is used to present the projection data; according to the interface control instruction, a minimization or closing operation is performed on the projection interface in the first window on the display interface of the flow control device; wherein the minimization or closing operation does not affect the flow display of the corresponding projection data by the flow receiving device that has received the corresponding projection data of the first window.

[0025] In some embodiments, the method further includes: the flow control device receives interface operation instructions for interacting with the projection interface in the second window; sends the interface operation instructions to the target data source device corresponding to the projection data of the second window; enables the target data source device to respond and process based on the interface operation instructions to obtain the projection data after interaction; receives the projection data after interaction fed back from the target data source device, and updates and displays it in the second window of the display interface of the flow control device.

[0026] In some embodiments, the above method also includes: the flow control device receives response information fed back by the flow receiving device to the above screen projection flow instruction; the above response information includes: whether the flow receiving device agrees to perform screen projection, and whether to establish a corresponding screen projection flow data channel; according to the above response information, the flow control device updates the subject disclosure status list accordingly; wherein the above subject disclosure status list is used to maintain the following information: the data distribution subject, the data distribution correspondence between the screen projection source device from which the subject comes and the flow receiving device to which it is directed, and the link status of whether the screen projection flow data channel between the flow receiving device to which it is directed and the screen projection source device is opened; the flow control device receives a status update instruction of the disconnection of the data channel connection sent by the flow receiving device when exiting the screen projection state; the flow control device updates the above subject disclosure status list according to the above status update instruction.

[0027] In some embodiments, the flow control device maintains an online device list; the online device list is used to indicate electronic devices that have established a communication link with the flow control device and are in an online state, and the electronic devices in the online device list serve as options for flow receiving devices.

[0028] In some embodiments, screen projection data is transmitted between the screen projection source device and the flow control device based on mirror projection or application projection, and screen projection data is transmitted between the screen projection source device and the flow receiving device based on mirror projection or application projection.

[0029] In some embodiments, the flow control device, the screen projection source device, and the flow receiving device are located in the same target space; the flow control device, the screen projection source device, and the flow receiving device are all installed with an interconnection application and perform data exchange based on a preset interconnection protocol. After the interconnection application is started, each device located in the target space configures each flow direction to point to the corresponding claiming role, and establishes a corresponding communication link according to the claiming role; wherein the flow direction points to multiple logical directions pre-given by the interconnection application.

[0030] In some embodiments, the communication link includes: a first communication link, a second communication link and a third communication link. The first communication link is used to connect the screen projection source device and the flow control device, and the first communication link is used to transmit the subject information corresponding to the screen projection data to be sent by the screen projection source device to the flow control device for screen projection display; wherein the screen projection source device serves as the publisher of the subject information, and the flow control device serves as the subscriber of the subject information. The second communication link is used to connect the flow control device and the flow receiving device in a directional manner, and the data transmission direction is from the flow control device to the flow receiving device, and the second communication link is used to transmit the subject information corresponding to the screen projection data to be circulated. The third communication link is used to connect the flow receiving device and the flow control device in a directional manner, and the data transmission direction is from the flow receiving device to the flow control device, and the third communication link is used to transmit at least one of the following information: event confirmation information, response information to the screen projection flow instruction. The target space is one of the following: vehicle, aircraft cabin, room.

[0031] In some embodiments, the above-mentioned screen projection source device includes one of the following situations: multiple applications within a device are respectively used as the projection source of the above-mentioned screen projection data; or, the display interfaces corresponding to multiple devices are respectively used as the projection source of the above-mentioned screen projection data; or, part of the above-mentioned screen projection data comes from one or more applications within a device, and the other part of the screen projection data comes from the display interface corresponding to at least one device.

[0032] In some embodiments, the screen projection source device and the flow receiving device are located in the same entity, which is one of the following situations: while the screen projection data of the first topic published by the screen projection source device is displayed on the screen by the flow control device, the screen projection source device also serves as a flow receiving device for receiving the second topic sent from the flow control device, establishing a data channel between the screen projection source device corresponding to the second topic, and performing flow display of the corresponding screen projection data according to a preset display strategy; wherein the first topic and the second topic are different topics. Or, another situation as follows: while the flow receiving device receives the third topic sent from the flow control device, establishes a data channel between the screen projection source device corresponding to the third topic, and performs flow display of the corresponding screen projection data, the flow receiving device also serves as a screen projection source device for publishing the screen projection data of the third topic or the fourth topic, and the screen projection is displayed by the flow control device; wherein the third topic and the fourth topic are different topics.

[0033] According to the second aspect of the embodiment of the present disclosure, a method for screen flow is provided. The above method is applied to a flow receiving device, including: receiving a screen projection flow instruction sent by a flow control device; the above screen projection flow instruction carries the connection information of the data channel corresponding to the screen projection data; the above connection information is held by the above flow control device and performs directional sending control on the receiving object; according to the above screen projection flow instruction, a data channel is established with the corresponding screen projection source device; wherein the data channel corresponding to the above screen projection data is controlled by the above flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above flow control device.

[0034] In some embodiments, the connection information includes: the target projection data subject information corresponding to the projection data to be transferred. The transfer receiving device that receives the projection transfer instruction determines the transmission channel of the projection data to be transferred according to the target projection data subject information, and subscribes to the corresponding subject by connecting to the transmission channel to establish a data channel with the corresponding projection source device.

[0035] In some embodiments, after receiving the screen projection flow instruction sent by the flow control device, the method further includes: presenting a prompt confirmation message on the display interface of the flow receiving device whether to agree to the screen projection; in response to the user choosing to agree to the screen projection in response to the prompt confirmation message, executing the following steps: according to the screen projection flow instruction, establishing a data channel with the corresponding screen projection source device; and sending a first response message to the screen projection flow instruction to the flow control device, the first response message including feedback content that the screen projection is agreed and the corresponding screen projection flow data channel has been established; so that the flow control device updates the subject disclosure status list according to the first response message. The method further includes: in response to the user choosing not to agree to the screen projection in response to the prompt confirmation message, sending a second response message to the screen projection flow instruction to the flow control device, the second response message including feedback content that the screen projection is not agreed and the corresponding screen projection flow data channel has not been established. The method further includes: receiving the corresponding screen projection data based on the data channel, and performing flow display based on the screen projection data. Among them, in the process of streaming display based on the above-mentioned screen projection data, the above-mentioned method also includes: receiving a screen projection end control instruction for exiting the screen projection interface; in response to the above-mentioned screen projection end control instruction, disconnecting the data channel between the corresponding screen projection source device and exiting the screen projection state; and sending a status update instruction of the disconnection of the data channel connection to the above-mentioned streaming control device; so that the above-mentioned streaming control device updates the above-mentioned subject disclosure status list according to the above-mentioned status update instruction.

[0036] In some embodiments, the screen projection data obtained by the flow receiving device based on the above-mentioned data channel is one of the following situations: the above-mentioned screen projection source device and the above-mentioned flow control device are different devices, and the above-mentioned screen projection data is the screen projection data from the screen projection source device; or, the above-mentioned screen projection source device and the above-mentioned flow control device are the same device, and the above-mentioned screen projection data is data from the built-in application interface of the flow control device.

[0037] In some embodiments, the above method also includes: receiving interface operation instructions for interacting with the displayed target projection interface; sending the above interface operation instructions to the target data source device corresponding to the projection data of the above target projection interface; enabling the above target data source device to respond based on the above interface operation instructions to obtain the projection data after interaction; receiving the projection data after interaction fed back from the above target data source device, and updating and displaying it on the above target projection interface.

[0038] According to the third aspect of the embodiment of the present disclosure, a screen flow system is provided. The above-mentioned screen flow system includes: a screen projection source device, a flow control device and a flow receiving device. The above-mentioned flow control device is used to: obtain the connection information of the data channel corresponding to the screen projection data in the above-mentioned screen projection source device; detect the flow control instruction for the above-mentioned screen projection data; generate the screen projection flow instruction according to the above-mentioned flow control instruction and send it to the corresponding flow receiving device in a directional manner; wherein, the data channel corresponding to the above-mentioned screen projection data is controlled by the above-mentioned flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above-mentioned flow control device; the above-mentioned screen projection flow instruction carries the connection information of the data channel corresponding to the above-mentioned screen projection data; the above-mentioned connection information is held by the above-mentioned flow control device and performs directional sending control of the receiving object. The above-mentioned flow receiving device is used to: receive the above-mentioned screen projection flow instruction and establish a data channel with the corresponding screen projection source device according to the above-mentioned screen projection flow instruction.

[0039] In some embodiments, in the above system, the above streaming receiving device is further used for:

[0040] Receive the above-mentioned screen projection data based on the above-mentioned data channel; or,

[0041] The above-mentioned screen projection data is received based on the above-mentioned data channel, and the above-mentioned screen projection data is displayed on the display interface of the above-mentioned flow receiving device.

[0042] In some embodiments, in the above system, the above flow control device is also used for:

[0043] Establish a data channel with the screen projection source device according to the above connection information; or

[0044] Establishing a data channel with the screen projection source device according to the connection information, and receiving the screen projection data of the screen projection source device based on the data channel; or

[0045] A data channel is established with the screen projection source device according to the above connection information, the screen projection data of the above screen projection source device is received based on the above data channel, and the above screen projection data is displayed on the display interface of the above flow control device.

[0046] According to the fourth aspect of the embodiment of the present disclosure, a vehicle is provided. The above-mentioned vehicle includes: at least one terminal device and multiple vehicle-mounted devices; the above-mentioned vehicle-mounted device is an electronic device based on the vehicle-mounted system and has a vehicle-mounted display screen, and the above-mentioned terminal device is a movable device or a wearable device with a display screen; the above-mentioned terminal device and the above-mentioned vehicle-mounted device are both located in the vehicle space; wherein the above-mentioned terminal device and the above-mentioned vehicle-mounted device share the following three roles: screen projection source device, flow control device and flow receiving device. The above-mentioned flow control device is used to: obtain the connection information of the data channel corresponding to the screen projection data in the above-mentioned screen projection source device; detect the flow control instruction for the above-mentioned screen projection data; generate the screen projection flow instruction according to the above-mentioned flow control instruction and send it to the corresponding flow receiving device in a directional manner; wherein the data channel corresponding to the above-mentioned screen projection data is controlled by the above-mentioned flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above-mentioned flow control device; the above-mentioned screen projection flow instruction carries the connection information of the data channel corresponding to the above-mentioned screen projection data; the above-mentioned connection information is held by the above-mentioned flow control device and performs directional sending control of the receiving object. The above-mentioned flow receiving device is used to: receive the above-mentioned screen projection flow instruction and establish a data channel with the corresponding screen projection source device according to the above-mentioned screen projection flow instruction.

[0047] In some embodiments, in the above-mentioned vehicle, the above-mentioned flow receiving device is also used for:

[0048] Receive the above-mentioned screen projection data based on the above-mentioned data channel; or,

[0049] The above-mentioned screen projection data is received based on the above-mentioned data channel, and the above-mentioned screen projection data is displayed on the display interface of the above-mentioned flow receiving device.

[0050] In some embodiments, in the above system, the above flow control device is also used for:

[0051] Establish a data channel with the screen projection source device according to the above connection information; or

[0052] Establishing a data channel with the screen projection source device according to the connection information, and receiving the screen projection data of the screen projection source device based on the data channel; or

[0053] A data channel is established with the screen projection source device according to the connection information, the screen projection data of the screen projection source device is received based on the data channel, and the screen projection data is displayed on the display interface of the flow control device.

[0054] According to the fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the above-mentioned processor; the above-mentioned processor is used to read the above-mentioned executable instructions from the above-mentioned memory, and execute the above-mentioned instructions to implement the method of screen display window flow provided by the first aspect or the second aspect of the present disclosure.

[0055] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for screen display window flow provided in the first aspect or the second aspect are implemented.

[0056] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0057] The flow control device controls the data channel corresponding to the screen projection data so that the data channel corresponding to the screen projection data cannot be directly established between any flow receiving device and the screen projection source device, but is guided to be established through the above-mentioned flow control device. The main role of the flow control device is to guide the specific flow receiving device to establish a data channel with the screen projection source device by controlling the data channel corresponding to the screen projection data. For the flow control device, there is no need to transfer the screen projection data, but to control which data channel connection information of the screen projection data is disclosed to which flow receiving devices, thereby realizing the control of the directional connection between the flow receiving device and the screen projection source device. In this way, regardless of whether the operating systems of the screen projection source device, the flow control device and the flow receiving device are the same, and whether these three types of devices are in the same account system, the flow of the screen display window can be realized, and the applicability is wide; and because the flow control device does not need to transfer the screen projection data, the computing resource consumption of the flow control device is effectively reduced. The flow control device can guide one or more newly added flow receiving devices to establish a data channel with the screen projection source device through directional control. The flow receiving device is easy to access and manage as a display terminal device, and has strong scalability.

[0058] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0060] Figure 1 The figure is a structural block diagram of a screen flow system according to an exemplary embodiment.

[0061] Figure 2 It is a schematic diagram of a communication link established between a screen projection source device, a flow control device and a flow receiving device in a screen flow system according to an exemplary embodiment.

[0062] Figure 3 It is a schematic diagram of a display interface for displaying screen projection data by a flow control device according to an exemplary embodiment.

[0063] Figure 4 It is a schematic diagram of a process of determining a target flow direction pointing that matches a target flow control instruction according to a mapping relationship between a preset operation control instruction and a flow direction pointing according to an exemplary embodiment.

[0064] Figure 5 The invention is a flowchart of a method for screen display window streaming applied to a streaming control device according to an exemplary embodiment.

[0065] Figure 6 The invention is a flowchart of a method for screen display window streaming applied to a streaming receiving device according to an exemplary embodiment.

[0066] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment.

[0067] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0068] Exemplary embodiments will be described in detail below with reference to the accompanying drawings.

[0069] It should be pointed out that the relevant embodiments and drawings are only for describing exemplary embodiments provided by the present disclosure, rather than all embodiments of the present disclosure, and it should not be understood that the present disclosure is limited to the relevant exemplary embodiments.

[0070] It should be noted that the terms "first", "second", etc. used in the present disclosure are only used to distinguish different steps, devices or modules, etc. The related terms neither represent any specific technical meanings nor indicate the order or interdependence between them.

[0071] It should be noted that the modifications of the terms "one", "multiple", and "at least one" used in the present disclosure are illustrative rather than restrictive. Unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0072] It should be noted that the term "and / or" used in this disclosure is used to describe the association relationship between associated objects, and generally indicates that there are at least three association relationships. For example, A and / or B can at least indicate the existence of three association relationships: A exists alone, A and B exist at the same time, and B exists alone.

[0073] It should be noted that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. Unless otherwise specified, the scope of the present disclosure is not limited by the order of description of the steps in the relevant embodiments.

[0074] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0075] Exemplary Systems

[0076] Figure 1 is a structural block diagram of a screen transfer system 100 according to an exemplary embodiment. Figure 1 As shown, the screen streaming system 100 provided in this embodiment includes: a screen projection source device 110, a streaming control device 120 and a streaming receiving device 130.

[0077] The above-mentioned flow control device 120 is used to: obtain the connection information of the data channel corresponding to the screen projection data in the above-mentioned screen projection source device 110; detect the flow control instructions for the above-mentioned screen projection data; generate the screen projection flow instructions according to the above-mentioned flow control instructions and send them to the corresponding flow receiving device 130 in a directed manner. Among them, the data channel corresponding to the above-mentioned screen projection data is managed and controlled by the above-mentioned flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above-mentioned flow control device. The above-mentioned screen projection flow instruction carries the connection information of the data channel corresponding to the above-mentioned screen projection data; the above-mentioned connection information is held by the above-mentioned flow control device 120 and performs directed sending control of the receiving object. For example, in Figure 1 Multiple flow receiving devices 130 are exemplified, namely device A, device B and device C, and the flow receiving device corresponding to one flow control instruction is device B.

[0078] The above-mentioned flow receiving device 130 is used to: receive the above-mentioned screen projection flow instruction and establish a data channel with the corresponding screen projection source device according to the above-mentioned screen projection flow instruction.

[0079] The main function of the flow control device 120 is to guide a specific flow receiving device to establish a data channel with the screen projection source device by controlling the data channel corresponding to the projection data. The flow control device does not need to transfer the projection data, but instead controls which flow receiving devices are disclosed to which connection information of the data channel of the projection data, thereby realizing the control of the directional connection between the flow receiving device and the projection source device.

[0080] In an embodiment of the present disclosure, the flow control device 120 obtains the connection information of the data channel corresponding to the projection data from the projection source device 110; by controlling the above connection information and performing directional sending control on the receiving object, whether a data channel is established between the flow receiving device 130 and the projection source device 110 is controlled. After the flow control device 120 obtains the connection information of the data channel corresponding to the projection data from the projection source device 110, there is no restriction on whether the flow control device 120 receives the projection data from the projection source device, or whether the received projection data is displayed on its own display interface. It can receive or not receive, and can display or not display. Holding and controlling this connection information in the flow control device 120 is the key to being able to control and guide the establishment of a data channel between the projection source device 110 and the flow receiving device 130.

[0081] In some embodiments, the flow control device 120 may only display the screen projection source device icon without displaying the screen information being displayed by the screen projection source device (corresponding to the mirror screen projection method) or not displaying the interface content of a certain application interface of the screen projection source device (the interface content is not displayed on the front end of the screen projection source device, corresponding to the application screen projection method); that is, it only controls the establishment of the data channel, and the flow control device itself does not need to display the projection data. In some embodiments, the flow control device 120 can present a list of connection information of data channels corresponding to one or more projection data in the projection source device in the window; the user can select the connection information corresponding to the projection data to be circulated in the connection information list presented in the window, thereby being regarded as receiving the flow control instruction for the projection data corresponding to the selected connection information in the flow control device 120.

[0082] In some embodiments, the above-mentioned streaming receiving device is further used for:

[0083] Receive the above-mentioned screen projection data based on the above-mentioned data channel; or,

[0084] The above-mentioned screen projection data is received based on the above-mentioned data channel, and the above-mentioned screen projection data is displayed on the display interface of the above-mentioned flow receiving device.

[0085] In some embodiments, the flow control device is further used to:

[0086] Establish a data channel with the screen projection source device according to the above connection information; or

[0087] Establishing a data channel with the screen projection source device according to the connection information, and receiving the screen projection data of the screen projection source device based on the data channel; or

[0088] A data channel is established with the screen projection source device according to the connection information, the screen projection data of the screen projection source device is received based on the data channel, and the screen projection data is displayed on the display interface of the flow control device.

[0089] In the above-mentioned screen flow system 100, the number of screen projection source devices 110 can be one or more; the number of flow receiving devices 130 can be one or more, and the flow control device 120 directs and controls the data channel corresponding to the screen projection data according to the flow control instructions received by itself, and discloses it to a specific flow receiving device, thereby guiding the establishment of the data channel between the specific flow receiving device and the screen projection source device.

[0090] In some embodiments, the flow control device 120 guides the establishment of a data channel in the following manner: after the flow control device receives the flow control instruction for the above-mentioned screen projection data, it generates a screen projection flow instruction according to the above-mentioned flow control instruction and sends it to the corresponding flow receiving device 130 in a directed manner; wherein the above-mentioned screen projection flow instruction carries the connection information of the data channel corresponding to the screen projection data; the above-mentioned connection information is held by the above-mentioned flow control device and performs directed sending control on the receiving object; the flow receiving device 130 that receives the above-mentioned screen projection flow instruction establishes a data channel with the corresponding screen projection source device 110.

[0091] In some embodiments, the connection information includes: the target screen projection data subject information corresponding to the screen projection data to be circulated. The flow receiving device 130 that receives the screen projection flow instruction determines the transmission channel of the screen projection data to be circulated according to the target screen projection data subject information, and subscribes to the corresponding subject by connecting to the transmission channel to establish a data channel with the corresponding screen projection source device 110.

[0092] In some embodiments, screen projection data is transmitted between the screen projection source device and the flow control device based on mirror projection or application projection, and screen projection data is transmitted between the screen projection source device and the flow receiving device based on mirror projection or application projection.

[0093] Mirror screen projection means that the content currently displayed on the sender is projected to the receiver. When the sender is closed, the window of the receiver will not be displayed. One implementation method of mirror screen projection is that the current display information of the local display screen of the sender is directly sent to the sender after video encoding, and the sender decodes and displays it. The application screen projection method that is different from this is: based on the projection technology of the virtual display screen, the local display screen of the sender is displaying the interface A of the first application. At this time, if the mobile phone detects the application projection operation, the mobile phone transfers the first application to the virtual display screen for operation, and the interface of the second application is displayed on the local display screen. The virtual display screen sends the display information to the encoder for encoding, and the encoded display information is sent to the receiver. In the implementation method of application screen projection, the display information projected by the sender received by the receiver can be different from the display information currently displayed on the foreground of the sender.

[0094] Figure 2 It is a schematic diagram of a communication link established between a screen projection source device, a flow control device and a flow receiving device in a screen flow system according to an exemplary embodiment.

[0095] In some embodiments, the flow control device, the screen projection source device, and the flow receiving device are located in the same target space. The flow control device, the screen projection source device, and the flow receiving device are all installed with interconnection applications and perform data exchange based on a preset interconnection protocol. For example, the interconnection application is a screen flow control application.

[0096] The above-mentioned interconnection protocols include but are not limited to: CarPlay, HiCAR, CarLink, MagicRing and other protocols.

[0097] After the above interconnection application is started, each device in the above target space configures each flow direction pointing to the corresponding claiming role, and establishes a corresponding communication link according to the above claiming role. Among them, the above flow direction points to multiple logical directions pre-given by the above interconnection application.

[0098] In some embodiments, the target space includes but is not limited to one of the following: a vehicle, an aircraft cabin, or a room, such as an airplane cabin, a low-altitude manned flight equipment cabin, a conference room, a showroom, a living room, a bedroom, a car cabin supporting assisted driving or intelligent driving, etc.

[0099] The following uses a vehicle window flow and multi-screen interaction usage scenario as an example to illustrate how to establish a communication link and perform display window flow across devices.

[0100] Reference Figure 2As shown, a vehicle is used as an example of the screen flow system 100. The vehicle includes 2 terminal devices and 4 vehicle-mounted devices. The 4 terminal devices are terminal device 1 used by the driver and terminal device 2 used by the person in the co-pilot position. The 2 terminal devices can have different operating systems (such as Android, Hongmeng, ISO and other operating systems), and the operating systems between the terminal devices and the vehicle-mounted devices are also different. For example, the 4 vehicle-mounted devices correspond to the Linux operating system. A vehicle-mounted device 1 is provided corresponding to the main driver's seat (which can be arranged in the middle of the front end of the vehicle, or arranged to the left of the middle of the front end of the vehicle), and the corresponding display screen of the vehicle-mounted device 1 is the main screen LD; a vehicle-mounted device 2 is provided corresponding to the co-driver's seat, and the corresponding display screen is the sub-screen VD1; a vehicle-mounted device 3 is provided in the left rear row (for example, located on the back of the main driver's seat), and the corresponding display screen is the sub-screen VD2; a vehicle-mounted device 4 is provided in the right rear row, and the corresponding display screen is the sub-screen VD3; wherein, Internet applications are installed in the above-mentioned 4 vehicle-mounted devices and 2 terminal devices, and multiple roles pre-given in the Internet application are: role X, role Y and role Z, role X corresponds to the screen projection source device, role Y is the flow control device, and role Z is the flow receiving device. In the case of multiple flow receiving devices, each flow receiving device is pre-assigned with a logical flow direction relative to the flow control device.

[0101] As an example, terminal device 2 claims role X corresponding to the screen projection source device, vehicle-mounted device 1 claims role Y corresponding to the flow control device, and vehicle-mounted devices 2 to 4 respectively claim role Z corresponding to the flow receiving device. In order to distinguish the differences in the logical positions corresponding to each flow receiving device, roles Z1 to Z3 are used to represent the roles claimed by vehicle-mounted devices 2 to 4 respectively. Each flow receiving device (i.e., vehicle-mounted devices 2 to 4) has received a logical position corresponding to the real physical position. The following claiming information is saved in the interconnection application: role X is claimed by terminal device 1; role Y is claimed by vehicle-mounted device 1, corresponding to the virtual main driver position; role Z1 corresponds to the virtual co-driver position (this position is the logical position, and the subsequent expressions are similar and will not be repeated), and is claimed by vehicle-mounted device 2; role Z2 corresponds to the virtual left rear seat position and is claimed by vehicle-mounted device 3; role Z3 corresponds to the virtual right rear seat position and is claimed by vehicle-mounted device 4.

[0102] After determining the claimed role corresponding to each device, each device establishes a corresponding communication link according to the claimed role.

[0103] In some embodiments, reference Figure 2 As shown, the above communication link includes: a first communication link, a second communication link and a third communication link.

[0104] The first communication link is used to connect the screen projection source device 110 and the streaming control device 120. Figure 2The dotted double arrows are used to indicate the first communication link, which connects the terminal device 2 and the vehicle-mounted device 1 corresponding to the main frequency LD.

[0105] In the embodiments of the present disclosure, the above-mentioned data channel can support data links corresponding to various protocols, and any method that can realize data interactive transmission is within the protection scope of the present disclosure.

[0106] In some embodiments, the above data channel transmits the projection data based on the data distribution service (DDS) corresponding to the publish-subscribe mechanism. Based on the publish-subscribe mechanism corresponding to DDS, it is helpful to switch data between multiple devices. Due to the architecture of a publisher role (the role of the projection source device) and multiple subscriber roles (including devices corresponding to the two roles of the flow control device and the flow receiving device), the subscriber role can be flexibly added and expanded. At the same time, when adding a new subscriber role, it is not necessary for the publisher role to build a content sending link with the newly added subscriber role alone. It is only necessary to subscribe to the newly added subscriber role according to the topic information disclosed by the flow control device, which also saves network resources. What is more important is that in the vehicle driving scenario, since the flow control device does not need to transfer the projection data, the computing resource consumption of the flow control device is effectively reduced; if the vehicle-mounted device claims the role of the flow control device, it also helps to save the computing resource consumption of the vehicle-mounted device, ensuring that the vehicle-mounted device has more computing power to undertake data processing tasks related to safe driving (such as road condition recognition, automatic driving route planning, automatic driving obstacle avoidance processing, etc.), while realizing audio and video entertainment interaction between multiple devices in the car, while improving the safety of vehicle driving.

[0107] In some embodiments, the above-mentioned screen projection source device acts as the publisher role of the topic corresponding to the screen projection data. After the above-mentioned flow receiving device is guided by the above-mentioned flow control device to establish a data channel with the corresponding screen projection source device, the above-mentioned flow receiving device acts as the subscriber role of the topic corresponding to the screen projection data to receive the projection data from the screen projection source device.

[0108] The above-mentioned flow control device obtains the connection information of the data channel corresponding to the projection data from the projection source device, and the acquisition method includes: the above-mentioned projection source device acts as the publisher of the above-mentioned connection information, and the above-mentioned flow control device acts as the subscriber of the above-mentioned connection information to obtain the above-mentioned connection information.

[0109] The screen projection source device serves as a publisher of connection information of a data channel corresponding to the screen projection data (e.g., a topic corresponding to the screen projection data), and the flow control device serves as a subscriber to receive the connection information published by the screen projection source device. For example, the first communication link is used to transmit the screen projection source device 110 ( Figure 2 The terminal device 2 is used as an example) to be sent to the above-mentioned flow control device 120 ( Figure 2 The vehicle-mounted device 1 is used as an example in the figure to perform screen projection and display corresponding to the subject information (theme) of the projection data; wherein, the terminal device 2 (as an example of the projection source device 110) serves as the publisher role Pub 01 of the above-mentioned subject information, and the vehicle-mounted device 1 (as an example of the flow control device 120) serves as the subscriber role Sub 01 of the above-mentioned subject information.

[0110] After the above-mentioned flow receiving device establishes a data channel with the corresponding screen projection source device based on the received screen projection flow instruction, the flow receiving device can receive the screen projection data based on the above-mentioned data channel; the above-mentioned screen projection source device acts as the publisher role of the topic corresponding to the screen projection data, and the above-mentioned flow receiving device acts as the subscriber role to receive the screen projection data from the screen projection source device.

[0111] The second communication link is used to connect the flow control device 120 and the flow receiving device 130 in a directional manner, and the data transmission direction is from the flow control device 120 to the flow receiving device 130. Figure 2 A single-point dashed arrow is used to indicate the second communication link, which points from the vehicle-mounted device 1 (as an example of the flow control device 120) to the vehicle-mounted device 3 (as an example of the flow receiving device 130). In some embodiments, the second communication link is used to transmit the subject information corresponding to the projection data to be circulated.

[0112] The third communication link is used to connect the flow receiving device 130 and the flow control device 120 in a direction manner, and the data transmission direction is from the flow receiving device 130 to the flow control device 120. The third communication link is used to transmit at least one of the following information: event confirmation information, response information to the screen projection flow instruction. For example, the event confirmation information is: device online information, device offline information, etc. The response information includes: whether the flow receiving device agrees to perform screen projection, whether to establish a corresponding screen projection flow data channel, etc.

[0113] It is understandable that in Figure 2 In this example, only one screen projection data is used to transfer across devices. In fact, there can be more screen projection data. For example, Figure 2 As shown in the dotted box, there are three screen projection source devices, namely terminal device 2, terminal device 3 and terminal device 4. According to the different sources of screen projection data, the role X claimed by terminal devices 2 to 4 is further described as role X1, role X2 and role X3. The process of establishing the first communication link between each role X1 to X3 and the corresponding device of role Y can be understood by referring to the previous process and will not be elaborated here.

[0114] It should be noted that the corresponding relationship between the claimed role and the device is only used as an example and for ease of understanding. In fact, it is not limited to the vehicle-mounted device claiming the role of the flow control device 120. The terminal device in the vehicle can also claim the role of the flow control device 120. For example, Figure 2 As shown in the figure, the terminal device 1 illustrated in the dashed box has claimed the role of the flow control device 120. In this case, the logic of building a communication link and performing data processing when the terminal device 1 acts as the flow control device 120 can be understood by referring to the process described above.

[0115] In some embodiments, the flow control device 120 controls the data channel corresponding to the screen projection data in the following manner: the connection information of the data channel corresponding to the screen projection data is held by the flow control device 120 and performs directional sending control on the receiving object.

[0116] For the flow control device, there is no need to transfer the projection data, but to control which flow receiving devices the connection information of the data channels of which projection data is disclosed, so as to realize the control of the directional connection between the flow receiving device and the projection source device. In this way, no matter whether the operating systems of the projection source device, the flow control device and the flow receiving device are the same, and whether these three types of devices are in the same account system, the flow of screen display windows can be realized, which has a wide range of applicability; and because the flow control device does not need to transfer the projection data, the computing resource consumption of the flow control device is effectively reduced; the flow control device can be used to guide one or more newly added flow receiving devices to establish a data channel with the projection source device through directional control. As a display end device, the flow receiving device is easy to access and manage, and has strong scalability.

[0117] In some embodiments, a screen projection data of a screen projection source device can be displayed separately to multiple display end devices such as a flow control device and a flow receiving device. It is understandable that in some embodiments, the flow control device may not display the screen projection data or the displayed object may be related to the screen projection data (for example, it may be the connection information of the data channel corresponding to the screen projection data, the screen projection source device corresponding to the screen projection data, etc.).

[0118] In some embodiments, the above-mentioned screen projection flow instruction includes but is not limited to: the target screen projection data subject information corresponding to the projection data to be circulated. The essence of the target screen projection data subject information corresponding to the projection data is the connection information of the data channel corresponding to the projection data. Among them, the flow receiving device that receives the screen projection flow instruction determines the transmission channel of the projection data to be circulated according to the above-mentioned target screen projection data subject information, and subscribes to the corresponding subject by connecting to the above-mentioned transmission channel to establish a data channel with the corresponding projection source device. The above-mentioned data channel transmits the projection data based on the data distribution service (DDS) corresponding to the publish-subscribe mechanism. By obtaining the target screen projection data subject information corresponding to the projection data, the data channel can be connected.

[0119] Reference Figure 2 As shown, as an example of the process of transferring display windows across devices, on the basis of the establishment of the first communication link, the terminal device 2 used by the co-pilot user in the vehicle projects the music playing window that is playing music to the main screen LD corresponding to the vehicle-mounted device 1 for display. The person on the main driver or co-pilot issues a flow control instruction to the main screen for the flow of the music playing window to the display screen in the left rear row by at least one method, such as voice instructions, touch screen control instructions, or visual perception instructions. After recognizing this flow control instruction, the main screen LD corresponding to the vehicle-mounted device 1 generates a screen projection flow instruction based on this flow control instruction. The screen projection flow instruction is an instruction sent directly to the vehicle-mounted device 3 (corresponding to the sub-screen VD2 in the left rear row) and carries the connection information of the data channel corresponding to the projection data of the music playing window. After the vehicle-mounted device 3 receives the above-mentioned screen projection flow instruction, it establishes a data channel with the screen projection source device according to the connection information carried in the above-mentioned screen projection flow instruction (for example, the connection information is transmitted based on the second communication link, and the connection information is the subject information corresponding to the screen projection data to be circulated), and obtains the corresponding screen projection data based on the above-mentioned data channel, and circulates and displays the screen projection data on the secondary screen VD2. The display logic displayed in the screen projection receiving device 130 (for example, the vehicle-mounted device 3) can be configured based on the operation of the screen projection receiving device 130 itself, and then combined with Figure 4 An example is given for illustration and will not be expanded here.

[0120] Figure 3 It is a schematic diagram of a display interface for displaying screen projection data by a flow control device according to an exemplary embodiment.

[0121] In some embodiments, the number of projection data obtained from the projection source device is one or more. For one or more projection data, the display interface of the flow receiving device is displayed based on the first preset display configuration information; the above-mentioned first preset display configuration information is information pre-configured by the flow receiving device, including: a first window layout and a first display style of one or more display windows corresponding to the number of the above-mentioned projection data. For one or more projection data, the display interface of the flow control device is displayed based on the second preset display configuration information, and the second preset display configuration information is information pre-configured by the flow control device, including: a second window layout and a second display style of one or more display windows corresponding to the number of the above-mentioned projection data. In order to simplify the explanation, only the flow control device is used as an example to illustrate the display layout.

[0122] Reference Figure 3 As shown in FIG. 1 , a display interface 300 for displaying screen projection data by the flow control device 120 is shown. The display interface 300 includes two areas, one is a screen projection data source display area 310, and the other is a screen projection display area 320. In the screen projection data source display area 310, all screen projection data received by the flow control device 120 are independently displayed in a tiled manner, for example, Figure 3 The example shows the screen projection data received by the flow control device 120 from the terminal devices 2 to 4 corresponding to the roles X1 to X3. The screen projection data T1 received from the role X1 corresponding to the terminal device 2 is, for example, a data stream of a music playback interface; the screen projection data T2 received from the role X2 corresponding to the terminal device 3 is, for example, a data stream of a map navigation interface; the screen projection data T3 received from the role X3 corresponding to the terminal device 4 is, for example, a data stream of a movie playback interface. Multiple screen projection data can be displayed in the screen projection display area 320 of the flow control device. For example, multiple windows are arranged in the screen projection display area 320. The layout of the multiple windows can be: multiple windows have small window options at the bottom and one is selected to be displayed independently in the main window above, multiple windows are tiled in different areas, or there are two windows or multiple windows in multiple windows. The nested display of two windows means that the content display area of ​​a window is located inside the display area of ​​another window.

[0123] exist Figure 3In the figure, multiple projection data windows corresponding to the projection data source display area 310 are used as small window options. Taking the projection display area 320 containing multiple windows displayed in tiled areas as an example, the projection data T1~T3 in each small window option can be automatically matched to the window corresponding to the projection display area 320 for display, or displayed in the window corresponding to the projection display area 320 according to the user's manual drag command; the layout of the projection display area 320 can be switched in the existing layout options as the user interacts with the flow control device 120.

[0124] exist Figure 3 The correspondence between multiple projection data and each window in the projection display area 320 is exemplified: the display window C1 of the navigation map displays the projection data T2 from the terminal device 3 (the corresponding character is X2); the music playback window C2 displays the projection data T1 from the terminal device 2 (the corresponding character is X1); the movie playback window C4 displays the projection data T3 from the terminal device 4 (the corresponding character is X3).

[0125] It should be noted that as a special scenario example of cross-device display window flow, refer to Figure 3 As shown in the dashed box, the projection display area 320 can also display data of the built-in application interface of the flow control device 120, such as a vehicle energy consumption curve window C3 for presenting the real-time energy consumption of the vehicle; if the user issues a flow control instruction for the real-time energy consumption data of the vehicle in the vehicle energy consumption curve window C3 (hereinafter described as projection data T4), the data of the vehicle energy consumption curve window C3 can be projected and transferred to the flow receiving device in accordance with the processing logic of the display interface flow described above. In this case, the real-time energy consumption data of the vehicle corresponding to the vehicle energy consumption curve window C3 will also be regarded as projection data T4; the disclosed embodiment covers this situation in the scenario and protection scope of cross-device display window flow, which corresponds to the scenario where the role of the flow control device and the role of the projection source device are combined into one. Therefore, it should be noted that the above-mentioned role X, role Y and role Z are for the data in a certain display window. Different display windows on the same device may have different role correspondences. For example, in Figure 3 In the example, windows C1, C2 and C4 (a simplified description of the previous windows, the same applies to the subsequent ones) correspond to the device where they are located corresponding to a single role and all correspond to the role of the flow control device; window C3 corresponds to the device where they are located corresponding to the role of the flow control device and the role of the screen projection source device combined into one.

[0126] In some embodiments, the flow control instructions include at least one of the following forms of instructions: voice instructions, touch screen control instructions, and human motion visual perception instructions. Among them, human motion visual perception instructions include but are not limited to the perception of at least one of the following types of actions: gestures, head movements, eye movements, and facial expressions.

[0127] The above-mentioned flow control instruction may carry the following information: the projection data to be transferred and the flow direction; or, carry the following information: the projection data to be transferred and the target flow receiving device to be transferred; or, carry the following information: the projection data to be transferred, the flow direction and the target flow receiving device to be transferred.

[0128] The above embodiment takes the display of projection data on the display interface of the flow control device as an example. In fact, the corresponding content related to the projection data can also be displayed on the flow control device; initiating a flow instruction for these contents is also regarded as receiving an instruction to flow the projection data. For example, the flow control device 120 can present a list of connection information of data channels corresponding to one or more projection data in the projection source device in the window (as an example of corresponding content related to the projection data, it can also be varied but not limited to icons corresponding to the projection source device, application icons corresponding to multiple projection data corresponding to the projection source device, etc.); the user can select the connection information corresponding to the projection data to be circulated in the connection information list presented in the window, which is regarded as receiving the flow control instruction for the projection data corresponding to the selected connection information in the flow control device 120.

[0129] When the flow control instructions include voice instructions, the voice instructions in the corresponding environment of the screen flow system (the space in which it is located, such as inside a vehicle, inside an aircraft cabin, inside a room, etc.) are semantically recognized and analyzed to obtain flow control instructions for instructing the flow of the projection data to be transferred in the display interface.

[0130] Touch screen control instructions refer to instructions that need to be identified by touching the screen. Users can issue flow control instructions by operating in the window where the projection data to be transferred is located, such as by sliding in a certain direction, by double-clicking, by long pressing, etc. The flow control device 120 identifies the preset operation and compares it with the known operations in the preset operation library to determine the direction in which the projection data in a specific window is to be transferred and the target flow receiving device to which it is to be transferred.

[0131] Human motion visual perception instructions refer to instructions that do not require contact with the screen and can be recognized through visual perception. The flow control device 120 can be equipped with a visual perception sensor (for example, a video perception device), which senses human motion (for example, the driver's motion) and identifies motion semantic information. Based on the identified motion semantic information, the specified actions in the preset action library are matched to determine the instruction content of the flow control instructions for the projection data in the above-mentioned display interface. As an example, in a vehicle, the specified actions and corresponding meanings in the preset action library include the following: Turn your head right-transfer the projection data of the target window to the display screen at the co-pilot position, nod once-transfer the projection data in the target window to the display screen at the left rear row, and nod three times-transfer the projection data in the target window to the display screen at the left rear row. It can be understood that it is possible to support the customization of human motions corresponding to each flow control instruction according to user habits.

[0132] Figure 4 It is a schematic diagram of a process of determining a target flow direction pointing that matches a target flow control instruction according to a mapping relationship between a preset operation control instruction and a flow direction pointing according to an exemplary embodiment.

[0133] In some embodiments, the target flow control instruction is at least one of the following instructions: a first control instruction in the first target window for controlling the screen projection interface in the current window to perform screen projection flow; or, a second control instruction in the second target window for controlling the built-in application interface in the current window to perform screen projection flow; or, a third control instruction in the third target window for controlling the flow of screen projection data received by the current window; or, a fourth control instruction in the fourth target window for controlling the flow of screen projection data corresponding to the data channel connected to the current window. For example, refer to Figure 4 As shown, the flow control instructions for windows C1, C2 and C4 shown in the solid line frame are examples of the first control instruction; the flow control instruction for window C3 shown in the dotted line frame is an example of the second control instruction. The third control instruction and the fourth control instruction may correspond to the embodiments of the corresponding contents related to the display and projection data of the flow control device, which will not be described in detail here.

[0134] In some embodiments, generating a screen casting flow instruction according to the flow control instruction and sending it to the corresponding flow receiving device in a directional manner includes the following steps:

[0135] In response to detecting a target flow control instruction for controlling the flow of projection data corresponding to the current window in the target window of the flow control device, a target flow direction matching the target flow control instruction is determined according to a mapping relationship between a preset operation control instruction and a flow direction; for example, referring to Figure 4As shown, if the touch screen track corresponding to the operation control instruction is a downward arrow, the corresponding matching flow direction points to the virtual left rear seat direction; if the touch screen track corresponding to the operation control instruction is a right arrow, the corresponding matching flow direction points to the virtual co-pilot direction; if the touch screen track corresponding to the operation control instruction is an L-shaped arrow, the corresponding matching flow direction points to the virtual right rear seat direction; in some embodiments, the touch screen track corresponding to the operation control instruction can be determined by detecting a sliding gesture of the user on the screen; the sliding gesture can be determined according to the detected initial position coordinates and end position coordinates and at least one intermediate position point;

[0136] According to the pre-configured claiming roles corresponding to the various flow directions, the target claiming role corresponding to the target flow direction is determined; for example, referring to Figure 4 As shown, according to the corresponding claiming roles and corresponding devices of each flow direction in the previous example, the claiming role of the virtual left rear seat direction is Z2, corresponding to the vehicle device 3; the claiming role of the virtual co-pilot direction is Z1, corresponding to the vehicle device 2; the claiming role of the virtual right rear seat direction is Z3, corresponding to the vehicle device 4;

[0137] Determine the electronic device under the above target claiming role as the target transfer receiving device; continue to refer to Figure 4 It can be seen that the target flow receiving device corresponding to window C1 and window C2 is vehicle-mounted device 3, the target flow receiving device corresponding to window C3 is vehicle-mounted device 2, and the target flow receiving device corresponding to window C4 is vehicle-mounted device 4;

[0138] Determine the screen projection source device corresponding to the screen projection data and the target screen projection data subject information of the screen projection source device for the screen projection data to be circulated during the data distribution process; this target screen projection data subject information is essentially the connection information corresponding to the data channel of the screen projection data to be circulated;

[0139] According to the above-mentioned target screen projection data subject information, a screen projection flow instruction corresponding to the above-mentioned target flow control instruction is generated; for example, the above-mentioned screen projection flow instruction contains subject information corresponding to the screen projection data to be circulated: the screen projection flow instruction for vehicle-mounted device 3 carries subject information corresponding to the projection data T1 and the projection data T2; the screen projection flow instruction for vehicle-mounted device 2 carries subject information corresponding to the projection data T4; the screen projection flow instruction for vehicle-mounted device 4 carries subject information corresponding to the projection data T3;

[0140] The above-mentioned screen projection flow instruction is sent to the above-mentioned target flow receiving device.

[0141] By executing the above steps, it is possible to determine the designated receiving object to which the connection information of the data channel of the screen projection data to be circulated is directed according to the mapping relationship between the pre-configured operation control instructions and the flow direction and the claiming role corresponding to each flow direction. By sending the above connection information to the designated receiving object, it is possible to realize the control of the directional connection between the flow receiving device and the screen projection source device. In this way, regardless of whether the operating systems of the screen projection source device, the flow control device and the flow receiving device are the same, and whether these three types of devices are in the same account system, the flow of the screen display window can be realized, which has a wide range of applicability.

[0142] In some embodiments, reference Figure 4 As shown, the display logic configured by the operation of each projection receiving device 130 is explained by taking the vehicle-mounted device 2 to the vehicle-mounted device 4 as an example.

[0143] In the vehicle-mounted device 2, there are two display windows, one display window is the display window corresponding to the built-in application 11 running on the vehicle-mounted device 2, and the other display window is the display window Z1-C3 for displaying the projection data T4 (real-time energy consumption data of the vehicle). According to the first display logic of the vehicle-mounted device 2, the display window Z1-C3 of the external projection data is nested in the upper left corner of the display window of the built-in application 11 for display, and the display window Z1-C3 floats on the display window of the built-in application 11.

[0144] In the vehicle-mounted device 3, there are three display windows, the first display window is the display window Z2-C2 corresponding to the projection data T1 (data stream of the music playback interface), the second display window is the display window Z2-C1 corresponding to the projection data T2 (data stream of the map navigation interface), and the third display window is the display window of the built-in application 21 of the vehicle-mounted device 3. According to the second display logic of the vehicle-mounted device 3, the display windows Z2-C2 and Z2-C1 of the external projection data and the display window of the built-in application 21 are tiled for display.

[0145] In the vehicle-mounted device 4, there is a display window, which is the display window Z3-C4 corresponding to the projection data T3 (data stream of the movie playback interface). According to the third display logic of the vehicle-mounted device 4, the display window Z3-C4 of the external projection data is displayed in full screen; other running applications in the vehicle-mounted device 4 run in the background.

[0146] In the embodiments of the present disclosure, the projection source device 110, the streaming control device 120 and the streaming receiving device 130 correspond to three device roles, and generally each device role corresponds to a different device. In some scenarios, there are two device roles located in the same entity.

[0147] For example, the screen projection source device 110 and the flow control device 120 can be different devices or the same device. When the screen projection source device 110 and the flow control device 120 are different devices, the flow control device receives the projection data obtained through the screen projection method from the screen projection source device. When the screen projection source device and the flow control device are the same device, the actual meaning of the flow control device receiving the projection data from the screen projection source device is: the flow control device receives the data of its own built-in application interface as the projection data, and this data is its own display data, not the data obtained through the screen projection method; because this data is subsequently circulated and displayed in the display windows of different devices, this data is also described as projection data, which means that it is used as the data for subsequent screen projection.

[0148] In some embodiments, the above-mentioned screen projection source device includes one of the following situations: multiple applications within a device are respectively used as the projection source of the above-mentioned screen projection data; or, the display interfaces corresponding to multiple devices are respectively used as the projection source of the above-mentioned screen projection data; or, part of the above-mentioned screen projection data comes from one or more applications within a device, and the other part of the screen projection data comes from the display interface corresponding to at least one device.

[0149] For example, the screen projection source device 110 and the stream receiving device 130 may be different devices, or they may be located in the same entity.

[0150] For example, the following situation: while the projection data of the first topic published by the projection source device is projected and displayed by the flow control device, the projection source device also serves as a flow receiving device for receiving the second topic sent from the flow control device, establishing a data channel between the projection source device corresponding to the second topic, and performing flow display of the corresponding projection data according to a preset display strategy; wherein the first topic and the second topic are different topics.

[0151] Alternatively, it is another situation as follows: while the flow receiving device receives the third topic sent from the flow control device, establishes a data channel with the screen projection source device corresponding to the third topic and performs flow display of the corresponding screen projection data, the flow receiving device also serves as a screen projection source device for publishing the screen projection data of the third topic or the fourth topic, and the screen projection is displayed by the flow control device; wherein the third topic and the fourth topic are different topics.

[0152] In an embodiment of the present disclosure, the above-mentioned flow control device 120 maintains an online device list; the above-mentioned online device list is used to indicate electronic devices that have established a communication link with the above-mentioned flow control device and are in an online state, and the electronic devices in the above-mentioned online device list are options for flow receiving devices. The above-mentioned online device list will be updated according to the status regularly reported by the flow receiving device, for example, each flow receiving device in the list is set to regularly report the online status to the flow control device. If the online status is not reported for more than a set time interval, it is considered offline.

[0153] The flow control device to which the projection flow control instruction is directed is an online device in the online device list. If it is detected that the target flow control device to which the projection flow control instruction is to be sent is in an offline state in the online device list, the following logic is executed: refuse to send the projection flow control instruction to the target flow control device, and prompt an error message that the target flow control device is offline.

[0154] In an embodiment of the present disclosure, the above-mentioned flow control device 120 maintains a topic disclosure status list. The above-mentioned topic disclosure status list is used to maintain the following information: the data distribution topic, the data distribution correspondence between the screen projection source device where the topic comes from and the flow receiving device to which it is directed, and the link status of whether the screen projection flow data channel between the flow receiving device to which it is directed and the screen projection source device is opened. The above-mentioned topic disclosure status list can be updated in real time according to the status feedback from the flow receiving device (for example, whether the flow receiving device agrees to perform screen projection, whether to establish a corresponding screen projection flow data channel, exit the screen projection status, etc.).

[0155] In the above-mentioned screen flow system 100, the flow control device controls the data channel corresponding to the screen projection data so that the data channel corresponding to the screen projection data cannot be directly established between any flow receiving device and the screen projection source device, but is guided to be established through the above-mentioned flow control device. The main role of the flow control device is to guide a specific flow receiving device to establish a data channel with the screen projection source device by controlling the data channel corresponding to the screen projection data. For the flow control device, there is no need to transfer the screen projection data, but to control which flow receiving devices the connection information of the data channel of which screen projection data is disclosed, thereby realizing the control of the directional connection between the flow receiving device and the screen projection source device. In this way, regardless of whether the operating systems of the screen projection source device, the flow control device and the flow receiving device are the same, and whether these three types of devices are in the same account system, the flow of the screen display window can be realized, and it has a wide range of applicability; and because the flow control device does not need to transfer the screen projection data, the computing resource consumption of the flow control device is effectively reduced. The flow control device can be used to guide one or more newly added flow receiving devices to establish a data channel with the screen projection source device through directional control. The flow receiving device is easy to access and manage as a display end device, and has strong scalability.

[0156] Other details of this embodiment can be combined with the detailed description of subsequent method embodiments, which will not be elaborated here. Similarly, other details of subsequent method embodiments can refer to the description of the system in this embodiment. The various embodiments can complement each other and be understood in combination.

[0157] Exemplary Methods

[0158] The method for screen display window flow provided by the embodiment of the present disclosure and executed by the flow control device includes: obtaining the connection information of the data channel corresponding to the screen projection data in the above-mentioned screen projection source device; detecting the flow control instruction for the above-mentioned screen projection data; generating the screen projection flow instruction according to the above-mentioned flow control instruction and sending it to the corresponding flow receiving device in a directed manner.

[0159] Among them, the data channel corresponding to the above-mentioned screen projection data is managed and controlled by the above-mentioned flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above-mentioned flow control device; the above-mentioned screen projection flow instruction carries the connection information of the data channel corresponding to the above-mentioned screen projection data; the above-mentioned connection information is held by the above-mentioned flow control device and performs directional sending control on the receiving object.

[0160] In some embodiments, the flow control device further performs the following steps:

[0161] Establish a data channel with the screen projection source device according to the above connection information; or

[0162] Establishing a data channel with the screen projection source device according to the connection information, and receiving the screen projection data of the screen projection source device based on the data channel; or

[0163] A data channel is established with the screen projection source device according to the above connection information, the screen projection data of the above screen projection source device is received based on the above data channel, and the above screen projection data is displayed on the display interface of the above flow control device.

[0164] Refer to the following Figure 5 To describe the process of display window flow of the flow control device when displaying projection data. It can be understood that for the flow control device, it can display the projection data or not, or even not receive the projection data from the projection source device. The flow control device only needs to hold the connection information of the data channel. Holding and controlling this connection information in the flow control device 120 is the key to being able to control and guide the establishment of a data channel between the projection source device 110 and the flow receiving device 130.

[0165] Figure 5 FIG. 1 is a flow chart showing a method for transferring a screen display window of a transfer control device according to an exemplary embodiment. Figure 5 As shown, the screen display window flow method applied to the flow control device provided in this embodiment includes the following steps: S510, S520 and S530.

[0166] In step S510, the projection data from the projection source device is received, and the projection data is displayed on the display interface; wherein, the data channel corresponding to the projection data is managed and controlled by the flow control device, and any flow receiving device does not directly establish a data channel with the projection source device but is guided to establish it through the flow control device.

[0167] In step S520, the flow control instruction for the projection data in the above-mentioned display interface is detected.

[0168] In step S530, a screen projection flow instruction is generated according to the above-mentioned flow control instruction and sent to the corresponding flow receiving device in a directed manner; so that the flow receiving device that receives the above-mentioned screen projection flow instruction establishes a data channel with the corresponding screen projection source device and performs flow display of the corresponding screen projection data.

[0169] In some embodiments, after the screen casting streaming instruction is directed to the corresponding streaming receiving device, the method further includes:

[0170] Receiving an interface control instruction for reducing or closing the projection interface in the first window;

[0171] According to the above-mentioned interface control instructions, a zoom-out or closing operation is performed on the projection interface in the above-mentioned first window in the display interface of the above-mentioned flow control device; wherein the above-mentioned zoom-out or closing operation does not affect the flow display of the corresponding projection data by the flow receiving device that receives the projection data corresponding to the above-mentioned first window.

[0172] In some embodiments, the above method further comprises:

[0173] Receiving an interface operation instruction for interacting with the projection interface in the second window;

[0174] Sending the above-mentioned interface operation instruction to the target data source device corresponding to the screen projection data of the above-mentioned second window; so that the above-mentioned target data source device performs response processing based on the above-mentioned interface operation instruction to obtain the screen projection data after interaction;

[0175] Receive the post-interaction projection data fed back from the above-mentioned target data source device, and update and display it in the second window of the display interface.

[0176] In some embodiments, the above method further comprises:

[0177] Receive the response information fed back by the streaming receiving device to the above-mentioned screen projection streaming instruction; the above-mentioned response information includes: whether the streaming receiving device agrees to perform screen projection and whether to establish a corresponding screen projection streaming data channel;

[0178] According to the above response information, the subject disclosure status list is updated accordingly; wherein the above subject disclosure status list is used to maintain the following information: the data distribution subject, the data distribution correspondence between the screen projection source device from which the subject comes and the streaming receiving device to which the subject is directed, and the link status of whether the screen projection streaming data channel between the streaming receiving device to which the subject is directed and the screen projection source device is opened;

[0179] Receive a status update instruction indicating that a data channel connection is disconnected, which is sent by the streaming receiving device when the device exits the screen projection state; and update the above-mentioned subject disclosure status list according to the above-mentioned status update instruction.

[0180] The above-mentioned embodiments can be combined, and other details can refer to the relevant description of the first embodiment, which will not be repeated here.

[0181] In the method provided in this embodiment, since any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above-mentioned flow control device, the main role of the flow control device is to guide the specific flow receiving device to establish a data channel with the screen projection source device by controlling the data channel corresponding to the screen projection data. For the flow control device, there is no need to transfer the screen projection data, but to control which flow receiving devices the connection information of the data channels of which screen projection data is disclosed, thereby realizing the control of the directional connection between the flow receiving device and the screen projection source device. In this way, regardless of whether the operating systems of the screen projection source device, the flow control device and the flow receiving device are the same, and whether these three types of devices are in the same account system, the flow of screen display windows can be realized, and it has a wide range of applicability; and since the flow control device does not need to transfer the screen projection data, the computing resource consumption of the flow control device is effectively reduced. When both the flow control device and the flow receiving device are displaying the projection data, one projection data of the projection source device can be displayed separately to multiple display end devices such as the flow control device and the flow receiving device, and the flow control device can perform directional flow control. The display end devices are easy to access and manage, and have strong scalability.

[0182] Exemplary Methods

[0183] The method for screen display window streaming provided by the streaming receiving device according to the embodiment of the present disclosure includes:

[0184] Receive a screen projection flow instruction sent by a flow control device; the screen projection flow instruction carries connection information of a data channel corresponding to the screen projection data; the connection information is held by the flow control device and performs directional sending control on the receiving object;

[0185] According to the above-mentioned screen projection flow instruction, a data channel is established with the corresponding screen projection source device; wherein, the data channel corresponding to the above-mentioned screen projection data is managed and controlled by the above-mentioned flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above-mentioned flow control device.

[0186] In some embodiments, the connection information includes: the target projection data subject information corresponding to the projection data to be transferred. The transfer receiving device that receives the projection transfer instruction determines the transmission channel of the projection data to be transferred according to the target projection data subject information, and subscribes to the corresponding subject by connecting to the transmission channel to establish a data channel with the corresponding projection source device.

[0187] In some embodiments, the above-mentioned streaming receiving device further performs the following steps:

[0188] Receive the above-mentioned screen projection data based on the above-mentioned data channel; or,

[0189] The above-mentioned screen projection data is received based on the above-mentioned data channel, and the above-mentioned screen projection data is displayed on the display interface of the above-mentioned flow receiving device.

[0190] Figure 6 FIG. 1 is a flow chart showing a method for streaming a screen display window of a streaming receiving device according to an exemplary embodiment. Figure 6 As shown, the method for screen display window streaming applied to a streaming receiving device includes the following steps: S610, S620 and S630.

[0191] In step S610, a screen projection flow instruction directed by a flow control device is received.

[0192] In step S620, according to the above-mentioned screen projection flow instruction, a data channel is established with the corresponding screen projection source device.

[0193] In step S630, the corresponding projection data is obtained based on the above data channel, and the projection data is circulated and displayed based on the above projection data. In some embodiments, the above projection flow instruction includes: the target projection data subject information corresponding to the projection data to be circulated. Among them, the flow receiving device that receives the projection flow instruction determines the transmission channel of the projection data to be circulated according to the above target projection data subject information, and subscribes to the corresponding subject by connecting to the above transmission channel to establish a data channel with the corresponding projection source device.

[0194] In some embodiments, after receiving the screen projection streaming instruction sent by the streaming control device, the method further includes:

[0195] A prompt confirmation message is displayed on the display interface of the above-mentioned streaming receiving device, asking whether to agree to screen projection;

[0196] In response to the user choosing to agree to screen projection in response to the above prompt confirmation information, the following steps are performed: according to the above screen projection flow instruction, a data channel is established between the corresponding screen projection source device; and a first response message for the above screen projection flow instruction is sent to the above flow control device, the above first response message includes feedback content that the screen projection is agreed and the corresponding screen projection flow data channel has been established; so that the above flow control device updates the subject disclosure status list according to the above first response message.

[0197] The above method also includes the execution logic of the negative branch: in response to the user choosing not to agree to screen projection in response to the above prompt confirmation information, a second response message for the above screen projection flow instruction is sent to the above flow control device, and the above second response message includes feedback content that the screen projection is not agreed to and the corresponding screen projection flow data channel has not been established.

[0198] In the process of streaming and displaying the projection data, the method further includes:

[0199] Receive a screen projection end control instruction for exiting the screen projection interface;

[0200] In response to the above-mentioned screen projection end control instruction, disconnect the data channel between the corresponding screen projection source device and exit the screen projection state;

[0201] And send a status update instruction indicating that the data channel connection is disconnected to the above-mentioned circulation control device; so that the above-mentioned circulation control device updates the above-mentioned subject disclosure status list according to the above-mentioned status update instruction.

[0202] In some embodiments, the screen projection data obtained by the streaming receiving device based on the above data channel is one of the following situations:

[0203] The screen projection source device and the flow control device are different devices, and the screen projection data is the screen projection data from the screen projection source device; or,

[0204] The above-mentioned screen projection source device and the above-mentioned flow control device are the same device, and the above-mentioned screen projection data are data from the built-in application interface of the flow control device.

[0205] In some embodiments, the above method further comprises:

[0206] Receive interface operation instructions for interacting with the displayed target projection screen interface;

[0207] Send the above-mentioned interface operation instruction to the target data source device corresponding to the projection data of the above-mentioned target projection screen interface; so that the above-mentioned target data source device performs response processing based on the above-mentioned interface operation instruction to obtain the projection screen data after interaction;

[0208] Receive the post-interaction projection data fed back from the above-mentioned target data source device, and update and display it on the above-mentioned target projection interface.

[0209] For more details and beneficial effects of this embodiment, please refer to the relevant descriptions of the previous two embodiments, which will not be repeated here.

[0210] Example Vehicles

[0211] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment. Figure 7As shown, the vehicle includes: at least one terminal device 710 and multiple vehicle-mounted devices 720; the vehicle-mounted device is an electronic device based on the vehicle-mounted system and has a vehicle-mounted display screen, and the terminal device is a movable device or a wearable device with a display screen; the terminal device and the vehicle-mounted device are both located in the vehicle space; wherein the terminal device and the vehicle-mounted device share the following three roles: role X corresponding to the screen projection source device, role Y of the flow control device, and role Z of the flow receiving device.

[0212] The above-mentioned flow control device is used to: obtain the connection information of the data channel corresponding to the screen projection data in the above-mentioned screen projection source device; detect the flow control instruction for the above-mentioned screen projection data; generate the screen projection flow instruction according to the above-mentioned flow control instruction and send it to the corresponding flow receiving device in a directional manner; wherein, the data channel corresponding to the above-mentioned screen projection data is managed and controlled by the above-mentioned flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the above-mentioned flow control device. The above-mentioned screen projection flow instruction carries the connection information of the data channel corresponding to the above-mentioned screen projection data; the above-mentioned connection information is held by the above-mentioned flow control device and performs directional sending control of the receiving object.

[0213] The above-mentioned flow receiving device is used to: receive the above-mentioned screen projection flow instruction and establish a data channel with the corresponding screen projection source device according to the above-mentioned screen projection flow instruction.

[0214] In some embodiments, the above-mentioned streaming receiving device is further used for:

[0215] Receive the above-mentioned screen projection data based on the above-mentioned data channel; or,

[0216] The above-mentioned screen projection data is received based on the above-mentioned data channel, and the above-mentioned screen projection data is displayed on the display interface of the above-mentioned flow receiving device (corresponding to obtaining the corresponding screen projection data based on the above-mentioned data channel, and flow displaying based on the above-mentioned screen projection data).

[0217] In some embodiments, the flow control device is further used to:

[0218] Establish a data channel with the screen projection source device according to the above connection information; or

[0219] Establishing a data channel with the screen projection source device according to the connection information, and receiving the screen projection data of the screen projection source device based on the data channel; or

[0220] A data channel is established with the screen projection source device according to the connection information, the screen projection data of the screen projection source device is received based on the data channel, and the screen projection data is displayed on the display interface of the flow control device.

[0221] Exemplary Electronic Devices

[0222] Figure 8 1 is a block diagram of an electronic device 800 according to an exemplary embodiment. The electronic device 800 may be a vehicle controller, a vehicle-mounted terminal, a vehicle-mounted computer, or other smart home or other electronic devices with display screens that support interconnection.

[0223] Reference Figure 8 , the electronic device 800 may include at least one processor 810 and a memory 820. The processor 810 may execute instructions stored in the memory 820. The processor 810 is communicatively connected to the memory 820 via a data bus. In addition to the memory 820, the processor 810 may also be communicatively connected to an input device 830, an output device 840, and a communication device 850 via a data bus.

[0224] The processor 810 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0225] The memory 820 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0226] In the embodiment of the present disclosure, executable instructions are stored in the memory 820, and the processor 810 can read the executable instructions from the memory 820 and execute the instructions to implement all or part of the steps of any of the above-mentioned methods for screen display window flow in the above-mentioned exemplary embodiments.

[0227] Exemplary computer-readable storage media

[0228] In addition to the above methods and devices, the exemplary embodiments of the present disclosure may also be a computer program product or a computer-readable storage medium storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to implement all or part of the steps described in any method in the above exemplary embodiments.

[0229] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for executing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages ​​and scripting languages ​​(e.g., Python). The program code may be executed entirely on the user computing device, partially on the user computing device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0230] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media include: a static random access memory (SRAM) electrically connected to one or more wires, an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk, or any suitable combination thereof.

[0231] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0232] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for transferring a screen display window, characterized in that: The method comprises: There is a data channel corresponding to the screen projection data between the streaming receiving device and the screen projection source device; Among them, the data channel corresponding to the screen projection data is managed and controlled by the flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the flow control device.

2. The method according to claim 1, characterized in that The flow control device guides the establishment of the data channel in the following manner: After receiving the flow control instruction for the screen projection data, the flow control device generates a screen projection flow instruction according to the flow control instruction and sends it to the corresponding flow receiving device in a directional manner; wherein the screen projection flow instruction carries the connection information of the data channel corresponding to the screen projection data; the connection information is held by the flow control device and performs directional sending control on the receiving object; The flow receiving device that receives the screen projection flow instruction establishes a data channel with the corresponding screen projection source device.

3. The method according to claim 2, characterized in that The connection information includes: target screen projection data subject information corresponding to the screen projection data to be transferred; Among them, the flow receiving device that receives the screen projection flow instruction determines the transmission channel of the projection data to be circulated according to the target screen projection data subject information, and subscribes to the corresponding subject by connecting to the transmission channel to establish a data channel with the corresponding screen projection source device.

4. The method according to any one of claims 1 to 3, characterized in that The data channel transmits the screen projection data based on a publish-subscribe mechanism; The screen projection source device acts as a publisher of the topic corresponding to the screen projection data. After the flow receiving device establishes a data channel with the corresponding screen projection source device under the guidance of the flow control device, the flow receiving device acts as a subscriber of the topic corresponding to the screen projection data to receive the screen projection data from the screen projection source device. The flow control device obtains the connection information of the data channel corresponding to the projection data from the projection source device, and the acquisition method includes: the projection source device acts as the publisher of the connection information, and the flow control device acts as the subscriber of the connection information to obtain the connection information.

5. The method according to any one of claims 1 to 3, characterized in that: The flow control device obtains connection information of the data channel corresponding to the screen projection data from the screen projection source device; controls whether a data channel is established between the flow receiving device and the screen projection source device by controlling the connection information and performing directional sending control on the receiving object; The method may further include one or more of the following: The streaming receiving device receives the screen projection data based on the data channel; or, The streaming receiving device receives the screen projection data based on the data channel, and displays the screen projection data on the display interface of the streaming receiving device; or, The streaming control device establishes a data channel with the screen projection source device according to the connection information; or The flow control device establishes a data channel with the screen projection source device according to the connection information, and receives the screen projection data of the screen projection source device based on the data channel; or, The flow control device establishes a data channel with the screen projection source device according to the connection information, receives the projection data of the screen projection source device based on the data channel, and displays the projection data on the display interface of the flow control device.

6. A screen transfer method, characterized in that: Applied to a streaming receiving device, the method comprises: Receive a screen projection flow instruction sent by a flow control device; the screen projection flow instruction carries connection information of a data channel corresponding to the screen projection data; the connection information is held by the flow control device and performs directional sending control on the receiving object; According to the screen projection flow instruction, a data channel is established with the corresponding screen projection source device; Among them, the data channel corresponding to the screen projection data is managed and controlled by the flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the flow control device.

7. A screen transfer system, characterized in that: include: Screen projection source device, streaming control device and streaming receiving device; The flow control device is used to: obtain the connection information of the data channel corresponding to the screen projection data in the screen projection source device; detect the flow control instruction for the screen projection data; generate the screen projection flow instruction according to the flow control instruction and send it to the corresponding flow receiving device in a directed manner; wherein the data channel corresponding to the screen projection data is managed and controlled by the flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the flow control device; the screen projection flow instruction carries the connection information of the data channel corresponding to the screen projection data; the connection information is held by the flow control device and performs directed sending control on the receiving object; The flow receiving device is used to: receive the screen projection flow instruction and establish a data channel with the corresponding screen projection source device according to the screen projection flow instruction.

8. A vehicle, characterized in that: include: At least one terminal device and multiple vehicle-mounted devices; The vehicle-mounted device is an electronic device based on a vehicle-mounted system and having a vehicle-mounted display screen, and the terminal device is a movable device or a wearable device having a display screen; the terminal device and the vehicle-mounted device are both located in the vehicle space; The terminal device and the vehicle-mounted device share the following three roles: screen projection source device, flow control device and flow receiving device; The flow control device is used to: obtain the connection information of the data channel corresponding to the screen projection data in the screen projection source device; detect the flow control instruction for the screen projection data; generate the screen projection flow instruction according to the flow control instruction and send it to the corresponding flow receiving device in a directed manner; wherein the data channel corresponding to the screen projection data is managed and controlled by the flow control device, and any flow receiving device does not directly establish a data channel with the screen projection source device but is guided to establish it through the flow control device; the screen projection flow instruction carries the connection information of the data channel corresponding to the screen projection data; the connection information is held by the flow control device and performs directed sending control on the receiving object; The flow receiving device is used to: receive the screen projection flow instruction and establish a data channel with the corresponding screen projection source device according to the screen projection flow instruction.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-6.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

Images