Screen projection picture control method and system and storage medium

Abstract

The embodiment of the invention discloses a projection screen picture control method and system and a storage medium, in the projection screen picture control method, a first display device receives a projection screen picture sent by a projection screen source device through a first data channel, and displays the projection screen picture in a full screen mode; the first display device sends the screen projection picture to the second display device through the second data channel, so that the second display device displays the screen projection picture in a picture-in-picture window mode; the first display device receives first position data sent by the second display device through the third data channel, wherein the first position data is position data generated by the second display device according to a touch operation acting on the picture-in-picture window; the first display device sends the second position data to the screen projection source device through the first data channel, so that the screen projection source device updates a screen projection picture according to the second position data, and the second position data is position data confirmed based on the first position data. According to the embodiment of the invention, the fluency of multi-person communication and the continuity of information transmission are ensured.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of interactive technology, and in particular, to a projection screen control method, system and storage medium. Background Art

[0002] In multi-person communication scenarios under the background of information technology, it is common for multiple people to communicate around the focus information presented by the display device. In this communication process, the source of the focus information may include files uploaded by the terminal devices equipped by each communication participant, as well as various files generated by the display device. If there are remote communication participants, there may also be files generated by remote conference devices. The display process of the focus information can have a variety of display strategies depending on the installation and connection method of the display device.

[0003] In a current multi-person communication scenario, there may be multiple display devices. Each display device is independently connected to a terminal device, and connections can be established between the display devices. Based on these connection relationships, the display screen of one device can be displayed in full screen on another device, or in the form of a picture-in-picture window on another device. In this multi-person communication scenario, usually only the presenter presents the focus information on one display device for communication demonstration control. When there is a need to display multiple information channels, different display devices are usually required to connect to different information sources for display, and the display is controlled by a dedicated operator. The comprehensive information display completed by multiple people requires interpersonal communication beyond the communication topic, which interferes with the fluency of multi-person communication and the continuity of information transmission. Summary of the invention

[0004] The present invention provides a projection screen control method, system and storage medium to solve the technical problem that the coordinated control of the comprehensive display of multi-channel information in the existing multi-person communication scenario interferes with the fluency of multi-person communication and the continuity of information transmission.

[0005] In a first aspect, an embodiment of the present invention provides a screen projection control method, the screen projection control method comprising:

[0006] The first display device receives the projection picture sent by the projection source device through the first data channel, and displays the projection picture in full screen;

[0007] The first display device sends the projection image to the second display device through the second data channel;

[0008] The second display device displays the projection screen in the form of a picture-in-picture window;

[0009] The second display device generates first position data according to the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through a third data channel;

[0010] The first display device confirms the second position data based on the received first position data, and sends it to the projection source device through the first data channel, so that the projection source device updates the projection screen according to the second position data.

[0011] In a second aspect, an embodiment of the present invention further provides a screen projection system, the screen projection system comprising a first display device and a second display device;

[0012] The first display device and the second display device both include:

[0013] one or more processors;

[0014] A memory for storing one or more programs;

[0015] When one or more programs are executed by one or more processors, the screen projection system implements the screen projection image control method as in the first aspect.

[0016] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the projection image control method as in the first aspect.

[0017] In the above-mentioned projection screen control method, system and storage medium, the first display device receives the projection screen sent by the projection source device through the first data channel, and displays the projection screen in full screen; the first display device sends the projection screen to the second display device through the second data channel; the second display device displays the projection screen in the form of a picture-in-picture window; the second display device generates first position data based on the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through the third data channel; the first display device confirms the second position data based on the received first position data, and sends it to the projection source device through the first data channel, so that the projection source device updates the projection screen according to the second position data. The projection screen of the projection source device is forwarded to the second display device through the first display device for display in the form of a picture-in-picture window, and the position data generated by the second display device detected in the picture-in-picture window is forwarded to the projection source device through the first display device. In this way, when the projection source device and the second display device are not directly connected, the second display device can control the projection source device and change the projection screen displayed by the first display device accordingly. In a multi-person communication scenario, the presenter can independently control the comprehensive display of multiple information on multiple display devices on one display device, thereby ensuring the smoothness of multi-person communication and the continuity of information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1A method flow chart of a screen projection control method provided in an embodiment of the present invention.

[0019] Figure 2 A hardware architecture and connection diagram of a second display device provided by an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of comprehensive display of multiple information in a first multi-person communication scenario provided by an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of comprehensive display of multiple information in a second multi-person communication scenario provided by an embodiment of the present invention.

[0022] Figure 5 A data transmission schematic diagram of a screen projection control method provided in an embodiment of the present invention.

[0023] Figure 6 A schematic diagram of the structure of a screen projection control device provided in an embodiment of the present invention.

[0024] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0026] It should be noted that due to space limitations, this application description does not list all optional implementation methods. After reading this application description, those skilled in the art should be able to understand that as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method.

[0027] In multi-person communication spaces such as conferences and teaching that are built based on multiple display devices, such as touch screen-based display devices, the information presentation requirements and the interactive requirements during the communication process are rich and varied. For example, during a multi-person communication process, the speaker displays the focus content through one of the combined display devices, and records and presents the information during the communication process through another of the combined display devices. If the combined display devices are just a combination of communication connections in the same space, there is a certain distance in distribution between the display device that displays the focus content and the other display device that records and presents information during the communication process. During the communication process, a dedicated operator may perform display control. The comprehensive information display completed by multiple people requires interpersonal communication beyond the communication topic. This interpersonal communication interferes with the fluency of multi-person communication centered on the communication topic and the continuity of information transmission.

[0028] In response to the above technical problems, the projection screen of the projection source device is forwarded to the second display device through the first display device for display in the form of a picture-in-picture window, and the position data generated by the second display device detected in the picture-in-picture window is forwarded to the projection source device through the first display device. In this way, when the projection source device and the second display device are not directly connected, the second display device can control the projection display device and change the projection screen displayed by the first display device accordingly. In a multi-person communication scenario, the speaker can independently control the comprehensive display of multiple information on multiple display devices on one display device, thereby ensuring the fluency of multi-person communication and the continuity of information transmission.

[0029] Each embodiment is described in detail below.

[0030] Figure 1 A method flow chart of a screen projection control method provided by an embodiment of the present invention, such as Figure 1 As shown, the screen projection image control method includes:

[0031] Step S110: The first display device receives the projection picture sent by the projection source device through the first data channel, and displays the projection picture in full screen.

[0032] In the embodiment of the present invention, for the sake of clarity, on the basis of the description centered on the first display device, a comprehensive description is further provided in combination with other device bodies involved in the screen projection process.

[0033] The first display device can be any device that can receive projection images generated and sent by other devices based on the desktop or window content, and reproduce and display the projection images sent by other devices. Generally speaking, the first display device is used to enlarge the screen of a device with a smaller screen size and present it to multiple people. In multi-person communication scenarios such as meetings, teaching, discussions, brainstorming, etc., information transmission between multiple participants has been achieved through speech, vision and hearing for a long period of history. This kind of information transmission can give participants a good communication experience, but it is inconvenient for information presentation and information recording during the communication process. The first display device can provide participants in multi-person communication scenarios with good information presentation and information recording experience.

[0034] The first display device mentioned in this solution can be an integrated device that controls the displayed content and realizes human-computer interaction through touch technology, which integrates one or more functions such as a projector, electronic whiteboard, screen, audio, television, and video conferencing terminal.

[0035] Generally speaking, the first display device includes at least one display screen. For example, the first display device is configured with a display screen with a touch function, and the display screen can be a capacitive screen, a resistive screen, or an electromagnetic screen. In this embodiment, further, the user can implement touch operation by touching the display screen with a finger or a stylus, and accordingly, the first display device detects the touch position and responds according to the touch position to implement the touch function. Typically, the first display device is installed with at least one operating system, wherein the operating system includes but is not limited to Android system, Linux system and Windows system.

[0036] The first display device is an interactive device mainly used to support multi-person communication scenarios, and the software therein is mainly used to realize the functional requirements in multi-person communication scenarios. For example, the first display device can install at least one application with a document presentation function, wherein the application can be an application that comes with the operating system; at the same time, an application downloaded from a third-party device or server is also installed. Optionally, in addition to the basic function of displaying the edited content, the program with the document presentation function also has other editing functions in the presentation process, such as inserting tables, inserting pictures, illustrations, drawing tables, drawing graphics, etc., to realize the real-time display of user information input and input content during the information presentation process, thereby improving the information presentation and interaction effect during the document presentation process. For example, the first display device can install at least one application with a whiteboard function, and the user can realize real-time writing, recording, modification and saving in a multi-person communication scenario based on the application. The above mainly describes the functional realization of the first display device when it is used independently.

[0037] In the scenario targeted by this solution, a screen projection system is usually equipped with multiple display devices, at least one of which serves as a first display device. Of course, all display devices can have the ability to serve as the first display device, but due to specific usage requirements, in a multi-person communication activity, only one display device serves as the first display device, and one or more display devices connected to the first display device through a second data channel serve as second display devices.

[0038] At the beginning of multi-person communication, the participant connects a terminal device (such as a laptop) to the first display device through the first data channel as a projection source device. The projection source device can generate a projection screen from the content of the desktop or a window through the first data channel, and send the projection screen to the first display device through the first data channel. When the first display device receives the projection screen sent by the projection source device through the first data channel, it displays the projection screen in full screen. The first data channel can be a Type-C channel or a wireless data channel, wherein the Type-C channel can be a wired channel directly connected through the Type-C interface, and the wireless data channel can be a wireless data channel directly formed through a local area network connection. The first data channel can also be a data channel formed by connecting the projection device with a Type-C interface to the projection source device, and the projection device is connected to the first display device in a wireless form, without specific limitation.

[0039] Step S120: The first display device sends the projected screen to the second display device through the second data channel, so that the second display device displays the projected screen in the form of a picture-in-picture window.

[0040] The first display device sends the projection screen to the second display device through the second data channel. After the second display device receives the projection screen through the second data channel, it displays the projection screen in the form of a picture-in-picture window. In the second display device, the other projection screen connected is mainly displayed in full screen form. The other projection screen needs to be displayed as the focus information of the multi-person communication scene, or the information recorded during the multi-person communication process is recorded using an installed application with a whiteboard function. The projection screen received by the second display device from the first display device is mainly displayed as a preview, and is displayed in the form of a picture-in-picture window accordingly, which meets the basic requirement of being able to operate the picture-in-picture window. The display of the projection screen corresponding to the projection source device is actually completed by the first display device. Of course, the second display device can also display local information as the focus information of the multi-person communication scene, and the specific source of the focus information does not affect the implementation of this solution. The second data channel and the third data channel can specifically be a reused Type-C channel; or the second data channel is an HDMI channel, and the third data channel is a touch channel.

[0041] Step S130: the first display device receives first position data sent by the second display device through the third data channel, where the first position data is position data generated by the second display device according to a touch operation applied to the picture-in-picture window.

[0042] For the second display device, all touch operations within the display range can be received through the touch module, but how to specifically process the position data generated by the touch operation needs to be determined according to the position range, that is, the position data acting within the range of the picture-in-picture window needs to be sent to the device of the source of the projection screen in the picture-in-picture window for response, and the position data acting outside the range of the picture-in-picture window needs to be responded by the device that generates the focus information (which may be the second display device or the second source device connected to the second display device). In the embodiment of the present application, for the improvement of the picture-in-picture window, the position data generated by the touch operation acting on the picture-in-picture window, that is, the first position data, is sent to the first display device through the third data channel, and the first display device will correspondingly receive the position data sent by the second display device through the third data channel.

[0043] Step S140: The first display device sends the second position data to the projection source device through the first data channel, so that the projection source device updates the projection screen according to the second position data, and the second position data is position data confirmed based on the first position data.

[0044] For the first display device, what is displayed in full screen is not the picture generated by its own processor, and accordingly it does not need to respond directly according to the first position data. Instead, it confirms the second position data according to the first position data, and sends the second position data to the projection source device through the first data channel, so that the projection source device updates the projection picture according to the second position data.

[0045] In the above position data processing process, in order to ensure the final normal response of the screen projection source device, the position data that is compatible with the screen parameters of the screen projection source device should be accurately obtained in the end, because the position data is transmitted through multiple devices, and the references for position expression of multiple devices may be different. Therefore, there needs to be a clear reference between the devices before and after the transmission process for position conversion during the transmission process. The first position data is the position data with reference to the screen parameters of the first display device, or the position data with reference to the window parameters of the picture-in-picture window; the second position data is the position data with reference to the screen parameters of the screen projection source device, or the position data with reference to the screen parameters of the first display device. It is equivalent to that in the process of position data transmission, each device can generate it with reference to its own screen parameters or window parameters. After establishing the connection channel, the receiving device can obtain the screen parameters or window parameters of the previous device in the direction of position data transmission, thereby completing the conversion of position data by itself. The specific screen parameters or window parameters can be confirmed in the form of EDID (Extended display identification data) when establishing the connection. Of course, it can also be that each device generates position data with the screen parameters or window parameters of the next device, and the next device can directly use or process it after receiving it. Alternatively, all conversions are completed on the first display device, which is also an optional implementation. The corresponding screen parameters or window parameters can be transmitted through the corresponding data channel according to actual needs.

[0046] In a specific implementation, the second display device generates first position data based on the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through a third data channel, including: the second display device obtains the current position data generated by the current touch operation, and confirms the window area where the current touch operation acts based on the current position data; when the second display device confirms that the window area where the current touch operation acts is the picture-in-picture window area, the current position data is used as the first position data, or the current position data is converted with reference to the window parameters of the picture-in-picture window to obtain the first position data; the first position data is sent to the first display device through the third data channel. The specific process of generating the first position data has been described in the previous text, and the size relationship of the specific screen or window can be converted accordingly. Similarly, the first display device confirms the second position data based on the received first position data, including: the first display device converts the first position data with reference to the window parameters of the first display device, or with reference to the screen parameters of the projection source device to obtain the second position data.

[0047] In a more specific position data transmission control method, such as Figure 2As shown, the second display device 22 includes a channel switching switch 223 and a main control chip 221; the first input end of the main control chip 221 is connected to the current position data, and the current position data is obtained by touch detection by the touch module 222; the first output end of the main control chip 221 is connected to the first input end of the channel switching switch 223, and is used to output the position data generated according to the current position data; one of the multiple output ends of the channel switching switch 223 is connected to the third data channel 201; the second output end of the main control chip 2221 is connected to the second input end of the channel switching switch 223, and is used to enable an output end of the channel switching switch 223 according to the window area affected by the current position data; accordingly, sending the first position data to the first display device 21 through the third data channel 201 includes: sending a first enable signal to the channel switching switch 223 to enable the output end corresponding to the third data channel. Based on this channel control, the first position data can only be sent to the first display device.

[0048] In addition to the first display device 21 being connected to the screen projection device (i.e., the screen projection source device 11), the second display device 22 can also be connected to the screen projection device (i.e., the second source device), specifically, one of the multiple output ends of the channel switching switch 223 is connected to the fourth data channel 202, and the fourth data channel 202 is used to connect to the second source device; accordingly, the second display device 22 obtains the current position data generated by the current touch operation, and after confirming the window area where the current touch operation is applied according to the current position data, it also includes: when the second display device 22 is connected to the second source device, when confirming that the window area where the current touch operation is applied is an area outside the picture-in-picture window area, it sends a second enable signal to the channel switching switch 223 to enable the output end corresponding to the fourth data channel 202. In this way, the second source device can be controlled by the second display device 22. For the second source device, the principle of implementing touch response is the same as that of the screen projection source device, and will not be repeated here. In addition, the channel switching switch 223 may also have more data channels, such as the fifth data channel 224, which is directly connected to the processor or OPS (Open Pluggable Specification) host inside the second display device 22, so that the position data can be processed and responded by the internal processor or OPS host without an external projection device. It should be understood that the second display device 22 is only a device confirmed based on the specific connection method in the embodiment of the present application, rather than a specific device. For example, when another display device in a conference room is in a connection state in the second display device 22 in another application scenario, the other display device is actually also the second display device 22. It should be noted that the channel switching switch 223 only performs channel enable control on the sending process of the position data, and the switching of the channel switching switch 223 does not affect the transmission status of the projection screen.

[0049] Please refer to Figure 3 and Figure 4 , which provides two specific application scenarios, in which the connection relationship between the screen projection source device 11, the first display device 21 and the second display device 22 is the same, Figure 3 The second display device 22 generates focus information itself, Figure 4 The second display device 22 performs display according to the projection of the second source device 12, that is, Figure 4 The focus information in is the screen generated by the projection of the second source device 12. The picture-in-picture window 222 in the second screen 221 of the second display device 22 displays the projection screen received from the first display device 21. The first screen 211 of the first display device 21 displays the projection screen received from the projection source device 11 in full screen. The projection screen sent from the first display device 21 to the second display device 22 is also the projection screen received from the projection source device 11. In the embodiment of the present application, the projection screen sent between the projection source device 11 and the first display device 21 may be different from the projection screen sent between the first display device 21 and the second display device 22 in data format, but the specific screen content is the same. In the embodiment of the present application, both are defined as projection screens based on the consistency of the screen content, and the two are defined differently according to the data format or the difference in the sending and receiving subjects, without departing from the design idea of ​​the present solution.

[0050] like Figure 3 As shown, it is assumed that the content to be presented in the current communication scenario includes focus information displayed in full screen on the second display device 22, and a video for auxiliary explanation of the focus information, wherein the video is played on the projection source device 11 and projected to the first display device 21 for full screen display. While the first display device 21 displays the video in full screen in the form of projection, it is also sent to the second display device 22 for display in the picture-in-picture window 222. During the multi-person communication process, the speaker needs to pause the content displayed in the video for detailed explanation. The speaker can click the picture-in-picture window 222 on the second display device 22. The position data of the click operation will eventually be forwarded to the projection source device 11 through the first display device 21 to respond with pausing the video. During this operation, although the speaker needs to operate the projection source device 11 to which the second display device 22 is not directly connected, it does not require the cooperation of others or switching of the operation object. The operation can be completed quickly on the second display device 22. There is no need to change the display state of the focus information during the operation, which ensures the fluency of multi-person communication and the continuity of information transmission from all aspects. In the above processing, the data transmission process between the screen projection source device 11, the first display device 21 and the second display device 22 can refer to Figure 5 .

[0051] In the above-mentioned projection screen control method, the first display device receives the projection screen sent by the projection source device through the first data channel, and displays the projection screen in full screen; the first display device sends the projection screen to the second display device through the second data channel; the second display device displays the projection screen in the form of a picture-in-picture window; the second display device generates first position data based on the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through the third data channel; the first display device confirms the second position data based on the received first position data, and sends it to the projection source device through the first data channel, so that the projection source device updates the projection screen according to the second position data. The projection screen of the projection source device is forwarded to the second display device through the first display device for display in the form of a picture-in-picture window, and the position data generated by the second display device detected in the picture-in-picture window is forwarded to the projection source device through the first display device. In this way, when the projection source device and the second display device are not directly connected, the second display device can control the projection display device and change the projection screen displayed by the first display device accordingly. In a multi-person communication scenario, the presenter can independently control the comprehensive display of multiple information on multiple display devices on one display device, thereby ensuring the smoothness of multi-person communication and the continuity of information transmission.

[0052] Based on the above embodiments, an embodiment of the present invention provides a screen projection control method, the screen projection control method comprising:

[0053] The first display device receives the projection picture sent by the projection source device through the first data channel, and displays the projection picture in full screen;

[0054] The first display device sends the projection image to the second display device through the second data channel;

[0055] The second display device displays the projection screen in the form of a picture-in-picture window;

[0056] The second display device generates first position data according to the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through the second data channel;

[0057] The first display device confirms the second position data based on the received first position data, and sends it to the projection source device through the first data channel, so that the projection source device updates the projection screen according to the second position data.

[0058] On the basis of the above embodiment, the second display device generates first position data according to the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through the third data channel, including:

[0059] The second display device obtains current position data generated by the current touch operation, and determines the window area affected by the current touch operation according to the current position data;

[0060] When the second display device determines that the window area affected by the current touch operation is a picture-in-picture window area, the current position data is used as the first position data, or the current position data is converted with reference to the window parameters of the picture-in-picture window to obtain the first position data;

[0061] The first position data is sent to the first display device through the third data channel.

[0062] On the basis of the above embodiment, the first display device confirms the second position data according to the received first position data, including:

[0063] The first display device converts the first position data with reference to the window parameters of the first display device or with reference to the screen parameters of the projection source device to obtain the second position data.

[0064] On the basis of the above embodiment, the second display device includes a channel switching switch and a main control chip; the first input end of the main control chip is connected to the current position data, and the first output end of the main control chip is connected to the first input end of the channel switching switch, and is used to output the position data generated according to the current position data; one of the multiple output ends of the channel switching switch is connected to the third data channel; the second output end of the main control chip is connected to the second input end of the channel switching switch, and is used to enable an output end of the channel switching switch according to the window area affected by the current position data;

[0065] Accordingly, sending the first position data to the first display device through the third data channel includes:

[0066] Sending a first enable signal to the channel switching switch to enable an output terminal corresponding to the third data channel. Based on the above embodiment, one of the multiple output terminals of the channel switching switch is connected to a fourth data channel, and the fourth data channel is used to access the second source device;

[0067] Correspondingly, after the second display device obtains the current position data generated by the current touch operation and confirms the window area where the current touch operation is applied according to the current position data, it further includes:

[0068] When the second display device is connected to the second source device, when it is confirmed that the window area affected by the current touch operation is outside the picture-in-picture window area, the second display device sends a second enable signal to the channel switching switch to enable the output end corresponding to the fourth data channel.

[0069] Based on the above embodiment, the first data channel is a Type-C channel or a wireless data channel.

[0070] Based on the above embodiment, the second data channel and the third data channel are multiplexed Type-C channels.

[0071] Based on the above embodiment, the second data channel is an HDMI channel, and the third data channel is a touch channel.

[0072] The projection screen control method in the embodiment of the present invention is based on the projection screen control method in the previous embodiment. Each display device involved in the projection process is taken as a parallel subject, and the main execution content of each display device is described. The specific implementation process has been described in the previous embodiment and will not be repeated here.

[0073] Figure 6 A schematic diagram of the structure of a screen projection control device provided by an embodiment of the present invention. Figure 6 The projection screen control device includes: a receiving and displaying unit 31, a screen forwarding unit 32, a position receiving unit 33 and a position forwarding unit 34.

[0074] A receiving and displaying unit 31 is used to receive a projection picture sent by a projection source device through a first data channel, and to display the projection picture in full screen; a picture forwarding unit 32 is used to send the projection picture to a second display device through a second data channel, so that the second display device displays the projection picture in the form of a picture-in-picture window; a position receiving unit 33 is used to receive first position data sent by the second display device through a second data channel, the first position data being position data generated by the second display device according to a touch operation acting on the picture-in-picture window; a position forwarding unit 34 is used to send the second position data to the projection source device through the first data channel, so that the projection source device updates the projection picture according to the second position data, the second position data being position data confirmed based on the first position data.

[0075] Based on the above embodiments, the first position data is position data with reference to the screen parameters of the first display device, or position data with reference to the window parameters of the PIP window; the second position data is position data with reference to the screen parameters of the projection source device, or position data with reference to the screen parameters of the first display device.

[0076] Based on the above embodiment, the first data channel is a Type-C channel or a wireless data channel.

[0077] Based on the above embodiment, the second data channel and the third data channel are multiplexed Type-C channels.

[0078] Based on the above embodiment, the second data channel is an HDMI channel, and the third data channel is a touch channel.

[0079] The projection screen control device provided in an embodiment of the present invention is included in an electronic device (specifically as a first display device), and can be used to execute any projection screen control method provided in the above embodiments, and has corresponding functions and beneficial effects.

[0080] It is worth noting that in the embodiment of the above-mentioned projection image control device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0081] An embodiment of the present invention also provides a projection screen control device, which includes: a receiving and displaying unit, used for a first display device to receive a projection screen sent by a projection source device through a first data channel, and to display the projection screen in full screen; a screen forwarding unit, used for the first display device to send the projection screen to a second display device through a second data channel; a screen display unit, used for the second display device to display the projection screen in the form of a picture-in-picture window; a generating and sending unit, used for the second display device to generate first position data according to position data generated by a touch operation acting on the picture-in-picture window, and send it to the first display device through the second data channel; a position forwarding unit, used for the first display device to confirm second position data according to the received first position data, and send it to the projection source device through the first data channel, so that the projection source device updates the projection screen according to the second position data.

[0082] On the basis of the above embodiment, the second display device generates first position data according to the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through the third data channel, including:

[0083] The second display device obtains current position data generated by the current touch operation, and determines the window area affected by the current touch operation according to the current position data;

[0084] When the second display device determines that the window area affected by the current touch operation is a picture-in-picture window area, the current position data is used as the first position data, or the current position data is converted with reference to the window parameters of the picture-in-picture window to obtain the first position data;

[0085] The first position data is sent to the first display device through the third data channel.

[0086] On the basis of the above embodiment, the first display device confirms the second position data according to the received first position data, including:

[0087] The first display device converts the first position data with reference to the window parameters of the first display device or with reference to the screen parameters of the projection source device to obtain the second position data.

[0088] On the basis of the above embodiment, the second display device includes a channel switching switch and a main control chip; the first input end of the main control chip is connected to the current position data, and the first output end of the main control chip is connected to the first input end of the channel switching switch, and is used to output the position data generated according to the current position data; one of the multiple output ends of the channel switching switch is connected to the third data channel; the second output end of the main control chip is connected to the second input end of the channel switching switch, and is used to enable an output end of the channel switching switch according to the window area affected by the current position data;

[0089] Accordingly, sending the first position data to the first display device through the third data channel includes:

[0090] A first enable signal is sent to the channel switching switch to enable an output end corresponding to the third data channel.

[0091] On the basis of the above embodiment, one of the multiple output ends of the channel switching switch is connected to the fourth data channel, and the fourth data channel is used to access the second source device;

[0092] Correspondingly, after the second display device obtains the current position data generated by the current touch operation and confirms the window area where the current touch operation is applied according to the current position data, it further includes:

[0093] When the second display device is connected to the second source device, when it is confirmed that the window area affected by the current touch operation is outside the picture-in-picture window area, the second display device sends a second enable signal to the channel switching switch to enable the output end corresponding to the fourth data channel.

[0094] Based on the above embodiment, the first data channel is a Type-C channel or a wireless data channel.

[0095] Based on the above embodiment, the second data channel and the third data channel are multiplexed Type-C channels.

[0096] Based on the above embodiment, the second data channel is an HDMI channel, and the third data channel is a touch channel.

[0097] The projection screen control device provided in an embodiment of the present invention is included in an electronic device of the projection screen system (specifically as a first display device and a second display device), and can be used to execute any projection screen control method provided in the above embodiments, and has corresponding functions and beneficial effects.

[0098] It is worth noting that in the embodiment of the above-mentioned projection image control device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0099] Figure 7 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 7 As shown, the electronic device includes a processor 310, a memory 320, an input device 330, an output device 340, and a communication device 350; the number of the processor 310 in the electronic device can be one or more. Figure 7 A processor 310 is taken as an example; the processor 310, the memory 320, the input device 330, the output device 340 and the communication device 350 in the electronic device can be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0100] The memory 320, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the projection screen control method in the embodiment of the present invention. The processor 310 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 320, that is, implements the above-mentioned projection screen control method.

[0101] The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include a memory remotely arranged relative to the processor 310, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0102] The input device 330 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 340 may include a display device such as a display screen.

[0103] The above-mentioned electronic device includes a projection screen control device, which can be used to execute any projection screen control method and has corresponding functions and beneficial effects.

[0104] An embodiment of the present invention also provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to perform relevant operations in the screen projection image control method provided in any embodiment of the present application, and have corresponding functions and beneficial effects.

[0105] Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products.

[0106] Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0108] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0109] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0110] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A screen projection control method, characterized in that: include: The first display device receives the projection screen sent by the projection source device through the first data channel, and displays the projection screen in full screen; The first display device sends the projection image to the second display device through the second data channel; The second display device displays the projection screen in the form of a picture-in-picture window; The second display device generates first position data according to the position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through a third data channel; The first display device confirms the second position data based on the received first position data, and sends it to the projection source device through the first data channel, so that the projection source device updates the projection screen according to the second position data.

2. The screen projection control method according to claim 1, characterized in that: The second display device generates first position data according to position data generated by the touch operation acting on the picture-in-picture window, and sends the first position data to the first display device through the third data channel, including: The second display device acquires current position data generated by the current touch operation, and determines the window area affected by the current touch operation according to the current position data; When the second display device determines that the window area affected by the current touch operation is a picture-in-picture window area, the current position data is used as the first position data, or the current position data is converted with reference to the window parameters of the picture-in-picture window to obtain the first position data; The second display device sends the first position data to the first display device through the third data channel.

3. The screen projection control method according to claim 2, characterized in that: The first display device confirms the second position data according to the received first position data, including: The first display device converts the first position data with reference to the window parameters of the first display device or with reference to the screen parameters of the projection source device to obtain the second position data.

4. The screen projection control method according to claim 2, characterized in that: The second display device includes a channel switching switch and a main control chip; a first input terminal of the main control chip is connected to the current position data, and a first output terminal of the main control chip is connected to the first input terminal of the channel switching switch, for outputting position data generated according to the current position data; One of the multiple output ends of the channel switching switch is connected to the third data channel; the second output end of the main control chip is connected to the second input end of the channel switching switch, and is used to enable an output end of the channel switching switch according to the window area affected by the current position data; Correspondingly, the second display device sends the first position data to the first display device through the third data channel, including: The main control chip sends a first enable signal to the channel switching switch to enable an output end corresponding to the third data channel.

5. The screen projection control method according to claim 4, characterized in that: One of the multiple output ends of the channel switching switch is connected to a fourth data channel, and the fourth data channel is used to access a second source device; Correspondingly, after the second display device obtains the current position data generated by the current touch operation and confirms the window area where the current touch operation acts according to the current position data, it further includes: When the second display device is connected to the second source device, when it is confirmed that the window area affected by the current touch operation is an area outside the picture-in-picture window area, the second display device sends a second enable signal to the channel switching switch to enable the output end corresponding to the fourth data channel.

6. The screen projection control method according to any one of claims 1 to 5, characterized in that: The first data channel is a Type-C channel or a wireless data channel.

7. The screen projection control method according to any one of claims 1 to 5, characterized in that: The second data channel and the third data channel are multiplexed Type-C channels.

8. The screen projection control method according to any one of claims 1 to 5, characterized in that: The second data channel is an HDMI channel, and the third data channel is a touch channel.

9. A screen projection system, characterized in that: comprising a first display device and a second display device; The first display device and the second display device both include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the screen projection system implements the screen projection image control method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the projection image control method as described in any one of claims 1 to 8 is implemented.

Images