Terminal screen projection method and system, first terminal and second terminal
The terminal screen projection method synchronizes touch operations and ensures consistent annotation and deletion across terminals, addressing network-dependent clarity and interaction delays, enhancing display quality and security.
Patent Information
- Application Number
- CN202310800468.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-30
- Publication Date
- 2024-12-31
AI Technical Summary
Existing screen projection methods suffer from network-dependent clarity issues, resolution inconsistencies, interaction delays, and inability to annotate or delete projected files on large screens, particularly in multi-terminal setups.
A method for terminal screen projection that synchronizes touch operations across terminals by converting touch parameters, ensuring consistent aspect ratios, and allowing real-time annotation and deletion of projected files, even in poor network conditions.
Improves display clarity and interaction efficiency by synchronizing touch operations and ensuring consistent annotation across different resolutions and screen sizes, while maintaining file security by allowing real-time deletion of projected files.
Smart Images
Figure CN119225669A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of screen projection display technology, and in particular to a terminal screen projection method, a first terminal, a second terminal, a terminal screen projection system, and a storage medium. Background Art
[0002] The purpose of the background technology description provided here is to give an overall background of the present application. The statements in this section merely provide background related to the present application and do not necessarily constitute prior art.
[0003] Currently, in addition to providing users with handwriting and display functions, all-in-one conference machines are often used in meetings for screen projection, where users project the content on their mobile phones onto the large screen for viewing, or project the content on their computers onto the large screen.
[0004] However, the traditional screen projection method is a wired connection, which has certain limitations on the connection distance and is not rich in functions, mainly focusing on screen copying and expansion. The commonly used wireless screen projection method now has strong functional scalability and supports functions such as screen projection, file projection, and reverse control. However, the existing screen projection solutions have the following defects when projecting files:
[0005] The solution of selecting pictures for mirroring is highly dependent on network conditions. Due to the influence of network conditions, there are defects such as poor clarity and low resolution, which will cause the projected file to become blurry. Moreover, when annotating, only the local file is annotated and then projected on the all-in-one machine. There are defects such as high delay when switching files.
[0006] In the solution of transferring pictures to the large screen for display on the large screen, the local device can only send the end control command, and cannot perform corresponding control operations on the pictures on the large screen, nor can it annotate and delete the projected files on the large screen. Moreover, after projecting the files, the user must look at the large screen content and perform corresponding local buttons when operating, which brings inconvenience in interaction. Furthermore, due to the inconsistency of screen resolution and screen size at both ends, the coordinates do not correspond, and the projected pictures cannot be controlled for annotation operations.
[0007] Therefore, a new terminal screen projection method is urgently needed to solve the above-mentioned defects currently existing. Summary of the invention
[0008] In view of the above problems, the present application proposes a terminal screen projection method, a first terminal, a second terminal, a terminal screen projection system and a storage medium, which at least solves the defects such as time delay during multi-terminal interaction and the inability to annotate and delete the projection file on the large screen.
[0009] In a first aspect of the present application, a terminal screen projection method is provided, which is applied to a first terminal, where the first terminal is communicatively connected with a second terminal, and the method includes:
[0010] In response to a screen projection operation on a target file, sending the target file to a second terminal so as to display a copy of the target file in the second terminal;
[0011] When the target file is in an opened state, in response to a first touch operation on the target file, touch parameters of the first touch operation are sent to the second terminal, so that the second terminal converts the touch parameters into first target touch parameters based on the second terminal, and performs corresponding operations on the copy of the target file according to the first target touch parameters.
[0012] Furthermore, the touch parameters include:
[0013] The type of touch operation and / or the coordinates of the touch operation.
[0014] Furthermore, the aspect ratio of the target file displayed in the first terminal is the same as the aspect ratio of the copy of the target file displayed in the second terminal.
[0015] Furthermore, it also includes:
[0016] In response to ending the screen projection operation for the target file, controlling the second terminal to delete the copy of the target file according to a preset deletion policy.
[0017] Furthermore, it also includes:
[0018] In response to an opening operation on the target file, the second terminal is controlled to open a copy of the target file.
[0019] A second aspect of the present application provides a terminal screen projection method, which is applied to a second terminal, where the second terminal is communicatively connected to the first terminal, and the method includes:
[0020] In response to receiving a copy of the target file for screen projection sent by the first terminal, displaying the copy of the target file;
[0021] In response to receiving the touch parameter sent by the first terminal, the touch parameter is converted into a first target touch parameter based on the second terminal, and a corresponding operation is performed on the copy of the target file according to the first target touch parameter.
[0022] Furthermore, the operation of converting the touch parameters includes:
[0023] Acquire a first display area of the target file in the first terminal, and a second display area of the copy of the target file in the second terminal;
[0024] determining a zoom ratio according to the first display area and the second display area;
[0025] The target touch parameter is determined according to the scaling ratio and the touch parameter.
[0026] Furthermore, it also includes:
[0027] In response to a second touch operation on the copy of the target file, touch parameters of the second touch operation are sent to the first terminal, so that the first terminal converts the touch parameters into second target touch parameters based on the first terminal, and performs corresponding operations on the target file according to the second target touch parameters.
[0028] Furthermore, the copy of the target file is displayed in full screen mode in the second terminal.
[0029] A third aspect of the present application provides a terminal screen projection method, which is applied to a second terminal, where the second terminal is communicatively connected with a plurality of first terminals, and the method includes:
[0030] In response to receiving a copy of a second target file for screen projection sent by the first terminal, performing a cache operation on the copy of the second target file, and sending a waiting screen projection message to the first terminal;
[0031] In response to ending the screen projection operation for the current screen projection file, a start screen projection message is sent to the first terminal to wait for receiving touch parameters for the second target file sent by the first terminal.
[0032] Furthermore, it also includes:
[0033] In response to an opening operation on the copy of the second target file, a screen projection prompt message is sent to the first terminal so that the first terminal opens the second target file.
[0034] Further, after sending the waiting screen projection message to the first terminal, the method further includes:
[0035] In response to the screen projection operation for the copy of the second target file, control the first terminal to exit the screen projection operation on the current screen projection file; and display the copy of the second target file in full screen mode; and send a start screen projection message to the first terminal to wait for receiving the touch parameters for the second target file sent by the first terminal.
[0036] Further, after sending the waiting screen projection message to the first terminal, the method further includes:
[0037] In response to the screen projection operation for the copy of the second target file, control another first terminal to execute the end of the screen projection operation, and the other terminal is the first terminal that sends the current screen projection file; and display the copy of the second target file in full screen mode; and send a start screen projection message to the first terminal to wait for receiving the touch parameters for the second target file sent by it.
[0038] The fourth aspect of the present application provides a first terminal, characterized in that it includes a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the terminal screen projection method applied to the first terminal as described above is executed.
[0039] The fifth aspect of the present application provides a second terminal, characterized in that it includes a memory and a processor, and a computer program is stored in the memory. When the computer program is executed by the processor, the terminal screen projection method applied to the second terminal as described above is executed.
[0040] A sixth aspect of the present application provides a terminal screen projection system, comprising the second terminal as described above and the first terminal as described above.
[0041] The seventh aspect of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by one or more processors to implement the steps of the terminal screen projection method as described above.
[0042] Compared with the prior art, the technical solution of the present application has the following advantages or beneficial effects:
[0043] When performing screen projection, the file to be projected is transferred to the all-in-one large-screen terminal used for screen projection and displayed on the all-in-one large-screen terminal, which solves the problem of screen projection operation delay caused by poor network environment during screen projection. At the same time, the user opens the file at the screen projection file sending end and can annotate the file by touch operation. The touch operation is synchronously annotated on the projection file in the all-in-one large-screen terminal after algorithm conversion; the projection file in the all-in-one large-screen terminal can be deleted after the screen projection is completed, ensuring the security and confidentiality of the file. At least the following pain points of users are solved: when the file is projected, the display effect of the file is improved, avoiding the related impact caused by poor network; at the same time, it also solves the problem of annotation consistency of projection files at different resolutions and screen sizes and the security of confidential files. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in the relevant field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0045] It should also be noted that, for the convenience of description, only the parts related to the present disclosure are shown in the drawings. The drawings constituting a part of the specification of this application are used to provide a further understanding of the present application. The illustrative embodiments and their descriptions in this application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0046] Figure 1 This is a schematic diagram of a screen projection scenario;
[0047] Figure 2 A flowchart of a terminal screen projection method applied to a first terminal provided in an embodiment of the present application;
[0048] Figure 3 A schematic diagram of a picture display area provided in an embodiment of the present application;
[0049] Figure 4 A flowchart of an annotation method provided in an embodiment of the present application;
[0050] Figure 5 A flowchart of a terminal screen projection method applied to a second terminal provided in an embodiment of the present application;
[0051] Figure 6 A flowchart of a terminal screen projection method provided in an embodiment of the present application;
[0052] Figure 7 A flowchart of another terminal screen projection method provided in an embodiment of the present application;
[0053] Figure 8 A flowchart of another terminal screen projection method applied to a second terminal provided in an embodiment of the present application. DETAILED DESCRIPTION
[0054] The following will describe the implementation methods of the present application in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present application applies technical means to solve technical problems and achieve corresponding technical effects can be fully understood and implemented accordingly. The embodiments of the present application and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the protection scope of the present application.
[0055] It should be clear that the embodiments described below are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by technicians in the relevant field without making creative work are within the scope of protection of the present application.
[0056] Embodiment 1
[0057] This embodiment provides a terminal screen projection method, which can be applied to / used in a first terminal (the sending end of the target file). The first terminal sends the target file to be projected to a second terminal (the receiving end of the target file) for display in the second terminal, wherein the first terminal can communicate with the second terminal via wired or wireless means.
[0058] Optionally, the first terminal includes a smart phone, a tablet computer or a personal computer, and the second terminal includes a large-screen terminal such as an all-in-one conference machine.
[0059] Figure 1 For a screen projection scenario diagram, please refer to Figure 1 :When casting the screen, the sending end and the receiving end are usually connected to the same local area network. Among them, the picture casting function usually refers to casting the local picture selected by the sending end to the receiving end for large-screen display. The usual casting process is: select the picture locally and open it, and after selecting the casting, synchronize the local picture to the receiving end for display. That is, the local picture of the sending end is cast, and the screen content of the sending end is synchronized on the receiving end.
[0060] As an example, Figure 2 A flowchart of a terminal screen projection method applied to a first terminal provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method disclosed in this embodiment is applied to a first terminal, comprising the following steps:
[0061] Step 210: In response to a screen projection operation for a target file, the target file is sent to a second terminal so as to display a copy of the target file in the second terminal.
[0062] The target file described in this embodiment refers to the file to be projected, and the type of the file includes other types of files such as pictures or PPT (PowerPoint) files. After the target file is sent to the second terminal, a copy of the target file is saved in the second terminal.
[0063] As an example, the specific process of the sending end sending the picture for screen projection to the receiving end may include:
[0064] The sender sends a picture to the receiver;
[0065] The receiving end downloads the picture according to the picture address in the local area network. After completing the download and opening the picture, the receiving end sends a first notification message to the sending end through UDP (User Datagram Protocol, UDP for short) to inform the sending end that the picture has been opened. To ensure reliability, an ack flag (Acknowledge character) and the picture address can be added to the first notification message;
[0066] After receiving the first notification message, the sending end verifies whether the image address is correct. When the image address is verified, the sending end can open the image and obtain control of the receiving end. Then, the receiving end can perform operations such as annotation and send a second notification message to the receiving end. The second notification message may be accompanied by an ack mark of receipt.
[0067] After receiving the second notification message, the receiving end checks the legitimacy of the ack flag. When the ack flag passes the verification, the connection is established, allowing the control commands and touch events subsequently sent by the sending end to be responded to.
[0068] As an example, the file to be projected in the first terminal is sent to the second terminal, so that a copy of the file is directly displayed in the second terminal, thereby avoiding the problem of freezes and delays caused by unstable network environment and blurry projection files.
[0069] For example, when the file to be projected is a picture, the user selects a local picture in the first terminal, and selects projection to start transmitting the picture to the second terminal. The picture is first compressed and then transmitted to the receiving end through the TCP protocol (Transmission Control Protocol, referred to as TCP protocol). After decompressing the picture, the receiving end can open it for display (a copy of the picture is saved at the receiving end).
[0070] It should be noted that the second terminal may cache the currently displayed image to improve rendering efficiency when the image is displayed next time.
[0071] In some embodiments, an aspect ratio of the target file displayed in the first terminal is the same as an aspect ratio of the copy of the target file displayed in the second terminal.
[0072] It should be noted that the aspect ratio of the projection file displayed on the screen at the sending end of the target file and the receiving end of the target file is the same to prevent problems such as stretching or deformation of the projection file during screen projection.
[0073] For example, if the aspect ratio of a picture is 9:16, then the aspect ratio of the picture displayed on both the sending end and the receiving end is 9:16; on the receiving end, the picture can be controlled to be displayed in the maximum displayable area on the receiving end according to the aspect ratio of the picture and the resolution of the receiving end display.
[0074] For details, please refer to Figure 3 , Figure 3 A schematic diagram of a picture display area provided in an embodiment of the present application, in Figure 3 The following provides three display solutions for images on mobile phones, tablet computers, and all-in-one conference machines by way of examples.
[0075] In some embodiments, it also includes:
[0076] In response to an opening operation on the target file, the second terminal is controlled to open a copy of the target file.
[0077] Optionally, an opening operation of the target file is performed at the first terminal, and a file opening message is sent to the second terminal to control the second terminal to open a copy of the target file.
[0078] Step 220: When the target file is in an opened state, in response to a first touch operation on the target file, touch parameters of the first touch operation are sent to the second terminal, so that the second terminal converts the touch parameters into first target touch parameters based on the second terminal, and performs corresponding operations on the copy of the target file according to the first target touch parameters.
[0079] It should be noted that the first touch operation described in the present application refers to a touch operation performed on a target file in a first terminal (a sending end of the target file).
[0080] In order to avoid the problem of interaction jamming caused by touch delay, the touch parameters of the first terminal need to be sent to the second terminal as quickly as possible, or they can be sent in real time.
[0081] For example, in the first terminal, the touch operation of the relevant gesture is monitored by monitoring the corresponding callback method. Whenever the touch parameters of the touch operation are monitored to change (that is, whenever the callback method is called once), the current touch parameters are sent to the second terminal; that is, in response to monitoring the change of the touch parameters of the first touch operation, the current touch parameters of the first touch operation are sent to the second terminal, and the touch parameters are converted into target touch parameters based on the second terminal by the second terminal, and then the second terminal is controlled to perform corresponding operations on the copy of the target file according to the target touch parameters.
[0082] In some embodiments, the touch parameters include:
[0083] The type of touch operation and / or the coordinates of the touch operation.
[0084] As an example, the receiving end can respond to multiple gesture types of the sending end and perform corresponding operations according to the touch coordinates corresponding to each gesture type. The type of touch operation includes the category or type of gesture, such as: pinch gesture, rotation gesture, long press gesture, single / double click gesture, translation gesture and sliding gesture, etc.; the coordinates of the touch operation include the coordinates of the current gesture in the View coordinate system of the first terminal.
[0085] Optionally, in response to a touch operation on the target file, the first terminal is controlled to perform a corresponding operation on the target file according to the touch parameter.
[0086] It is understandable that, in response to the touch operation on the target file in the first terminal, a corresponding operation is performed on the target file according to the touch parameters of the touch operation. For example, when annotating the target file in the first terminal, not only can the annotation result be directly displayed in the first terminal, but the annotation can also be performed and displayed synchronously in the second terminal, thereby realizing the synchronous annotation function in the first terminal and the second terminal.
[0087] Optionally, when the first terminal performs a zoom-in or zoom-out operation on the projected file through a corresponding gesture, the coordinates of the current touch operation are converted into target coordinates in the second terminal, and the corresponding zoom-in or zoom-out operation is performed on the copy of the projected file in the second terminal according to the current zoom-in or zoom-out operation (type of touch operation) and the target coordinates.
[0088] For the specific annotation process, please refer to Figure 4 , Figure 4 A flowchart of an annotation method provided in an embodiment of the present application.
[0089] As an example, the picture in the first terminal is projected here, taking the first terminal as a smart phone (sender) and the second terminal as an all-in-one machine (receiver) as an example. For the function of annotation synchronization, since the display area of the smart phone is inconsistent with the display area of the all-in-one machine, the touch coordinates in the smart phone cannot be directly transplanted to the all-in-one machine. The solution in this embodiment is based on the fact that the picture is displayed in equal proportions, so the picture display aspect ratio of the sending end and the receiving end is consistent, so it is necessary to convert the local touch coordinates into the coordinates of the receiving end all-in-one machine through calculation, and generate a new touch event to pass it to the all-in-one machine and implement the picture annotation function in the view layer, while solving the delay problem of displaying annotations at the sending end and the receiving end. Based on this, the sensory experience of users watching on both ends is improved. After the screen projection is completed, you can also choose to let the receiving end delete the local picture, or you can delete it at a scheduled time, so the security of the projected picture can be better guaranteed.
[0090] For example, when the second terminal annotates a copy of the projected picture, relevant annotation operations can be performed on the picture based on a series of coordinates of the pan gesture (or other gesture types) during the sliding process (during the annotation process, you can first confirm that the type of gesture is a pan gesture, and then monitor the coordinates of the gesture during the sliding process).
[0091] To give another example, when the second terminal performs a zoom operation on a copy of the projected image, it first determines the type of gesture as a pinch gesture based on the touch parameters, and then performs a corresponding zoom operation on the image based on the coordinate change of the pinch gesture.
[0092] As an example, the first terminal can perform page turning operations on the projected PPT file through a single-click / double-click gesture (or other gesture types), and perform related annotation operations on the current page of the PPT file through a series of coordinates of a translation gesture (or other gesture types) during the sliding process.
[0093] In some embodiments, it also includes:
[0094] In response to ending the screen projection operation for the target file, controlling the second terminal to delete the copy of the target file according to a preset deletion policy.
[0095] Optionally, in response to an end screen projection operation for a target file, an end screen projection instruction is issued to control the second terminal to end the screen projection and delete the copy of the projection file according to a preset deletion policy.
[0096] It should be noted that the preset deletion policy can be set according to actual needs. For example, the copy of the projection file in the second terminal and all temporary files or directories related to the copy of the projection file can be deleted; or it can be deleted after the projection is ended and delayed for a preset period of time or deleted at a scheduled time.
[0097] The terminal screen projection method provided in this embodiment solves the defects of time delay during multi-terminal interaction and inability to annotate and delete the projection file on the large screen. Specifically, it includes the following steps: in response to the projection operation for the target file, the target file is sent to the second terminal to display a copy of the target file in the second terminal; in response to the first touch operation for the target file, the touch parameters of the first touch operation are sent to the second terminal, so that the second terminal converts the touch parameters into the first target touch parameters based on the second terminal, and performs corresponding operations on the copy of the target file according to the first target touch parameters. When performing the screen projection operation, the file to be projected is transferred to the integrated large-screen terminal for projection and displayed on the integrated large-screen terminal, which solves the problem of screen projection operation delay caused by poor network environment during projection. At the same time, the user opens the file at the projection file sending end and can annotate the file by touch operation. After the touch operation is converted by algorithm, the projection file is annotated synchronously in the integrated large-screen terminal; after the projection is completed, the projection file in the integrated large-screen terminal can be deleted, which ensures the security and confidentiality of the file. At least the following pain points of users have been solved: the display effect of files is improved when they are projected on the screen, avoiding the related impact caused by poor network; at the same time, the consistency problem of annotations of projected files at different resolutions and screen sizes and the security problem of confidential files are solved.
[0098] Embodiment 2
[0099] This embodiment provides a terminal screen projection method, which can be applied to / used in a second terminal (the receiving end of the target file, used to display the projection file), wherein the second terminal is communicatively connected to the first terminal described in Embodiment 1.
[0100] As an example, Figure 5 A flowchart of a terminal screen projection method applied to a second terminal provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the following steps are included:
[0101] Step 510: In response to receiving a copy of a target file for screen projection sent by the first terminal, displaying the copy of the target file;
[0102] Step 520: In response to receiving the touch parameters sent by the first terminal, convert the touch parameters into first target touch parameters based on the second terminal, and perform corresponding operations on the copy of the target file according to the first target touch parameters.
[0103] In some embodiments, the copy of the target file is displayed in full screen mode in the second terminal.
[0104] Optionally, the second terminal displays the content in full screen mode, and then displays the content in the maximum display area.
[0105] It should be noted that, for the copy of the first target file received by the second terminal: it can be displayed in full screen immediately after receiving it; in some possible situations, the received copy can also be saved in a fixed location in a preset directory, and then displayed in full screen in response to the preview operation. Among them, the display mode of the copy and the specific storage location of the copy can be set according to actual needs.
[0106] It can be understood by those skilled in the art that the algorithm for displaying with equal aspect ratio includes the following processing steps: first, the width and height of the receiving end screen and the width and height of the projected image are obtained; then, the width of the projected image is set to the width of the receiving end screen; finally, the converted image height is determined by equal aspect ratio according to the width of the projected image, and it is determined whether the converted image height is greater than the height of the receiving end screen. If the converted image height is greater than the height of the receiving end screen, the height of the projected image is set to the height of the receiving end screen, and then the converted image width is determined by equal aspect ratio according to the height of the projected image, and the width of the projected image is set to the converted image width; if the converted image height is not greater than the height of the receiving end screen, the projected image is displayed according to the converted image height. Based on this, the projected image can reach the maximum display area when displayed in equal aspect ratio on the screen of the receiving end.
[0107] It should be noted that when setting the display width and height of the projected image on the second terminal, it can also be set according to the actual placement of the second terminal (for example, whether the screen of the second terminal is placed horizontally or vertically).
[0108] For example, assuming that the width and height of the receiving end screen are w1 and h1 respectively, and the width and height of the projected image are w2 and h2 respectively. The expansion factor can be determined by the following formula:
[0109] sp1=w1 / w2;
[0110] Then, the height of the expanded image h3=h2*sp1 is determined according to the expansion factor sp1;
[0111] If h3 is smaller than the height h1 of the receiving end screen, the width and height of the projected image are set to w1 and h3 respectively to achieve the purpose of the maximum display area of the same aspect ratio display; if h3 is larger than the height h1, the height h1 of the receiving end screen is used as the height of the projected image after expansion, and the expansion factor is re-determined according to the following formula:
[0112] sp2=h1 / h2;
[0113] Then, the width of the expanded image w3=w2*sp2 is determined according to the expansion factor sp2; the width and height of the projected image are set to w3 and h1 respectively, so that the projected image can reach the maximum display area when displayed in the same aspect ratio on the screen of the receiving end.
[0114] In some embodiments, the operation of converting the touch parameters includes:
[0115] Acquire a first display area of the target file in the first terminal, and a second display area of the copy of the target file in the second terminal;
[0116] determining a zoom ratio according to the first display area and the second display area;
[0117] The target touch parameter is determined according to the scaling ratio and the touch parameter.
[0118] It should be noted that the first display area refers to the display area of the target file on the screen of the first terminal, and the second display area refers to the display area of the copy of the target file on the screen of the second terminal.
[0119] It should be further explained that, in some possible situations, the first display area and the second display area may both include the position coordinates of the display area and the size of the display area.
[0120] To illustrate, let's take the example of marking the projected picture. Assume that the display area of the picture at the sending end is represented by the following four points: (X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2), and the display area of the projected picture at the receiving end is represented by the following four points: (X3, Y3), (X3, Y4), (X4, Y3), (X4, Y4), and then determine the scaling ratio according to the display area at both ends (for example: height scaling ratio Wc = (Y4-Y3) / (Y2-Y1), width scaling ratio Hc = (X4-X3) / (X2-X1)). Each time the receiving end receives the touch coordinates (X, Y) from the sending end, it subtracts the starting coordinate point of the image display area to obtain the annotation coordinates (XA, YA) (for example: XA=X-X1, YA=Y-Y1). If XA or YA has a negative value, it means that the touch point is not on the projected image, and no response is required at this time; if both XA and YA are positive values, they can be considered as the annotation coordinates in the projected image; according to the annotation coordinates (XA, YA), the coordinates are converted at the receiving end to obtain the converted annotation coordinates (for example: XB=XA*Wc, YB=YA*Hc), and so on, to obtain the coordinate values that meet the receiving end's own resolution, and perform corresponding operations at the receiving end based on the converted coordinates.
[0121] Optionally, it also includes: in response to an end screen projection instruction sent by the first terminal, controlling the second terminal to delete the copy of the target file according to a preset deletion policy.
[0122] Optionally, after receiving the end screen projection instruction sent by the sending end, the receiving end closes the opened screen projection file (a copy of the target file) and ends the screen projection. Furthermore, the receiving end may delete the screen projection file and / or cache files / directories related to the screen projection file.
[0123] Furthermore, after deleting the projection file (a copy of the target file), the projection file can also be initialized or erased at its original storage location in the second terminal to further reduce the risk of leakage of the projection file and improve the security of the projection operation.
[0124] In some embodiments, it also includes:
[0125] In response to a second touch operation on the copy of the target file, touch parameters of the second touch operation are sent to the first terminal, so that the first terminal converts the touch parameters into second target touch parameters based on the first terminal, and performs corresponding operations on the target file according to the second target touch parameters.
[0126] It should be noted that the second touch operation described in the present application refers to a touch operation performed on a copy of the target file in the second terminal (the receiving end of the target file).
[0127] For example, the touch operation of the related gesture can also be monitored in the second terminal by monitoring the corresponding callback method. Whenever the touch parameters of the touch operation are monitored to change (that is, whenever the callback method is called once), the current touch parameters are sent to the first terminal; that is, in response to monitoring the change of the touch parameters of the second touch operation, the current touch parameters of the second touch operation are sent to the first terminal, and the touch parameters are converted into second target touch parameters based on the first terminal by the first terminal, and then the first terminal is controlled to perform corresponding operations on the target file according to the second target touch parameters.
[0128] For example, in this embodiment, the annotation operation performed on the copy of the target file in the second terminal can also be performed on the target file in real time in the first terminal after the corresponding coordinate conversion.
[0129] Optionally, a screenshot button for taking screenshots, an undo button for undoing the previous operation, or a save button for saving the projection file, etc. can be set in the first terminal or the second terminal. Moreover, the button can be set to control the current end to perform the corresponding operation alone (the button in the first terminal controls the first terminal to perform the corresponding operation, and the button in the second terminal controls the second terminal to perform the corresponding operation) according to actual needs, or it can be set to control both ends (the sending end and the receiving end of the target file) to perform the corresponding operation at the same time, and the operation results of the button can also be displayed synchronously on both ends.
[0130] In some embodiments, it also includes:
[0131] In response to the screenshot operation, a screenshot operation is performed on the current display page of the second terminal and the screenshot file is sent to the first terminal.
[0132] Optionally, a screenshot file of the screen projection file is sent to the first terminal to save important annotation information in the first terminal.
[0133] In some possible embodiments, the annotation information may also be saved by triggering a save button, thereby directly saving the important annotation information in the target file being projected in the first terminal.
[0134] Furthermore, the screenshot file generated by the screenshot can be deleted immediately after being sent to the first terminal; or the screenshot file can be saved in a preset directory of the second terminal so that it can be deleted according to a preset deletion policy after the screen projection is completed.
[0135] In some embodiments, in response to an exit screen projection operation on the copy of the target file, an exit screen projection message is sent to the first terminal so that the first terminal exits screen projection of the target file.
[0136] For example, the second terminal sends an exit screen projection message to the first terminal after executing the exit screen projection operation for the copy of the target file currently being projected, so that the first terminal exits the projection of the target file.
[0137] Embodiment 3
[0138] This embodiment provides a terminal screen projection method, which can be applied to / used in a second terminal (a receiving end of a target file, used to display the projection file), wherein the second terminal is communicatively connected with a plurality of first terminals.
[0139] As an example, Figure 8 A flowchart of another terminal screen projection method applied to a second terminal provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the following steps are included:
[0140] Step 810: In response to receiving a copy of a second target file for screen projection sent by the first terminal, performing a cache operation on the copy of the second target file, and sending a waiting screen projection message to the first terminal;
[0141] Step 820: In response to ending the screen projection operation for the current screen projection file, a start screen projection message is sent to the first terminal to wait for receiving touch parameters for the second target file sent by the first terminal.
[0142] In some embodiments, it also includes:
[0143] In response to an opening operation on the copy of the second target file, a screen projection prompt message is sent to the first terminal so that the first terminal opens the second target file.
[0144] As an example, the screen projection prompt message sent to the first terminal may be a push message, through which the user is reminded to open the second target file in the first terminal.
[0145] As another example, a screen projection prompt message is sent to the first terminal, and in response to the screen projection prompt message, the first terminal is controlled to directly open the second target file.
[0146] After sending the waiting screen projection message to the first terminal, the method further includes:
[0147] In response to the screen projection operation for the copy of the second target file, control the first terminal to exit the screen projection operation on the current screen projection file; and display the copy of the second target file in full screen mode; and send a start screen projection message to the first terminal to wait for receiving the touch parameters for the second target file sent by the first terminal.
[0148] For example, the same first terminal sends three pictures for screen projection (including: a copy of picture 1, a copy of picture 2, and a copy of picture 3) to the second terminal. Assume that the copy of picture 1 is being projected in the second terminal. When the second terminal opens the copy of picture 3 and starts screen projection, the first terminal is controlled to exit the screen projection operation on the current screen projection file (picture 1); and display the copy of picture 3 in full screen mode; send a start screen projection message to the first terminal (the start screen projection message may include relevant information about the copy of picture 3), and wait to receive the touch parameters for the second target file (picture 3) sent by the first terminal.
[0149] In some embodiments, after sending the waiting screen projection message to the first terminal, the method further includes:
[0150] In response to the screen projection operation for the copy of the second target file, control another first terminal to execute the end of the screen projection operation, and the other terminal is the first terminal that sends the current screen projection file; and display the copy of the second target file in full screen mode; and send a start screen projection message to the first terminal to wait for receiving the touch parameters for the second target file sent by it.
[0151] For example, different first terminals send three pictures for screen projection to the second terminal (for example, receiving a copy of picture 1 sent by first terminal 1, receiving a copy of picture 2 sent by first terminal 2, and receiving a copy of picture 3 sent by first terminal 3). Assume that the copy of picture 1 is being projected in the second terminal. When the second terminal opens the copy of picture 3 and starts projecting, the first terminal 1 is controlled to perform an end projecting operation on the current projected file (picture 1) to end the projecting. After the projecting ends, the first terminal 1 is disconnected from the second terminal; and the copy of picture 3 is displayed in full screen mode; and a start projecting message is sent to the first terminal 3 (the start projecting message may include relevant information about the copy of picture 3), and waits to receive the touch parameters for the second target file (picture 3) sent by the first terminal 3.
[0152] As an example, when multiple sending ends send pictures for screen projection to the same receiving end, the specific process may include:
[0153] The first sending end sends a picture p1 to the receiving end;
[0154] The receiving end downloads the picture p1 according to the picture address of the picture p1 in the local area network. After completing the download and opening the picture p1, the receiving end sends a first notification message to the first sending end through UDP to inform the first sending end that the picture has been opened. To ensure reliability, an ack flag and the picture address may be added to the first notification message;
[0155] After receiving the first notification message, the first sending end verifies whether the picture address is correct. When the picture address is verified, the picture p1 can be opened at the first sending end and the control right of the receiving end can be obtained. Then, the receiving end can perform operations such as annotation and send a second notification message to the receiving end. The second notification message can be accompanied by an ack mark of receiving.
[0156] After receiving the second notification message, the receiving end checks the legitimacy of the ack flag, and establishes a connection when the ack flag passes the verification, allowing the response to the control commands and touch events subsequently sent by the first sending end;
[0157] The second sender starts sending a picture p2 to the receiver;
[0158] The receiving end downloads the picture p2 according to the picture address of the picture p2 in the local area network, saves the downloaded picture in the picture cache list, and sends a waiting screen projection message to the second sending end;
[0159] After the first sending end exits the screen projection control, or after the second sending end switches to the picture p2 for screen projection display and sends a screen projection end message to the first sending end to disconnect, a third notification message is sent to the second sending end to inform the second sending end that the picture has been opened, wherein the third notification message includes the address of the picture p2 and the ack flag;
[0160] After receiving the third notification message, the second sending end verifies whether the picture address is correct. When the picture address is verified, the second sending end can open the picture p2 and obtain the control right of the receiving end, and then the receiving end can perform operations such as annotation and send a fourth notification message to the receiving end. The fourth notification message may be accompanied by a received ack mark;
[0161] After receiving the fourth notification message, the receiving end checks the legitimacy of the ack flag, and establishes a connection when the ack flag passes the verification, allowing the second sending end to respond to subsequent control commands and touch events.
[0162] In this embodiment, the touch parameters of the screen projection file are converted accordingly to generate the touch coordinates of the receiving end, and then the screen projection file is controlled accordingly at the receiving end according to the converted touch coordinates; further, the touch parameters of the screen projection file are converted accordingly to generate the touch coordinates of the sending end, and then the screen projection file is controlled accordingly at the sending end according to the converted touch coordinates. The synchronous annotation function for the screen projection file at the sending end and the receiving end is realized, solving the problem of being unable to control the screen projection file at the receiving end.
[0163] Embodiment 4
[0164] This embodiment provides a terminal screen projection method. Figure 6 A flowchart of a terminal screen projection method provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the following steps are included:
[0165] Step 610: In response to a screen projection operation on a target file in a first terminal, control the first terminal to send the target file to a second terminal so that a copy of the target file is displayed in the second terminal;
[0166] Step 620: When the target file is in an opened state, in response to a touch operation on the target file in the first terminal, control the first terminal to send a touch parameter of the touch operation to the second terminal;
[0167] Step 630: In response to receiving the touch parameters sent by the first terminal, control the second terminal to convert the touch parameters into first target touch parameters based on the second terminal, and perform corresponding operations on the copy of the target file according to the first target touch parameters.
[0168] It should be noted that when the touch parameters are sent between the two ends, the sending can be done in real time to minimize the touch delay.
[0169] In some embodiments, it also includes:
[0170] In response to ending the screen projection operation on the target file in the first terminal, controlling the second terminal to delete the copy of the target file according to a preset deletion policy.
[0171] It can be understood that the first terminal described in this embodiment is the sending end of the target file, which is used to send the target file to the second terminal for display; the second terminal is the receiving end of the target file, which is used to display a copy of the target file through a large-screen display.
[0172] For easier understanding of the technical solution of this application, please refer to Figure 7 , Figure 7 A flowchart of another terminal screen projection method provided in an embodiment of the present application.
[0173] Those skilled in the art can clearly understand that for the convenience and brevity of description, for a more specific working process, reference can be made to the corresponding process in the aforementioned embodiments, and this embodiment will not be repeated here.
[0174] Embodiment 5
[0175] This embodiment provides a first terminal for sending a target file to a second terminal for screen projection. The first terminal includes a memory and a processor, the memory stores a computer program, the memory and the processor are connected to each other for communication, and when the computer program is executed by the processor, the terminal screen projection method described in the first embodiment is executed.
[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the first terminal can refer to the corresponding process in the aforementioned embodiment 1, and this embodiment will not be repeated here.
[0177] Embodiment 6
[0178] This embodiment provides a second terminal for receiving and displaying a target file to be projected. The second terminal includes a memory and a processor, the memory stores a computer program, the memory and the processor are connected to each other for communication, and when the computer program is executed by the processor, the terminal screen projection method described in Embodiment 2 or Embodiment 3 is executed.
[0179] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the first terminal can refer to the corresponding process in the aforementioned embodiment 2 or embodiment 3, and this embodiment will not be repeated here.
[0180] Embodiment 7
[0181] This embodiment provides a terminal screen projection system, including the second terminal described in Embodiment 6 and the first terminal described in Embodiment 5.
[0182] It can be understood that the terminal screen projection system may include one or more first terminals as described in Example 5.
[0183] Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the terminal screen projection system can refer to the corresponding processes in the aforementioned Example 1, Example 2 and Example 3, and can also refer to the corresponding process in the aforementioned Example 4, and this embodiment will not be repeated here.
[0184] Embodiment 8
[0185] This embodiment provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps in the above method embodiment can be implemented, and this embodiment will not be repeated here.
[0186] Among them, the computer-readable storage medium may also include computer programs, data files, data structures, etc. alone, or include a combination thereof. The computer-readable storage medium or computer program may be specifically designed and understood by a technician in the field of computer software, or the computer-readable storage medium may be known and available to a technician in the field of computer software. Examples of computer-readable storage media include: magnetic media, such as hard disks, floppy disks, and tapes; optical media, such as CDROM disks and DVDs; magneto-optical media, such as optical disks; and hardware devices, specifically configured to store and execute computer programs, such as read-only memory (ROM), random access memory (RAM), flash memory; or servers, app application stores, etc. Examples of computer programs include machine code (e.g., code generated by a compiler) and files containing high-level code, which can be executed by a computer using an interpreter. The described hardware device may be configured to be used as one or more software modules to perform the operations and methods described above, and vice versa. In addition, the computer-readable storage medium may be distributed in a networked computer system, and program code or computer programs may be stored and executed in a decentralized manner.
[0187] Embodiment 9
[0188] This embodiment provides a computer program product. The computer program product includes a computer program or instructions, and when the computer program or instructions are executed by a processor, all or part of the steps of the method in the above-mentioned method embodiment 1, embodiment 2, embodiment 3 or embodiment 4 are implemented, and this embodiment will not be repeated here.
[0189] Further, the computer program product may include one or more computer executable components configured to perform the embodiment when the program is run; the computer program product may also include a computer program tangibly contained on a readable medium, the computer program including program code for performing any method in the embodiments of the present disclosure. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.
[0190] In summary, the present application provides a terminal screen projection method, a first terminal, a second terminal, a terminal screen projection system, and a storage medium. When performing a screen projection operation, the file to be projected is transferred to the all-in-one large-screen terminal used for screen projection and displayed on the all-in-one large-screen terminal, which solves the problem of screen projection operation delay caused by poor network environment during screen projection. At the same time, the user opens the file at the screen projection file sending end and can annotate the file by touch operation. The touch operation is synchronously annotated on the projection file in the all-in-one large-screen terminal after algorithm conversion; after the screen projection is completed, the projection file in the all-in-one large-screen terminal can be deleted to ensure the security and confidentiality of the file. At least the following pain points of users are solved: when the file is projected, the display effect of the file is improved, and the related effects caused by poor network are avoided; at the same time, the consistency problem of annotations of projection files at different resolutions and screen sizes and the security problem of confidential files are also solved.
[0191] In addition, it should be understood that the method or system disclosed in the embodiments provided in the present application can also be implemented in other ways. The method or system embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the methods and systems according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a computer program segment or a part of a computer program, and the module, computer program segment or a part of a computer program contains one or more computer programs for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings, and can actually be executed in parallel. They can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer programs.
[0192] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements that are inherent to such process, method, article, or apparatus. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, device or equipment including the elements; if there is a description of "first", "second", etc., it is only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features; in the description of this application, unless otherwise specified, the terms "multiple" and "multiple" mean at least two; if there is a description of a server, it should be noted that the server can be an independent physical server or terminal, or a server cluster composed of multiple physical servers, or a cloud server that can provide basic cloud computing services such as cloud servers, cloud databases, cloud storage and CDN; if there is a description of a smart terminal or mobile device in this application, it should be noted that the smart terminal or mobile device can be a mobile phone, a tablet computer, a smart watch, a netbook, a wearable electronic device, a personal digital assistant (PDA), an augmented reality technology device (AR), a virtual reality device (VR), a smart TV, a smart speaker, a personal computer (PC), a tablet computer, a smart phone, a smart TV, a smart speaker, a smart phone ... Computer, referred to as PC), etc., but not limited to this, the present application does not specifically limit the specific form of the smart terminal or mobile device.
[0193] Finally, it should be noted that, in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "an example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0194] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary, and the contents described are only embodiments adopted for the convenience of understanding the present application, and are not intended to limit the present application. Any technician in the technical field to which the present application belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present application, but the scope of protection of the present application shall still be based on the scope defined in the attached claims.
Claims
1. A terminal screen projection method, characterized in that: Applied to a first terminal, the first terminal is in communication connection with a second terminal, the method includes: In response to a screen projection operation on a target file, sending the target file to a second terminal so as to display a copy of the target file in the second terminal; When the target file is in an opened state, in response to a first touch operation on the target file, touch parameters of the first touch operation are sent to the second terminal, so that the second terminal converts the touch parameters into first target touch parameters based on the second terminal, and performs corresponding operations on the copy of the target file according to the first target touch parameters.
2. The terminal screen projection method according to claim 1, characterized in that: The touch parameters include: The type of touch operation and / or the coordinates of the touch operation.
3. The terminal screen projection method according to claim 1, characterized in that: The aspect ratio of the target file displayed in the first terminal is the same as the aspect ratio of the copy of the target file displayed in the second terminal.
4. The terminal screen projection method according to any one of claims 1 to 3, characterized in that: Also includes: In response to ending the screen projection operation for the target file, controlling the second terminal to delete the copy of the target file according to a preset deletion policy.
5. The terminal screen projection method according to claim 1, characterized in that: Also includes: In response to an opening operation on the target file, the second terminal is controlled to open a copy of the target file.
6. A terminal screen projection method, characterized in that: Applied to a second terminal, the second terminal is communicatively connected with the first terminal, the method includes: In response to receiving a copy of the target file for screen projection sent by the first terminal, displaying the copy of the target file; In response to receiving the touch parameter sent by the first terminal, the touch parameter is converted into a first target touch parameter based on the second terminal, and a corresponding operation is performed on the copy of the target file according to the first target touch parameter.
7. The terminal screen projection method according to claim 6, characterized in that: The operation of converting the touch parameters includes: Acquire a first display area of the target file in the first terminal, and a second display area of the copy of the target file in the second terminal; determining a zoom ratio according to the first display area and the second display area; The target touch parameter is determined according to the scaling ratio and the touch parameter.
8. The terminal screen projection method according to claim 6, characterized in that: Also includes: In response to a second touch operation on the copy of the target file, touch parameters of the second touch operation are sent to the first terminal, so that the first terminal converts the touch parameters into second target touch parameters based on the first terminal, and performs corresponding operations on the target file according to the second target touch parameters.
9. The terminal screen projection method according to claim 6, characterized in that: The copy of the target file is displayed in full screen mode in the second terminal.
10. A terminal screen projection method, characterized in that: Applied to a second terminal, the second terminal being communicatively connected with a plurality of first terminals, the method comprising: In response to receiving a copy of a second target file for screen projection sent by the first terminal, performing a cache operation on the copy of the second target file, and sending a waiting screen projection message to the first terminal; In response to ending the screen projection operation for the current screen projection file, a start screen projection message is sent to the first terminal to wait for receiving touch parameters for the second target file sent by the first terminal.
11. The terminal screen projection method according to claim 10, characterized in that: Also includes: In response to an opening operation on the copy of the second target file, a screen projection prompt message is sent to the first terminal so that the first terminal opens the second target file.
12. The terminal screen projection method according to claim 10, characterized in that: After sending the waiting screen projection message to the first terminal, the method further includes: In response to the screen projection operation for the copy of the second target file, control the first terminal to exit the screen projection operation on the current screen projection file; and display the copy of the second target file in full screen mode; and send a start screen projection message to the first terminal to wait for receiving the touch parameters for the second target file sent by the first terminal.
13. The terminal screen projection method according to claim 10, characterized in that: After sending the waiting screen projection message to the first terminal, the method further includes: In response to the screen projection operation for the copy of the second target file, control another first terminal to execute the end of the screen projection operation, and the other terminal is the first terminal that sends the current screen projection file; and display the copy of the second target file in full screen mode; and send a start screen projection message to the first terminal to wait for receiving the touch parameters for the second target file sent by it.
14. A first terminal, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the terminal screen projection method as described in any one of claims 1 to 5 is executed.
15. A second terminal, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the terminal screen projection method as described in any one of claims 6 to 9 or 10 to 13 is executed.
16. A terminal screen projection system, characterized in that: It comprises the first terminal as claimed in claim 14 and the second terminal as claimed in claim 15.
17. A computer-readable storage medium, characterized in that: The computer program stored in the computer-readable storage medium, when executed by one or more processors, implements the steps of the method as claimed in any one of claims 1 to 5, 6 to 9, or 10 to 13.