Control method and system

Abstract

The invention discloses a control method and system, and relates to the technical field of terminals, a display interface of a control device independently presents screen projection windows corresponding to a plurality of screen projection devices, and the control device obtains a dragging operation of a user on a first display object in a first screen projection window of a first screen projection device; and the control equipment displays a global dragging dynamic effect corresponding to the dragging operation based on the dragging operation and a global view layer independent of each projection screen window, and the global dragging dynamic effect is configured to be capable of moving across each projection screen window along with the dragging operation track of the user so as to drag the first display object to a second projection screen window of target second projection screen equipment.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a control method and system. Background Technology

[0002] With the popularization of wireless screen projection technology, multiple terminal devices (such as laptops, smartphones, or tablets) can project their screen content onto a single display device (such as a large smart screen, conference tablet, or control device), and each device will be presented in an independent projection window. This has created a demand for cross-device collaborative operation, in which dragging and dropping objects across projection windows has become a natural interaction method.

[0003] However, when a user attempts to drag an object from one projection window to another, the rendering of the drag animation is scattered across various projection devices, causing a serious problem of fragmented drag animation and reducing the smoothness of the user experience. Summary of the Invention

[0004] In view of the above problems, this application provides the following solution:

[0005] This application provides a control method for use in a control device, wherein the display interface of the control device independently presents projection windows corresponding to multiple projection devices, and the method includes:

[0006] Get the user's drag operation on the first display object within the first projection window of the first projection device;

[0007] Based on the drag operation and a global view layer independent of each of the projection windows, a global drag animation corresponding to the drag operation is displayed. The global drag animation is configured to follow the user's drag operation trajectory and move across each of the projection windows to drag the first display object to the second projection window of the destination second projection device.

[0008] In one possible implementation, the method further includes:

[0009] In response to the drag operation, a first drag event is sent to the first projection device so that the projection window data of the first projection device does not include the native drag animation of the first display object.

[0010] In one possible implementation, the method further includes:

[0011] Receive a first transmission message sent by the first projection device based on the first drag event, the first transmission message including drag animation visual data of the first display object;

[0012] The drag-and-drop animation visual data in the first transmission message is used for display rendering, and the global drag-and-drop animation is presented on the global view layer.

[0013] In one possible implementation, the first transmission message further includes data storage information for locating and accessing file data corresponding to the first display object; the data storage information includes the storage path of the file data corresponding to the first display object in a shared directory of a network file system; the method further includes:

[0014] In response to the drag-and-release operation of the global drag-and-release animation, the second projection device is triggered to directly access the storage path shared by the first projection device through the direct communication channel between the second and first projection devices, following the network file system protocol, to obtain the file data corresponding to the first display object; the second projection device is the projection device corresponding to the second projection window where the drag-and-release position is located.

[0015] In one possible implementation, the triggering method for the second screen projection device to obtain the file data corresponding to the first display object includes any of the following:

[0016] The system determines the second projection device corresponding to the second projection window where the drag-and-release position is located, and sends a data acquisition message to the second projection device. The data acquisition message contains the data storage information, so that the second projection device responds to the data acquisition message and acquires the file data corresponding to the first display object.

[0017] A drag-and-release event is sent to each of the screen projection devices, so that each screen projection device determines that it belongs to the second screen projection device based on the drag-and-release event, and then obtains the file data corresponding to the first display object in response to the cached data storage information.

[0018] In one possible implementation, the control device and the plurality of projection devices are all connected to a publish-subscribe message bus and subscribe to a preset projection topic in the message bus, so that the first projection device responds to the first drag event and publishes the first transmission message to the projection topic through the message bus.

[0019] The receipt of the first transmission message sent by the first projection device based on the first drag event includes:

[0020] The first transmission message under the projection topic is received through the message bus.

[0021] In one possible implementation, the data acquisition message is transmitted through a screen projection communication channel established between the control device and the second screen projection device based on a screen projection protocol;

[0022] The drag-and-release event is sent to each of the screen projection devices via a publish-subscribe message bus.

[0023] A second aspect of this application provides a control method applied to a first screen projection device among multiple screen projection devices, wherein the screen projection windows of each of the multiple screen projection devices are independently displayed on the display interface of the control device, the method comprising:

[0024] The control device receives a first drag event; the first drag event is generated by the control device in response to a user's drag operation on a first display object within the first projection window of the first projection device.

[0025] Intercept the native drag animation triggered by the first drag event on the first screen projection device, so that the screen projection window data of the first screen projection device does not include the native drag animation of the first display object;

[0026] The control device displays a global drag animation corresponding to the drag operation on its display interface. The global drag animation is configured to follow the user's drag operation trajectory and move across each of the projection windows to drag the first display object to the second projection window of the target second projection device.

[0027] In one possible implementation, the interception of the native drag animation triggered by the first drag event on the first projection device includes any of the following implementation methods:

[0028] In response to the animation interception command corresponding to the first drag event, the drag animation rendering module in the first projection device is controlled to be turned off to prevent the generation of native drag animation.

[0029] The first projection device is prohibited from sending projection data of native drag-and-drop animation to the control device, so that the native drag-and-drop animation is not displayed in the first projection window; the projection data is generated by the drag-and-drop animation rendering module in the first projection device in response to the first drag event.

[0030] A third aspect of this application provides a control system, the system including a control device and a plurality of projection devices communicatively connected to the control device, wherein:

[0031] The display interface of the control device independently presents the projection windows corresponding to each of the multiple projection devices.

[0032] The control device includes:

[0033] The user interface is used to obtain the user's drag and drop operation on the first display object in the first projection window of the first projection device.

[0034] The display screen is used to display a global drag animation corresponding to the drag operation based on the drag operation and a global view layer independent of each of the projection windows. The global drag animation is configured to follow the user's drag operation trajectory across each of the projection windows to drag the first display object to the second projection window of the destination second projection device. Attached Figure Description

[0035] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0036] Figure 1 This is a schematic diagram of the architecture of the control system proposed in Embodiment 1 of this application;

[0037] Figure 2 This is a schematic diagram of the control system proposed in Embodiment 2 of this application;

[0038] Figure 3 This is a schematic diagram of the architecture of the control system proposed in Embodiment 3 of this application;

[0039] Figure 4 This is a flowchart illustrating a control method proposed in Embodiment 1 of this application;

[0040] Figure 5 This is a schematic diagram of an optional scenario in a control method proposed in an embodiment of this application, where an object is dragged and displayed across a projection window.

[0041] Figure 6 This is a flowchart illustrating a control method proposed in Embodiment 2 of this application;

[0042] Figure 7 This is a schematic diagram illustrating a scenario based on MQTT message synchronization in a control method proposed in an embodiment of this application.

[0043] Figure 8 This is a flowchart illustrating a control method proposed in Embodiment 3 of this application;

[0044] Figure 9 This is a schematic diagram of the structure of a control device provided in Embodiment 1 of this application;

[0045] Figure 10 This is a schematic diagram of the structure of a control device provided in Embodiment 2 of this application. Detailed Implementation

[0046] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0047] The terms "first," "second," etc., used throughout this application and in the foregoing figures are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0048] The problem described in the background section of this application is that currently, when a user drags a displayed object from one projection window (virtual window) presented by a control device to another projection window, the dragging animation of the displayed object is only displayed within these two projection windows. Once the dragged position moves away from the boundaries of these two projection windows, the dragging animation disappears, and different dragging animations may even be displayed in the two different projection windows. This results in the user not visually perceiving a continuous and seamless dragging process, but rather a jumpy operation interrupted by the boundaries of each projection window, which greatly impairs the directness of the interaction and the smoothness of the user experience. To improve these problems, embodiments of this application provide a control method that can achieve seamless visual display of dragging animation during object dragging across projection windows. The control method and its application scenarios of embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0049] Reference Figure 1 This is a schematic diagram of the architecture of the control system proposed in Embodiment 1 of this application. The system may include: a control device 110 and multiple (at least two) projection devices 120 that are communicatively connected to the control device 110. The display interface of the control device 110 is configured to independently present the projection windows of the multiple projection devices 120, so that users can directly drag and drop display objects between the multiple projection windows displayed on the control device to complete the cross-device data transmission scheme in the multi-device projection environment.

[0050] In practical applications, the control device 110 and each projection device 120 can establish a wired or wireless projection connection on the same local area network (such as an enterprise intranet, home WIFI network, etc.) or through a wide area network to realize data transmission between the control device 110 and the corresponding projection device 120. This application does not limit the implementation method of establishing a communication connection between each projection device 120 and the control device 110 based on projection technology. Different projection devices 120 and the control device 110 can establish the same or different projection connections, which can be determined according to the actual scenario requirements. It should be understood that after each projection device 120 establishes a communication connection with the control device 110 through projection technology, the projection device 120 can send projection data, such as real-time video streams, to the control device 110 for synchronous display in the corresponding projection window (virtual projection window) displayed on the display interface of the control device 110. After that, the displayed objects in the projection window can be operated directly without directly operating the corresponding projection device 120. The process of acquiring and transmitting projection data is not described in detail in this application.

[0051] Based on the above analysis, and referring to Figure 2 The diagram shows a scenario where the display interface of the control device 110 displays the respective projection windows of multiple projection devices. Since the projection windows of different projection devices 120 are simultaneously displayed on the same interface, in cases where data needs to be transferred from one projection device to another, such as... Figure 2 As shown, users can directly drag and drop objects (objects displayed in the interactive interface, such as icons, file names, or other identifiers) from one projection window in the control device 110 to another projection window without needing to go to the projection device at the data source location. During the dragging of display objects across projection windows, both the control device 110 and the projection device 120 corresponding to the projection window can execute the control method proposed in this application, displaying a visually continuous and seamless dragging animation in the control device 110, so that users are unaware of the projection window switching, and the dragging process is completed in one go.

[0052] In some embodiments, under the control system architecture, for the projection windows corresponding to each of the multiple projection devices 120, it can be as follows: Figure 2The images shown are simultaneously displayed on the display interface of the control device 110, allowing users to intuitively see at least one display object displayed in each projection window. Users can then select the display object corresponding to the data to be transmitted (denoted as the first display object) to complete the drag-and-drop operation across projection windows. Optionally, especially when there are a large number of projection devices 120, users can minimize some projection windows or hide them in other ways (such as overlapping some projection windows), presenting another part of the projection windows on the display interface of the control device 110, making it easier for users to clearly read the content displayed in this part of the projection windows. In this way, this application can simultaneously present the projection windows corresponding to the source and destination devices of the data to be transmitted on the display interface, and then select the display object for drag-and-drop operation. This application does not restrict the independent display method of the projection windows corresponding to multiple projection devices.

[0053] Based on the above analysis, the control device 110 proposed in this application may include a user interface 111 and a display screen 112. The user interface 111 can be used to obtain the user's drag operation on the first display object within the first projection window of the first projection device. The first projection window can be any one of the multiple projection windows currently displayed on the display interface of the control device 110. The first projection device is the projection device corresponding to the first projection window, that is, the source device of the data to be transmitted. It can dynamically determine which projection device is the first projection device according to the changes in actual drag requirements, and record the corresponding projection window as the first projection window.

[0054] The user interface 111 can be a hardware interface for input components such as a keyboard, mouse, touchscreen, stylus, or audio acquisition device. It supports users selecting a first display object and performing a drag-and-drop operation using the input component. It should be understood that the implementation of the drag-and-drop operation for the first display object can differ for different types of input components, and this application does not impose any restrictions on this. Furthermore, during the process of the user dragging and dropping the first display object via voice command, the audio acquisition device can capture the user's input audio. This audio can then be recognized by the intelligent program controlling the device (such as an intelligent agent or intelligent assistant) as a cross-projection window drag task for the first display object, automatically executing the task—that is, dragging the first display object from the first projection window to the destination projection window (which can be referred to as the second projection window).

[0055] The display screen 112 in the control device 110 can display a global drag animation corresponding to the drag operation based on the drag operation of the first display object and a global view layer independent of each projection window. The global drag animation of this application is configured to follow the user's drag operation trajectory across each projection window to drag the first display object to the second projection window of the destination second projection device.

[0056] In this process, the control device 10 captures the drag operation on the first display object. To address the problem of fragmented drag animations caused by traditional object drag operations where the drag animation is only displayed within the corresponding projection window, this application can create a global view layer independent of each projection window through the system-level graphics interface of the control device 110. The display layer of this global view layer is higher than that of each projection window, ensuring that the global drag animation is not obscured by any projection window. Therefore, the global view layer is a new top-level image with the highest display priority. Thus, by displaying the global drag animation of the drag operation on the first display object based on the global view layer, the global drag animation is ensured to always be physically above all projection windows. Its movement trajectory is a continuous curve on the display interface, completely unaffected by the boundaries of the projection windows below. What the user intuitively sees is a display object smoothly flying from one place on the display screen 112 (the original display position within the first projection window) to another place (the destination position within the second projection window), achieving true visual seamlessness.

[0057] In one possible implementation, at any given moment, only one global drag animation is presented for a single drag operation, avoiding conflicts between multiple animations and achieving uniformity in the global drag animation. During this single drag operation, the display position (movement trajectory) of the global drag animation on the screen can be determined and dynamically updated based on the user's drag operation position on the display interface, achieving visual continuity when crossing different projection window boundaries. Optionally, the global drag animation may include at least one of the following: a first display object, a file data type identifier corresponding to the first display object, a drag status prompt, etc. This application does not limit the type of global drag animation or the content of its elements; it can be determined as appropriate.

[0058] Regarding the implementation process of the user interface 111 and display screen 112 in the control device 110 respectively executing the corresponding contents described above, please refer to the description of the corresponding parts of the method embodiments below, which will not be detailed here. Further, the control device 110 may also include at least one first memory and at least one first processor. The first memory is used to store multiple first computer instructions for implementing the control method, as well as messages or data from each projection device 120, etc. The first processor is used to execute the first computer instructions to implement the control method on the control device 110 side.

[0059] In some embodiments, the screen projection device 120 may include at least one second memory and at least one second processor. The second memory may be used to store file data that can be shared with other screen projection devices, messages from other screen projection devices or control devices, and second computer instructions for implementing the control method executed on the screen projection device side. The second processor executes the second computer instructions to implement the various steps of the control method executed on the screen projection device side. The implementation process can be referred to the description of the method embodiment below, which will not be described in detail here.

[0060] In order to achieve message synchronization between the control device 110 and the multiple projection devices 120 connected to it, in some embodiments, such as Figure 3 As shown, a message server and a matching message client can be deployed in the control device 110, and a message client that matches the message server and can communicate is deployed in each projection device 120, so as to realize message synchronization between the control device 110 and each projection device 120 through a publish and subscribe mode message bus.

[0061] Optionally, the control device 110 and its multiple screen-sharing devices 120 can achieve real-time and reliable message synchronization via an MQTT (Message Queuing Telemetry Transport, a lightweight message transmission protocol based on a publish / subscribe model) message bus to meet the communication needs of resource-constrained terminal devices. In this case, the message server can be an MQTT server, and the message client can be an MQTT client, thus avoiding the complexity of point-to-point communication based on a one-to-many message distribution mechanism. During message synchronization, a hierarchical topic structure can be used to ensure the accuracy and scalability of message routing. In this embodiment, a screen-sharing topic (equivalent to a session, configured with a one-to-one corresponding topic / session identifier) ​​can be pre-created in the message bus, allowing multiple screen-sharing devices 120 communicating with the control device 110 to subscribe to the preset screen-sharing topic. The control device 110 can then record the device identifiers (such as device IDs) of each screen-sharing device 120 subscribing to the topic. This can be achieved by associating the topic identifier with each device identifier and managing the status of each device accordingly. The implementation process is not detailed in this application.

[0062] Thus, after any device subscribing to a preset projection topic publishes a message, such as to the projection topic on the message bus, other devices subscribing to that projection topic can receive the messages under that projection topic through the message bus to parse the message content and complete the tasks they need to perform. This application does not detail the processing logic of the control device 110 after receiving messages in this manner, but is not limited to this method. It supports fault tolerance and recovery mechanisms for both the control device 110 and each projection device 120 to ensure timely recovery in case of communication failures and meet the control requirements of the corresponding devices. Furthermore, to meet the expansion needs of the system, such as adding a projection device, controlling that projection device to subscribe to the projection topic will allow it to obtain the messages published under that projection topic. The implementation process is not detailed in this application. It should be noted that the message synchronization methods between the control device 110 and each projection device 120 include, but are not limited to, the MQTT communication method.

[0063] Furthermore, to support point-to-point transmission between different projection devices 120, a direct communication channel can be established between them. During drag-and-drop operations on the first displayed object across projection windows, and when the message synchronization method described above determines that the second projection device needs to obtain file data corresponding to the first displayed object shared by the first projection device, the file data can be obtained through the corresponding direct communication channel. Optionally, each projection device 120 can be mounted on a shared directory of a Network File System (NFS), allowing the second projection device to directly access the storage path of the shared file data of the first projection device through the direct communication channel, and read or copy the file data. This application does not elaborate on the detailed process of how data sharing between different projection devices is achieved through NFS. Of course, in a multi-device projection environment, data transmission between different projection devices can also be achieved using other network sharing methods, and this application does not limit this.

[0064] It should be understood that, Figures 1-3The structure of the control system and its included control devices shown does not constitute a limitation on the control system and devices in the embodiments of this application. In practical applications, the control devices and projection devices may include more or fewer components than those shown in the figures, or combine certain components. Moreover, the control device 110 in the control system proposed in this application can be any terminal device such as a mobile phone, tablet computer, wearable device with display and wireless communication functions, vehicle terminal device, virtual reality (VR) device, augmented reality (AR) device, personal computer (PC), laptop computer, netbook, smart home device (such as smart TV), etc. The projection device 120 can be a terminal device with data processing, storage, and communication capabilities. Multiple projection devices can act as independent data senders or data receivers, and the same projection device can switch roles according to the scenario. The projection device may also include, but is not limited to, these product forms of terminal devices, which can be determined according to the actual projection scenario requirements. In addition, the control device 110 may also include components such as various sensors and power management modules. This application does not provide detailed examples of the composition of the control system and the composition of each included device.

[0065] Based on the control system provided in the embodiments of this application described above, the control method applicable to the control system will be described below.

[0066] Reference Figure 4 This is a flowchart illustrating a control method proposed in Embodiment 1 of this application. The method proposed in this embodiment can be applied to control device 110. Based on the above description of the control system, in a multi-device projection environment, control device 110 is communicatively connected to multiple projection devices 120. The display interface of control device 110 independently presents the projection windows corresponding to each of the multiple projection devices 120. Based on this, as... Figure 4 As shown, the method proposed in this embodiment may include:

[0067] Step S41: Obtain the user's drag operation on the first display object within the first projection window of the first projection device;

[0068] Step S42: Based on the drag operation and a global view layer independent of each projection window, display the global drag animation corresponding to the drag operation; the global drag animation is configured to follow the user's drag operation trajectory across each projection window to drag the first display object to the second projection window of the destination second projection device.

[0069] In this embodiment of the application, based on the above analysis, it is desired that the user, on the control device, can seamlessly move the first display object from the projection window of one projection device to the projection window of another projection device through a continuous drag operation. This allows the user to visually perceive a continuous and seamless drag animation without being interrupted by the boundaries of each projection window, thus creating a sense of jumpiness. In other words, the drag animation is visually seamlessly displayed on the screen of the control device, improving the smoothness of the user experience.

[0070] The first display object can be any visual element that the user can drag and drop, possessing interactivity, visibility, and state variability, and capable of being associated with data in the projection device corresponding to the projection window, i.e., having data association. It can include, but is not limited to, application interface elements such as application icons, window thumbnails, or interface controls; file system objects such as folder identifiers / file identifiers (icons / shortcuts); media content such as video preview frames, audio waveforms, image thumbnails, or other image identifiers; and verification objects such as search result sets, multi-select object groups, or intelligent recommendation items. This application does not limit the type of the first display object and can determine it based on the drag and drop requirements of the actual scenario.

[0071] In one possible implementation, the user selects the first display object presented in the first projection window on the display interface of the control device using input components such as a mouse, keyboard, or stylus, and then begins a drag operation, such as clicking and holding the first display object with a mouse to start dragging. The input event manager of the control device can continuously capture the drag operation position of the input component on the screen through the underlying interface of the operating system, such as the position of the cursor in the absolute coordinate system of the screen. At the same time, the window manager of the control device can calculate which projection window the first display object belongs to based on the position, and can further determine that the projection window corresponds to the projection window of the first projection device through the projection session mapping table, and record it as the first projection window. In this way, it can be recognized that the user is performing a drag operation on the first display object in the first projection window of the first projection device, but it is not limited to this drag operation capture and recognition method.

[0072] After the control device recognizes the drag operation, it can call the view manager to create or call a new, transparent, top-level view layer with the highest display priority, denoted as the global view layer. This ensures that the global view layer is displayed above all projection windows, meaning it is independent of each projection window. The motion effects engine then creates and displays a dynamically controllable global drag effect within this global view layer, precisely aligning its initial position with the position at the start of the drag operation. Subsequently, as the captured drag operation position changes, the display position of the global drag effect is updated synchronously, allowing it to smoothly track the user's drag trajectory and move directly across projection windows. This ensures that the global drag effect does not disappear even when it reaches the projection window boundary. Throughout the drag operation, the global drag effect moves continuously within the global view layer, visually unable to cross projection windows, achieving seamless crossing of different projection window boundaries.

[0073] In one possible implementation, the visual data for the global drag animation effect can be rendering data of a pre-configured specified animation effect / default animation effect on the control device, or it can come from the first projection device, to ensure consistency between the global drag animation effect of the control device and the native drag animation effect of the projection device. This application does not impose any restrictions on this. Optionally, for different types of display objects, the type of global drag animation effect displayed by the control device can be the same or different, depending on the actual needs.

[0074] Optionally, during the entire drag-and-drop operation of the first display object, the control device, according to, but not limited to, the method described above, detects that the first display object has been dragged into a certain projection window. It can adjust the visual style of the projection window (e.g., highlighting) or output its corresponding projection device identifier via voice / pop-up text to remind the user which projection device's window the object is currently being dragged to, ensuring the user is clear about the operation's status and result, and preventing accidental dragging out of the target projection window, thus achieving deterministic drag-and-drop operation. Alternatively, if the first display object is detected being dragged into an invalid area, a visual cue can be provided through global drag-and-drop animations (e.g., color change or shaking) to prevent accidental operation. This application does not limit the method for implementing visual state changes during the drag-and-drop of the first display object across projection windows; it can be flexibly configured or adjusted according to actual needs.

[0075] Based on the above analysis, by dragging the first display object to the second projection window (the projection window of the target projection device, referred to as the second projection device) through a continuous drag operation, the mouse / touch can be released directly. The event manager captures the drag release operation, and the projection window where the drag release position is located can still be determined as the second projection window of the second projection device according to the position calculation method of the drag operation above.

[0076] Therefore, it can be seen that in the multi-device screen projection environment proposed in the embodiments of this application, combined with Figure 5 The diagram illustrates a drag-and-drop scenario across projection windows. Users can directly drag the currently displayed first projection device (e.g., [device name]) from the control device's display interface. Figure 5 The first display object (e.g., in the first projection window (office device window) corresponding to the projection device A shown (an office device) is... Figure 5 The file icon for document A.docx shown is shown, but is not limited to this. Drag and drop it once onto a second screen-casting device (such as...). Figure 5 In the second projection window (conference device window) corresponding to the projection device B shown (a type of conferencing device), the user perceives the same as dragging an object on a single device, without needing to adapt to the special characteristics of a multi-window environment. This intuitive operation enhances the directness of the interaction. Moreover, during a single drag, this application uses a control device based on the drag operation and a global view layer independent of each projection window to continuously display a global drag animation following the user's drag trajectory. This animation can smoothly and continuously cross the boundaries of each projection window without interruption or disappearance, ensuring the visual continuity of the drag animation and improving the smoothness of the user experience.

[0077] In this application, the global drag-and-drop animation is centrally rendered by the control device and displayed on the global view layer. This solves the problem of asynchronous and discontinuous drag-and-drop animation caused by independent rendering by each projection device. Furthermore, this application resolves the errors caused by coordinate transformations between multiple window coordinate systems by unifying the display coordinate system, thus ensuring that the display position of the global drag-and-drop animation is aligned with the drag operation position, achieving a seamless visual experience for dragging objects across windows. Additionally, the smoothness of the global drag-and-drop animation is guaranteed according to the display frequency of the control device's screen. Moreover, compared to coordinating the rendering state of drag-and-drop animations across multiple devices, this application utilizes a single rendering engine on the control device, resulting in a simpler logic.

[0078] Reference Figure 6 This is a flowchart illustrating a control method proposed in Embodiment 2 of this application. This embodiment is still applicable to the aforementioned control device, such as... Figure 6 As shown, the method proposed in this embodiment may include:

[0079] Step S61: Obtain the user's drag operation on the first display object within the first projection window of the first projection device;

[0080] Step S62: In response to the drag operation, a first drag event is sent to the first projection device so that the projection window data of the first projection device does not include the native drag animation of the first display object.

[0081] During the drag operation of the first display object within the first projection window, the first projection device typically responds to the first drag event after receiving it, displaying the native drag animation of the first display object on its local display screen. This is essentially a drag animation generated by dragging the first display object on the first projection device's screen. Since the projection mode is currently active, the projection data of the original drag animation is synchronously displayed in the first projection window. This causes the view layer where the first projection window is located to display the original drag animation, which overlaps with the global drag animation rendered by the control device itself and displayed in the global view layer, resulting in visual confusion and wasted resources.

[0082] To address this, this application proposes a solution to the problem described above at its source. The first drag event sent by the control device to the first projection device can include task information related to intercepting and suppressing the native drag animation of the first projection device. This allows the first projection device to parse the first drag event, accurately intercept and suppress the native drag animation, and ensure that the projection window data of the first projection device does not include the native drag animation of the first display object. In this case, the projection data transmitted in real-time from the first projection device to the control device may not contain data used to directly display the native drag animation on the display view layer of the first projection window; instead, it may represent the animation data of the native drag animation itself. Thus, the native drag animation will not be displayed on the display view layer of the first projection window displayed by the control device. The global drag animation rendered and displayed by the control device is the unique drag animation of the first display object, avoiding inconsistencies in animation across multiple platforms. This achieves cross-platform visual uniformity and conflict-free operation, ensuring smooth operation. Users do not need to worry about the differences between the underlying projection device and platform, and there is no need for repeated rendering that consumes a lot of resources, improving performance and enabling immediate feedback on drag operations without perceptible delay, thus ensuring real-time response.

[0083] In one possible implementation, the first drag event also does not contain the task information described above. The first projection event, determined by the first projection device, is generated in response to a drag operation on the first display object within the first projection window displayed on the control device, rather than a drag operation on the first display object within the first projection device's own interface. Furthermore, by intercepting the original response logic of the first drag event, preventing the generation of native drag animation effects, or preventing the data (animation data) rendered for directly displaying native drag animation effects on the first projection window's display view layer from being fed back to the control device, the projection window data of the first projection device does not include the native drag animation effects of the first display object, thus resolving the animation conflict problem in a multi-device projection environment. It should be noted that this application does not limit the implementation method of the first projection device intercepting native drag animation effects. It can be implemented using interception technologies supported by the first projection device's own system platform, such as intercepting system messages through a Hook mechanism and window subclassing, or intercepting system events through an event monitor and CGEventTap.

[0084] In some embodiments, the control device can directly send the first drag event to the first projection device through a projection communication channel established with the first projection device based on a projection protocol. This could be achieved through a SmartConnect projection reverse channel (a proprietary projection protocol that provides a high-efficiency bidirectional communication channel for low-latency event transmission), or through a mechanism for precisely intercepting native drag effects in the first projection event. However, this is not the only possibility. Optionally, the first drag event can also be sent to each projection device via a publish / subscribe message bus. In this case, the first projection device can include the starting position of the drag operation (the initial display position of the first display object), allowing each projection device to calculate whether the starting position is within its corresponding projection window. If so, it acts as the first projection device and performs the interception operation for native drag effects described above.

[0085] Since the control device and the projection device may have differences in display parameters such as resolution and scaling ratio, precise coordinate transformation can be performed so that the projection device can determine whether the starting position of the reception is within its corresponding projection window. This allows the control device to align the display content in each projection window with the screen display content / content to be displayed on the corresponding projection device, ensuring the projection effect. This application does not describe the coordinate transformation process in detail.

[0086] Step S63: Receive a first transmission message sent by the first projection device based on the first drag event, the first transmission message including drag animation visual data of the first display object;

[0087] Step S64: Display rendering is performed based on the drag animation visual data in the first transmission message, and global drag animation is presented on the global view layer;

[0088] In one possible implementation, the visual data of the drag animation used by the control device to render and display the global drag animation can come from the control device's own storage device, such as the drag animation visual data determined after pre-configuration or default drag animation. This ensures the consistency of the global drag animation throughout the entire drag operation, without having to consider the differences between the platforms of each projection device.

[0089] Preferably, in order to achieve personalized display of drag effects for different projection devices, the aforementioned drag animation visual data can originate from the source device of the first display object, i.e., the first projection device. In this case, during the process of the first projection device responding to the first drag event, it can also determine the drag animation visual data of the first display object (i.e., the native drag animation visual data), such as a screenshot of the first display object or a pre-configured drag animation icon, thumbnail, or lightweight drag animation code, and then send a first transmission message containing the drag animation visual data. That is, the first transmission message can also include the drag animation visual data, so that the control device can parse the first transmission message and render a unified drag animation (global drag animation) on the global view layer based on the drag animation visual data, so that the global drag animation is visually highly similar to or completely consistent with the native drag animation displayed when the user performs a local drag operation on the first projection device, reducing the user's cognitive burden and achieving a seamless cross-window experience. This application does not limit the rendering architecture for implementing step S64.

[0090] Among them, such as Figure 5 As shown, during the user's drag operation on the first display object, the control device can detect the position of the drag operation in real time, that is, the position where the first display object is dragged. Based on this position and the visual data of the drag animation, the display rendering can be performed. The global drag animation can be presented on the global view layer at this position, or the global drag animation presented in the global view layer can be dynamically adjusted based on this position. This avoids repeated rendering and thus avoids wasting resources. In this way, the global drag animation moves across the projection window following the user's drag operation trajectory, eliminating the problem of visual disjointedness, ensuring natural and smooth operation, and facilitating intuitive operation for the user.

[0091] In one possible implementation, the global drag-and-drop animation can be obtained by rendering the drag-and-drop animation visual data through a rendering engine. Alternatively, a global view layer (such as the highest display level of the Z-axis display layer) independent of each projection window can be created at the top display layer. The drag-and-drop animation visual data can be drawn as a texture on the global view layer, thereby instantiating the visual entity of the global drag-and-drop animation. For example, the system-level drag can be initiated by calling the startDragAndDrop method (a View class method provided by the Android system to start drag-and-drop operations, allowing users to drag view elements with touch gestures and place them at the target position). The received animation screenshot (a kind of drag-and-drop animation visual data) can be drawn as a coherent drag-and-drop animation (global drag-and-drop animation) that follows the movement trajectory of the drag operation through DragShadowBuilder (an abstract class used to create and update drag shadows during drag operations, which identify the dragged display object). This ensures that the global drag-and-drop animation is seamless and has no delay when crossing different projection windows.

[0092] In some embodiments, the first transmission message can be transmitted via a publish-subscribe message bus, such as an MQTT communication bus, to a preset projection topic (topic / dragFile), enabling devices subscribed to that projection topic to receive messages published to that topic, such as the first transmission message. It should be noted that in a multi-device projection environment, other broadcast methods can also be used to achieve message synchronization between multiple devices; this application only uses MQTT message synchronization as an example.

[0093] For example, such as Figure 7 As shown, the control device and multiple projection devices are all connected to a publish / subscribe message bus. Figure 7 Taking the MQTT message bus as an example, the first screen-sharing device subscribes to a preset screen-sharing topic (such as topic / dragFile) in the message bus. In response to the first drag event, the first screen-sharing device publishes the first transmission message to the screen-sharing topic through the message bus. That is, the first screen-sharing device uses its own deployed message client (such as...) Figure 7 The MQTT client shown sends a message to the message server (such as...). Figure 7 In the MQTT server shown, the control device receives the first transmission message under the MQTT topic through the message bus. That is, the MQTT client deployed on the control device itself obtains the first transmission message from the subscribed MQTT topic. Simultaneously, other MQTT devices subscribed to the MQTT topic (such as...) receive the message. Figure 7The second screen-casting device shown may also include other screen-casting devices, etc. It can also obtain and cache (such as storing in a queue or other form) the first transmission message from the screen-casting topic subscribed to in the MQTT server through its own MQTT client, in order to wait for subsequent drag-and-release operations.

[0094] Step S65: In response to the drag-and-release operation of the global drag-and-drop animation, the second projection device is triggered to obtain the file data corresponding to the first display object through the direct communication channel between the second projection device and the first projection device, based on the data storage information contained in the first transmission message.

[0095] Following the above analysis, such as Figure 5 In the scenario shown, the user moves the first display object from the first projection window (e.g., ...). Figure 5 Drag the office window shown to the target projection window (such as...). Figure 5 The conference device window shown can be any other projection window different from the first projection window (denoted as the second projection window). Within this window, drag operations on the first display object can be released, such as lifting the mouse button or removing the stylus from the touchscreen. The control device receives the drag release operation on the first display object and, in response to the drag release operation, ends the global drag animation. For example, by calling the graphical interface to remove the global view layer, the global drag animation is cleared from the display screen, thus ending the complete lifecycle of a global drag animation.

[0096] In this embodiment of the application, the purpose of dragging the first display object is to directly transmit the file data (such as image data, audio data, text data, or video data, or one or more) corresponding to the first display object in the first projection device to the second projection device. When the user performs a drag-and-release operation on the global drag animation, the projection device corresponding to the projection window at the location of the drag-and-release operation (i.e., the drag-and-release position) can be determined based on the position of the drag-and-release operation. This allows the projection device to be notified to directly obtain the file data corresponding to the first display object from the first projection device, without the need for the control device to forward the file data.

[0097] As can be seen, the control device of this application can handle lightweight position mapping and graphics rendering, and forward the first transmission message between multiple projection devices. It does not participate in the transmission of file data corresponding to the first display object, but is directly completed by different projection devices through a direct communication channel, thus realizing the separation of message control flow and data flow. Compared with the method of transmitting file data by means of control device, this application improves the system response speed and resource utilization.

[0098] The data storage information in the first transmission message is used to locate and access the file data corresponding to the first display object, and may include the storage path of the file data in the NFS shared directory. Thus, for example... Figure 7As shown, during the drag-and-release operation response, the control device can trigger the second projection device (i.e., the projection device corresponding to the second projection window where the drag-and-release position is located) to directly access the shared storage path of the first projection device (such as the absolute path of the file data in the NFS shared directory) through the direct communication channel between the control device and the first projection device, in accordance with the NFS protocol, to obtain the file data corresponding to the first display object. That is, the file data is downloaded from the NFS shared directory to the local machine of the second projection device, thereby quickly realizing point-to-point fast file data transmission without the need to transfer to the control device.

[0099] The process of mounting an NFS shared directory on multiple screen-projecting devices includes two steps: "establishing a direct communication channel" and "establishing an NFS connection." After the connection is established, the second screen-projecting device performs an NFS mounting operation, mapping the NFS shared directory of the first screen-projecting device to a local virtual directory. In this way, when it is necessary to obtain the shared storage path of the first screen-projecting device, the storage path of the virtual directory can be directly accessed, and the file data corresponding to the first display object in the first screen-projecting device can be directly read. However, this data transmission implementation method is not limited to this method.

[0100] In some embodiments, during the process of the control device triggering the second projection device to acquire file data, the control device can directly determine the second projection device corresponding to the second projection window where the drag-and-release position is located, and directly send a data acquisition message to the second projection device. This data acquisition message includes data storage information, thereby instructing the second projection device to acquire the file data under the data storage information. For example, the control device triggers a drag-and-release event in its global view layer, identifies its location in the second projection window of the second projection device by the drag-and-release position, and then terminates the system-level global drag animation. At the same time, through the projection communication channel established between the control device and the second projection device based on the projection protocol, it transmits the data acquisition message containing data storage information to the second projection device, such as through the SmartConnect projection reverse channel, so that the second projection device can download the file data corresponding to the first display object to its local machine based on the data storage information through the direct communication channel with the first projection device.

[0101] In one possible implementation, the data acquisition message may also include a second device identifier of the second projection device. In distributed message broadcasting, each projection device uses this identifier to determine if it belongs to the second projection device before acquiring the file data corresponding to the first display object according to the method described above, thus avoiding the mistaken acquisition of data shared by the first projection device with other projection devices (the third projection device). It may also include a first device identifier of the first projection device (such as a device ID, which may come from the first transmission message sent by the first projection device), allowing the second projection device to verify whether it has access rights to the file data corresponding to the first display object from the first projection device. If it does not have access rights, the steps described above for acquiring the file data are prohibited; if it has access rights, the steps described above for acquiring the file data are then executed, thereby improving data access security.

[0102] Optionally, in another implementation of the control device triggering the second projection device to obtain file data, the control device, according to, but not limited to, the method described above, determines a drag-and-release event containing the drag-and-release position, and sends the drag-and-release event to each projection device. This allows each projection device to determine that it belongs to the second projection device based on the drag-and-release event, and then, in response to the cached data storage information, obtains the file data corresponding to the first display object. For example, it may invoke the NFS protocol and, based on the previously stored NFS path of the file data from the first projection device, download the file data corresponding to the first display object from the NFS shared target to its local machine. It should be noted that the triggering implementation methods for the control device to trigger the second projection device to obtain the file data corresponding to the first display object include, but are not limited to, the two methods described above.

[0103] In some embodiments, the first transmission message sent by the first projection device may also include a drag start timestamp of the drag operation, so that if each projection device and control device does not detect the corresponding global drag animation / drag release operation / drag release event of the first display object within a preset cache time, the message stored for the first display object within the preset cache time, such as the first transmission message, is cleared to release the memory resources it occupies. In particular, when the first transmission message contains a large amount of data, such as drag animation visual data, and occupies a lot of memory resources, invalid message data is cleared in time to avoid memory resource shortage affecting the processing of other messages. However, it is not limited to the memory resource release triggering mechanism described in this application. It is also possible to clear the temporarily stored message data when the memory resource occupation reaches a threshold.

[0104] Reference Figure 8This is a flowchart illustrating a control method proposed in Embodiment 3 of this application. The method proposed in this embodiment can be applied to a first screen-sharing device among multiple screen-sharing devices, i.e., a screen-sharing device acting as a data sending end. Multiple screen-sharing devices are communicatively connected to the same control device, and each device's corresponding screen-sharing window is independently displayed on the control device's display interface. In this multi-device screen-sharing environment, such as... Figure 8 As shown, the control method executed by the first projection device may include:

[0105] Step S81: Receive a first drag event sent by the control device; the first drag event is generated by the control device in response to the user's drag operation on the first display object in the first projection window of the first projection device;

[0106] The method of generating the first drag event can be referred to the description of the corresponding part of the control method embodiment described above from the control device side. It can be generated by the control device itself responding to the drag operation and sending it to the first projection device, or it can be generated by the first projection device itself detecting the drag operation, etc. The implementation process will not be described in detail in this embodiment.

[0107] Step S82: Intercept the native drag animation triggered by the first drag event on the first screen projection device, so that the screen projection window data of the first screen projection device does not include the native drag animation of the first display object;

[0108] Step S83: In conjunction with the control device, display the global drag animation corresponding to the drag operation on its display interface; the global drag animation is configured to follow the user's drag operation trajectory and move across each projection window to drag the first display object to the second projection window of the destination second projection device.

[0109] In this embodiment, to address the issues of animation conflicts and poor operational continuity during dragging across projection windows, it is proposed that after the first projection device receives the first drag event, it promptly intercepts its own native drag animation. This prevents the native drag animation from being displayed on its own screen, thus avoiding overlap with the global drag animation displayed on the control device's global view layer, which is independent of each projection window, when it is simultaneously displayed in the display view of the first projection window. Therefore, the interception step performed by the first projection device, in conjunction with the control device displaying only the global drag animation on its display interface, defines the role of the first projection device in the entire cross-device dragging scheme as a data sender, not a drag animation rendering end, thus avoiding drag animation conflicts and improving the dragging interaction experience across window display objects.

[0110] It should be noted that this application does not limit the implementation method of how the first projection device intercepts native drag-and-drop animation effects. In one possible implementation, the first projection device can respond to the animation interception command corresponding to the first drag event, such as the animation interception command generated and sent by the control device or generated by the rules pre-stored by the first projection device, thereby controlling the drag-and-drop animation rendering module in the first projection device to be in a closed state (such as switching from an open state to a closed state, or keeping it closed, prohibiting state switching), so as to prevent the generation of native drag-and-drop animation effects of the first projection device. Optionally, the first projection device can also generate projection data of native drag-and-drop animation effects, such as native drag-and-drop animation visual data, in response to the first drag event by its own drag-and-drop animation rendering module, but the first projection device is prohibited from sending the projection data to the control device, so that the native drag-and-drop animation effects are not displayed in the first projection window, thereby eliminating the phenomenon of flickering or jumping of native drag-and-drop animation effects from the root, and avoiding the overlapping and conflict problems of multiple sets of drag-and-drop animation effects displayed at the same time, ensuring consistent cross-platform experience, and improving interactive experience and performance.

[0111] The above describes a control method provided by an embodiment of this application. The following describes an apparatus for performing the above control method.

[0112] Reference Figure 9 This application provides a schematic diagram of the structure of a control device, which can be applied to control devices, such as... Figure 9 As shown, the control device proposed in this embodiment may include:

[0113] The screen projection display module 91 is used to independently display the screen projection windows of multiple screen projection devices on the display interface of the control device.

[0114] The drag operation acquisition module 92 is used to acquire the drag operation of the user on the first display object in the first projection window of the first projection device.

[0115] The global drag animation display module 93 is used to display a global drag animation corresponding to the drag operation based on the drag operation and a global view layer independent of each of the projection windows. The global drag animation is configured to follow the user's drag operation trajectory across each of the projection windows to drag the first display object to the second projection window of the destination second projection device.

[0116] In some embodiments, the control device may further include:

[0117] The first drag event sending module is used to send a first drag event to the first projection device in response to the drag operation, so that the projection window data of the first projection device does not include the native drag animation of the first display object.

[0118] In some embodiments, the control device may further include:

[0119] The first transmission message receiving module is used to receive a first transmission message sent by the first projection device based on the first drag event, wherein the first transmission message includes drag animation visual data of the first display object;

[0120] The display rendering module is used to perform display rendering based on the drag animation visual data in the first transmission message, and to present the global drag animation on the global view layer.

[0121] In some embodiments, the first transmission message may further include data storage information for locating and accessing file data corresponding to the first display object; the data storage information may include the storage path of the file data corresponding to the first display object in a shared directory of a network file system. Based on this, the control device may further include:

[0122] The drag-and-release trigger module is used to respond to the drag-and-release operation of the global drag-and-release effect, triggering the second projection device to directly access the storage path shared by the first projection device through the direct communication channel between the second and first projection devices, following the network file system protocol, and obtain the file data corresponding to the first display object; the second projection device is the projection device corresponding to the second projection window where the drag-and-release position is located.

[0123] In one possible implementation, the drag-and-release trigger module may include:

[0124] The first determining unit is used to determine the second projection device corresponding to the second projection window where the drag release position is located;

[0125] A data acquisition message sending unit is used to send a data acquisition message to the second projection device; the data acquisition message contains the data storage information, so that the second projection device responds to the data acquisition message and acquires the file data corresponding to the first display object.

[0126] Optionally, the data acquisition message may be transmitted through a screen projection communication channel established between the control device and the second screen projection device based on a screen projection protocol.

[0127] In one possible implementation, the drag-and-release trigger module may also include:

[0128] The drag-and-release event sending unit is used to send drag-and-release events to each of the screen projection devices, so that after each of the screen projection devices determines that it belongs to the second screen projection device based on the drag-and-release event, it responds to the cached data storage information and obtains the file data corresponding to the first display object.

[0129] Optionally, the drag-and-release event may be sent to each of the screen-casting devices via a publish-and-subscribe message bus, but is not limited thereto.

[0130] In some embodiments, the control device and the plurality of projection devices are all connected to a publish-subscribe message bus and subscribe to preset projection topics in the message bus, so that the first projection device, in response to the first drag event, publishes the first transmission message to the projection topic through the message bus. Based on this, the first transmission message receiving module may include:

[0131] The first receiving unit is configured to receive the first transmission message under the projection topic via the message bus.

[0132] Reference Figure 10 The present application provides a schematic diagram of a control device in Embodiment 2. This control device can be applied to the first screen projection device among multiple screen projection devices. The screen projection windows of each of the multiple screen projection devices are independently displayed on the display interface of the control device, such as... Figure 10 As shown, the control device proposed in this embodiment may include:

[0133] The first drag event receiving module 101 is used to receive a first drag event sent by the control device; the first drag event is generated by the control device in response to the user's drag operation on the first display object in the first projection window of the first projection device;

[0134] Interception module 102 is used to intercept the native drag animation triggered by the first drag event on the first projection device, so that the projection window data of the first projection device does not include the native drag animation of the first display object;

[0135] The configuration processing module 103 is used to cooperate with the control device to display a global drag animation corresponding to the drag operation on its display interface. The global drag animation is configured to follow the user's drag operation trajectory and move across each of the projection windows to drag the first display object to the second projection window of the destination second projection device.

[0136] Optionally, the interception module 102 may include any of the following control units:

[0137] The first control unit is used to respond to the motion effect interception command corresponding to the first drag event and control the drag motion effect rendering module in the first projection device to be turned off to prevent the generation of native drag motion effects;

[0138] The second control unit is used to prevent the first projection device from sending projection data of the native drag animation to the control device, so that the native drag animation is not displayed in the first projection window; the projection data is generated by the drag animation rendering module in the first projection device in response to the first drag event.

[0139] This application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on a computer device (control device or projection device), the computer device implements any of the control methods executed on the corresponding device side provided in this application embodiment. The computer program product can be stored in a readable storage medium, such as a computer floppy disk, USB flash drive, portable hard drive, ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training device, or network device, etc.) to execute the task processing methods described in the various embodiments of this application.

[0140] This application also provides a computer-readable storage medium carrying one or more computer programs. When these programs are executed by a computer device (control device or projection device), the computer device can implement any of the control methods executed on the device side as provided in this application. This application does not limit the product form of the computer-readable storage medium.

[0141] Finally, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0142] In the above embodiments, the invention can be implemented entirely or partially by software, hardware, firmware, or any combination thereof. Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware, or it can be implemented using dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memory, dedicated components, etc. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The various embodiments in this specification are described in a progressive or parallel manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to mutually. For the apparatus, system, and device disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

Claims

1. A control method applied to a control device, wherein the display interface of the control device independently displays projection windows corresponding to multiple projection devices, the method comprising: Get the user's drag operation on the first display object within the first projection window of the first projection device; Based on the drag operation and a global view layer independent of each of the projection windows, a global drag animation corresponding to the drag operation is displayed. The global drag animation is configured to follow the user's drag operation trajectory and move across each of the projection windows to drag the first display object to the second projection window of the destination second projection device.

2. The method according to claim 1, further comprising: In response to the drag operation, a first drag event is sent to the first projection device so that the projection window data of the first projection device does not include the native drag animation of the first display object.

3. The method according to claim 2, further comprising: Receive a first transmission message sent by the first screen projection device based on the first drag event, the first transmission message including drag animation visual data of the first display object; The drag-and-drop animation visual data in the first transmission message is used for display rendering, and the global drag-and-drop animation is presented on the global view layer.

4. The method according to claim 3, wherein, The first transmission message further includes data storage information for locating and accessing file data corresponding to the first display object; the data storage information includes the storage path of the file data corresponding to the first display object in a shared directory of the network file system; the method further includes: In response to the drag-and-release operation of the global drag-and-release animation, the second projection device is triggered to directly access the storage path shared by the first projection device through the direct communication channel between the second and first projection devices, following the network file system protocol, to obtain the file data corresponding to the first display object; the second projection device is the projection device corresponding to the second projection window where the drag-and-release position is located.

5. The method according to claim 4, wherein, The triggering method for the second screen projection device to obtain the file data corresponding to the first display object includes any of the following: The system determines the second projection device corresponding to the second projection window where the drag-and-release position is located, and sends a data acquisition message to the second projection device. The data acquisition message contains the data storage information, so that the second projection device responds to the data acquisition message and acquires the file data corresponding to the first display object. A drag-and-release event is sent to each of the screen projection devices, so that each screen projection device determines that it belongs to the second screen projection device based on the drag-and-release event, and then obtains the file data corresponding to the first display object in response to the cached data storage information.

6. The method according to any one of claims 3-5, wherein, The control device and the plurality of projection devices are all connected to a publish-subscribe message bus and subscribe to a preset projection topic in the message bus, so that the first projection device responds to the first drag event and publishes the first transmission message to the projection topic through the message bus; The receipt of the first transmission message sent by the first projection device based on the first drag event includes: The first transmission message under the projection topic is received through the message bus.

7. The method according to claim 5, wherein: The data acquisition message is transmitted through a screen projection communication channel established between the control device and the second screen projection device based on the screen projection protocol; The drag-and-release event is sent to each of the screen projection devices via a publish-subscribe message bus.

8. A control method applied to a first projection device among multiple projection devices, wherein the projection windows of each of the multiple projection devices are independently displayed on the display interface of the control device, the method comprising: Receive the first drag event sent by the control device; The first drag event is generated by the control device in response to the user's drag operation on the first display object within the first projection window of the first projection device; Intercept the native drag animation triggered by the first drag event on the first screen projection device, so that the screen projection window data of the first screen projection device does not include the native drag animation of the first display object; The control device displays a global drag animation corresponding to the drag operation on its display interface. The global drag animation is configured to follow the user's drag operation trajectory and move across each of the projection windows to drag the first display object to the second projection window of the target second projection device.

9. The method according to claim 8, wherein intercepting the native drag animation triggered by the first drag event on the first projection device includes any of the following implementation methods: In response to the animation interception command corresponding to the first drag event, the drag animation rendering module in the first projection device is controlled to be turned off to prevent the generation of native drag animation. The first projection device is prohibited from sending projection data containing native drag-and-drop animation to the control device, so that the native drag-and-drop animation is not displayed in the first projection window; The projection data is generated by the drag animation rendering module in the first projection device in response to the first drag event.

10. A control system, the system comprising a control device and a plurality of projection devices communicatively connected to the control device, wherein: The display interface of the control device independently presents the projection windows corresponding to each of the multiple projection devices. The control device includes: The user interface is used to obtain the user's drag and drop operation on the first display object in the first projection window of the first projection device. The display screen is used to display a global drag animation corresponding to the drag operation based on the drag operation and a global view layer independent of each of the projection windows. The global drag animation is configured to follow the user's drag operation trajectory across each of the projection windows to drag the first display object to the second projection window of the destination second projection device.

Images