A multi-screen control method and device, a computing device, and a storage medium

Abstract

A multi-screen control method and device, a computing device and a storage medium are disclosed. The multi-screen control method is executed in a computing device connected with multiple displays. The method comprises: providing, in a display management interface, virtual images indicating each display, wherein each virtual image has a first arrangement order, and any virtual image is connected with at least one other virtual image, so that each virtual image forms a first image connected domain; in response to a moving operation on a target virtual image, detecting whether the moved target virtual image is connected with at least one second image connected domain formed by other virtual images; if not, splicing the target virtual image with the second image connected domain closest to the target virtual image to generate a second virtual image arrangement order, so that a user controls each display based on the second virtual image arrangement order.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a multi-screen control method, device, computing equipment, and storage medium. Background Technology

[0002] For users of desktop operating systems, connecting multiple displays to computing devices is almost a necessity. For example, most mainstream operating systems nowadays support dual screens in software, and users can customize the screen arrangement in the operating system's settings.

[0003] However, users' needs for multi-screen setups go beyond just dual screens; they want operating systems to offer unlimited multi-screen splicing capabilities. However, existing multi-screen control technologies require all displays to be arranged horizontally or vertically in the order they are connected, with forced top or left alignment, which has significant limitations. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a multi-screen control method, apparatus, computing device, and storage medium that overcomes or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a multi-screen control method is provided, executed in a computing device connected to a plurality of displays. The method includes: providing virtual images that indicate each display in a display management interface, wherein each of the virtual images has a first arrangement order, and any virtual image is connected to at least one other virtual image such that the virtual images form a first image connectivity region; in response to a movement operation on a target virtual image, detecting whether the moved target virtual image is connected to at least one second image connectivity region formed by other virtual images; if not, concatenating the target virtual image with its nearest second image connectivity region to generate a second virtual image arrangement order, so that a user can control each display based on the second virtual image arrangement order.

[0006] Optionally, in the multi-screen control method according to the present invention, before the step of responding to the movement operation of the target virtual image, the method further includes: obtaining the position information of each virtual image on the display management interface; and forming an identifier for each virtual image based on the position information.

[0007] Optionally, in the multi-screen control method according to the present invention, the method further includes the step of: for each virtual image, recording the connectivity relationship between it and other virtual images connected to it.

[0008] Optionally, in the multi-screen control method according to the present invention, the step of detecting whether the moved target virtual image is connected to at least one second image connected component formed by other virtual images includes: determining whether the target virtual image is connected to other virtual images based on the connectivity relationship of the target virtual image.

[0009] Optionally, in the multi-screen control method according to the present invention, the step of stitching the target virtual image with its nearest second image connected component includes: determining the nearest second image connected component to the target virtual image based on location information; determining a first stitching path between the target virtual image and its nearest second image connected component; and moving the target virtual image to connect with its nearest second image connected component according to the first stitching path.

[0010] Optionally, in the multi-screen control method according to the present invention, before the step of detecting whether the moved target virtual image is connected to at least one second image connected region formed by other virtual images, the method further includes: obtaining the distance between the target virtual image and each second image connected region during the movement based on position information; when the distance is less than a distance threshold, moving the target virtual image to the edge of the second image connected region closest to it, so that the target virtual image moves along the edge.

[0011] Optionally, in the multi-screen control method according to the present invention, after the step of stitching the target virtual image with its nearest second image connected component, the method further includes: determining the image region formed by connecting the target virtual image with its nearest second image connected component as a third image connected component; detecting whether each second image connected component and the third image connected component are connected; and stitching each second image connected component that is not connected to the third image connected component with the third image connected component.

[0012] Optionally, in the multi-screen control method according to the present invention, the step of detecting whether each second image connected component and the third image connected component are connected includes: for each second image connected component, determining whether at least one virtual image among its included virtual images has a connection relationship with any virtual image in the third image connected component; if so, the second image connected component is connected to the third image connected component.

[0013] Optionally, in the multi-screen control method according to the present invention, the method further includes the step of sorting and deduplicating the connection relationships of each virtual image.

[0014] Optionally, in the multi-screen control method according to the present invention, the step of stitching each second image connected domain that is not connected to the third virtual image connected domain with the third image connected domain includes: determining a second stitching path between the second image connected domain and the third image connected domain based on position information; and moving the second image connected domain to be connected with the third image connected domain according to the second stitching path.

[0015] According to another aspect of the present invention, a computing multi-screen control device is provided, residing in a computing device connected to a plurality of displays. The device includes: a display module adapted to provide, on a display management interface, virtual images indicating each display to be displayed, wherein each virtual image has a first arrangement order, and any virtual image is connected to at least one other virtual image, such that the virtual images form a first image connectivity region; a detection module adapted to, in response to a movement operation of a target virtual image, detect whether the moved target virtual image is connected to at least one second image connectivity region formed by other virtual images; and a stitching module adapted to, when detecting whether the moved target virtual image is not connected to at least one second image connectivity region formed by other virtual images, stitch the target virtual image with its nearest second image connectivity region and generate a second virtual image arrangement order, so that a user can control each display based on the second virtual image arrangement order.

[0016] According to another aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the methods described above.

[0017] According to another aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the method described above.

[0018] According to the solution of this invention, when a user accesses multiple screens, they can complete the corresponding screen splicing by dragging and pointing virtual images of each display on the multi-screen operation interface. Users can customize the screen splicing, giving them maximum freedom. This makes user operation more convenient and provides a more user-friendly experience.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0021] Figure 1 A block diagram of the physical components (i.e., hardware) of the computing device 100 is shown;

[0022] Figure 2 A flowchart of a multi-screen control method 200 according to an embodiment of the present invention is shown;

[0023] Figure 3 A schematic diagram of a display management interface according to an embodiment of the present invention is shown;

[0024] Figure 4 A schematic diagram illustrating the movement process of a target virtual image according to an embodiment of the present invention is shown;

[0025] Figure 5 A schematic diagram of target virtual image movement provided according to another embodiment of the present invention is shown;

[0026] Figure 6 A schematic diagram of target virtual image stitching according to an embodiment of the present invention is shown;

[0027] Figure 7 A flowchart illustrating the determination of a first splicing path according to an embodiment of the present invention is shown;

[0028] Figure 8 A schematic diagram showing the splicing of a second image connected component and a third image connected component according to an embodiment of the present invention is shown;

[0029] Figure 9 A schematic diagram of the structure of a multi-screen control device 900 according to an embodiment of the present invention is shown. Detailed Implementation

[0030] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0031] With the continuous development of computing devices, most can now support multiple displays simultaneously. Connecting multiple external monitors greatly improves work efficiency and facilitates presentations. However, in current technology, after multiple monitors are connected to a computing device, the device defaults to arranging the monitors horizontally or vertically according to their connection order, forcibly aligning them top- or left-hand to control the display. Even if the user adjusts the monitor's position, the display control remains unchanged, and the corresponding virtual image cannot be dragged to calibrate the monitor's position. This is a significant limitation for users who need multiple monitors, and this defect becomes more pronounced as more monitors are connected.

[0032] To address the problems existing in the prior art, the present invention is proposed. One embodiment of the present invention provides a multi-screen control method that can be executed in a computing device. Figure 1 A block diagram of the physical components (i.e., hardware) of a computing device 100 is shown. In a basic configuration, the computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 may be implemented as a processor. The system memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memory. According to one aspect, the system memory 104 includes an operating system 105 and a program module 106, the operating system 105 including a display manager 120 for executing the search method of the present invention.

[0033] According to one aspect, operating system 105 is, for example, suitable for controlling the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1 The basic configuration is illustrated by the components within the dashed lines 108. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1 The image is shown by removable storage device 109 and non-removable storage device 110.

[0034] As stated above, according to one aspect, a program module 106 is stored in system memory 104. According to one aspect, program module 106 may include one or more applications. The present invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc.

[0035] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1 Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0036] According to one aspect, computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0037] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computer device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0038] According to one aspect, a communication medium is implemented by computer-readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, a communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0039] In one embodiment of the present invention, a computing device 100 includes one or more processors and one or more readable storage media storing program instructions. When the program instructions are configured to be executed by one or more processors, the computing device performs the multi-screen control method of the present invention.

[0040] Figure 2 A flowchart of a multi-screen control method 200 according to an embodiment of the present invention is shown. Method 200 is adapted to be executed in a computing device (e.g., the aforementioned computing device 100). The computing device 100 may be connected to multiple displays.

[0041] like Figure 2 As shown, the purpose of method 200 is to implement a multi-screen control method that allows users to customize and drag virtual images that identify each monitor to achieve arbitrary splicing display between monitors. Method 200 supports arbitrary splicing of virtual images with intersecting points and lines under fully connected conditions, making user operation more convenient and user-friendly.

[0042] Method 200 begins with step S202, in which a virtual image representing each monitor is provided in the display management interface. The display management interface is provided by a display manager, which controls each monitor. For example, the user can use the display manager to set information such as monitor resolution and splicing objects. When multiple monitors are connected to the computing device, the display manager generates a virtual image representing each monitor based on its connection order, position, and resolution.

[0043] Figure 3 A schematic diagram of a display management interface according to an embodiment of the present invention is shown. The display management interface includes five virtual images, indicating that the computing device is connected to five displays. It should be noted that this embodiment uses five displays as an example; in practical applications, any number of displays can be connected as needed. This application does not limit this.

[0044] The display manager records the position information (e.g., position coordinates) of each virtual image within the display management interface. Based on this position information, it generates an identifier for each virtual image. For example, if a computing device connects to five monitors sequentially, the virtual images corresponding to the five monitors are assigned identifiers according to the order or position of the monitor connections. For instance, virtual image 1 represents the virtual image of the first monitor connected to the computing device; virtual image 2 represents the virtual image of the second monitor connected to the computing device, and so on.

[0045] It's easy to understand that the position of virtual images in the display management interface generally corresponds to the position of their respective monitors. For example, if virtual image 2 is to the right of virtual image 1, then monitor 2 is also placed to the right of monitor 1; if virtual image 4 is below virtual image 1, then monitor 4 is also placed below monitor 1. In other words, when monitor 2 is placed to the right of monitor 1, during initialization, virtual image 2 will also be formed to the right of virtual image 1. Similarly, when monitor 4 is placed below monitor 1, during initialization, virtual image 4 will also be formed below virtual image 1.

[0046] In this system, each virtual image is connected to at least one other virtual image, forming a connected component (a region in an image consisting of pixels with the same pixel value and adjacent positions). In other words, any image in the connected component has other adjacent but non-overlapping images. This connectivity can be a point-to-point connection (e.g., ...). Figure 3 Virtual images 1 and 5 can be connected as lines (e.g., virtual images 1 and 5), or they can be connected as lines (e.g., virtual images 1 and 5). Figure 3 The virtual images (1 and 2) can also be point-line connections, etc., which will not be listed here.

[0047] The virtual images are arranged in a first order according to their access sequence, allowing users to control each display. For example, to move the cursor from display 1 (corresponding to virtual image 1) to display 2 (corresponding to virtual image 2), the user simply moves the cursor to the right edge of display 1 and continues moving it to the right. Similarly, to move the cursor from display 1 to display 4 (corresponding to virtual image 4), the user moves the cursor to the bottom edge of display 1 and continues moving it downwards.

[0048] In addition, for each virtual image, the display manager records the connectivity relationships of all connected virtual images in real time. For example, for virtual image 1, which is connected to virtual images 2, 4, and 5, the connectivity relationships of virtual image 1 can be recorded as 1-2, 1-4, and 1-5. Similarly, the connectivity relationships of virtual image 2 can be recorded as 2-1, 2-4, and 2-5. And so on, recording the connectivity relationships of all virtual images.

[0049] It is worth noting that connectivity relation 1-2 in virtual image 1 is duplicated with connectivity relation 2-1 in virtual image 2. To reduce system performance loss and computational load on computing devices, in some embodiments, the connectivity relations of each virtual image are sorted and deduplicated. For example, connectivity relation 1-2 in virtual image 1 or connectivity relation 2-1 in virtual image 2 can be removed, retaining only one of them.

[0050] Subsequently, in step S204, in response to the movement operation of the target virtual image, it is detected whether the moved target virtual image is connected to at least one second image connectivity component formed by other virtual images. Specifically, the user can move the target virtual image by dragging with the mouse or by using keyboard shortcuts.

[0051] Preferably, during the user's movement of the target virtual image, the display manager obtains the distance between the target virtual image and each connected component of the second image in real time based on the target virtual image's position information. When the distance is less than a distance threshold, the target virtual image is moved to the edge of the nearest connected component of the second image, so that the target virtual image moves along that edge. In other words, during the movement of the target virtual image, when the distance between it and a certain connected component of the second image is less than the distance threshold, it will "stick" to the edge of that connected component of the second image and move accordingly.

[0052] In a specific example, refer to Figure 4 , Figure 4A schematic diagram illustrating the movement process of a target virtual image according to an embodiment of the present invention is shown.

[0053] In the first state, virtual images 1-5 constitute the first image connected component. Virtual image 3 is the target virtual image, meaning virtual image 3 is the virtual image being moved.

[0054] In the second state, virtual image 3 is moved, the first image connected region is destroyed, and the remaining virtual images 1, 2, 4 and 5 constitute a second image connected region. When the distance between virtual image 3 and the second image connected region is less than the distance threshold, virtual image 3 will be snapped to the edge of the second image connected region.

[0055] Specifically, the display manager can obtain the distance between the virtual image 3 and the connected domain of the second image in real time based on the coordinates of the center point of the virtual image 3 and the coordinates of the center points of the other virtual images.

[0056] The distance threshold can be set by those skilled in the art according to actual needs; for example, it can be 2 centimeters. This application does not limit it in this regard.

[0057] In the third state, the virtual image 3 moves along the edge of the connected domain of the second image.

[0058] Furthermore, in some application scenarios, when the target virtual image is moved, the remaining virtual images may form two or even more second image connected components. (Reference) Figure 5 , Figure 5 A schematic diagram of target virtual image movement provided according to another embodiment of the present invention is shown.

[0059] In the first state, virtual images 1-5 constitute the first image connected component. Virtual image 2 is the target virtual image, that is, virtual image 2 is the virtual image that is being moved.

[0060] In the second state, virtual image 2 is moved, and the first image connected component is destroyed, wherein virtual image 1 and virtual image 4 form a second image connected component; virtual image 3 and virtual image 5 form a second image connected component.

[0061] In the third state, when the distance between virtual image 2 and the second image connected region formed between virtual image 3 and virtual image 5 during the movement is less than the distance threshold, virtual image 2 moves along the edge of the second image connected region formed by virtual image 3 and virtual image 5.

[0062] It should be noted that if the target virtual image moves continuously along the edge of a certain second image connected region, then the target virtual image will eventually be connected to that second image connected region. However, if the target virtual image cannot move along the edge of a certain second image connected region, it is necessary to detect whether the moved target virtual image is connected to at least one second image connected region. Specifically, based on the connectivity of the target virtual image, it can be determined whether the target virtual image is connected to other virtual images. That is, if the target virtual image is not connected to any other virtual image, it means that the target virtual image is not connected to any of the second image connected regions.

[0063] When the target virtual image is not connected to any of the connected components of the second image, step S206 is executed to concatenate the target virtual image with the connected component of the nearest second image to generate the arrangement order of the second virtual images, so that the user can control each display based on the arrangement order of the second virtual images.

[0064] Specifically, firstly, based on the positional information between the virtual images, the second connected component of the image closest to the target virtual image is determined. The display manager then sequentially obtains the distance values ​​between the target virtual image and other virtual images based on the center point coordinates of the target virtual image and the center point coordinates of the other virtual images, and determines the second connected component of the image containing the virtual image with the smallest distance value as the second connected component closest to the target virtual image.

[0065] Then, a first stitching path is determined between the target virtual image and its nearest connected component of the second image. This can also be understood as determining a first stitching path between the target virtual image and its nearest virtual image. Based on the position coordinates of the target virtual image and its nearest neighboring virtual image, the display manager determines the direction and distance the target virtual image will move towards the nearest virtual image, thereby determining the first stitching path.

[0066] Finally, based on the first stitching path, the target virtual image is moved to the second image connected domain that is closest to it.

[0067] In a specific example, refer to Figure 6 , Figure 6 A schematic diagram of target virtual image stitching according to an embodiment of the present invention is shown.

[0068] like Figure 6 As shown, virtual image 3 is moved from the first state to the second state (where 3-1, 3-2, 3-3, and 3-4 represent the positions of virtual image 3 after the move). The display manager will automatically stitch virtual image 3 to the nearest virtual image, ultimately forming the third state.

[0069] In the first state, the user controls the displays based on a first arrangement order. For example, to move the cursor from display 1 to display 3, the cursor must first be moved right to display 2, then moved right from display 2, and finally reach display 3. In the third state, the user controls the displays based on a second arrangement order. For example, after the virtual image 3 has moved to position 3-1, to move the cursor from display 1 to display 3, the cursor can be moved directly upwards to display 3.

[0070] In a specific example, the method for splicing the target virtual image and the connected components of the second image is described in detail. First, the distances between the center points of the target virtual image and the other virtual images are arranged in ascending order.

[0071] Then, the virtual image with the smallest distance is selected as the virtual image to be stitched together. For example, Figure 6 In the process, when virtual image 3 is at position 3-1, the distance between virtual image 1 and virtual image 3 is the smallest, so virtual image 1 is determined to be the virtual image to be stitched together. When virtual image 3 is at position 3-2, the distance between virtual image 2 and virtual image 3 is the smallest, so virtual image 2 is determined to be the virtual image to be stitched together.

[0072] Next, the direction and distance of movement of the target virtual image are determined (i.e., the first stitching path is determined).

[0073] Finally, based on the determined direction and distance of movement, the target virtual image is moved so that it is connected to its nearest virtual image.

[0074] In a specific example, refer to Figure 7 , Figure 7 A flowchart illustrating the determination of a first splicing path according to an embodiment of the present invention is shown.

[0075] First, the display manager adds the identifiers of all virtual screens to a list of items.

[0076] Then, when the target virtual image is moved, the currently moved virtual image (i.e., the target virtual image) is marked in the item list.

[0077] Subsequently, the distances between the center points of the target virtual image and other virtual images are obtained and sorted in ascending order to determine the virtual image closest to the target virtual image.

[0078] Next, the X-axis coordinates of the right border of the target virtual image are sorted in ascending order along the first axis, while the X-axis coordinates of the left borders of all other virtual images are sorted in ascending order along the second axis. The Y-axis coordinates of the top border of the target virtual image are then sorted in ascending order along the second axis, while the Y-axis coordinates of the bottom borders of all other virtual images are sorted in ascending order. This determines the relative position of the target virtual image to the other virtual images in the display manager.

[0079] Next, obtain the first index value (indexLeft, indexRight) of the target virtual image, that is, determine whether the target virtual image needs to be moved to the left or right in the X-axis direction, and determine the distance to be moved; obtain the second index value (indexTop, indexBottom) of the target virtual image, that is, determine whether the target virtual image needs to be moved up or down in the Y-axis direction, and determine the distance to be moved.

[0080] The movement of the target virtual image is divided into three cases: moving only to the left or right (e.g., moving only to the left or right). Figure 6 3-3), move only up or down (e.g. Figure 6 3-1) moves both left and right (e.g.) Figure 6 (3-2 in the middle).

[0081] For images that move only upwards or downwards, identify the virtual images that coincide with the target virtual image in the X-axis direction from the first ascending order. Sort the coordinate values ​​of the identified virtual images and the target virtual image in the Y-axis direction. Based on the sorted index values, determine which direction the target virtual image needs to move towards the virtual image to be stitched (the virtual image with the smallest difference in coordinate values ​​in the Y-axis direction from the target virtual image is identified as the virtual image to be stitched).

[0082] For movements that are only left or right, the virtual images that coincide with the target virtual image in the Y-axis direction are determined from the second ascending order. The coordinate values ​​of the determined virtual images and the target virtual image in the X-axis direction are sorted. Based on the sorted index values, it is determined in which direction the target virtual image needs to move towards the virtual image to be stitched (the virtual image with the smallest difference in coordinate value in the X-axis direction from the target virtual image is determined as the virtual image to be stitched).

[0083] It should be noted that for some special application scenarios, the target virtual image can be stitched by moving only left and right, or by moving only up or down (e.g., Figure 6(3-4 in the text). At this point, it is necessary to determine the difference between the distance it moves to the left or right and the distance it moves up or down. If the distance it moves to the left or right is less than the distance it moves up or down, then choose to move to the left or right; if the distance it moves up or down is less than the distance it moves to the left or right, then choose to move up or down.

[0084] For moves that are both left and right, the direction of the current target virtual image is determined based on the index value, and the move is made to the nearest vertex to achieve a fixed-point intersection.

[0085] Next, the calculated movement distance dpoint(dx, dy) is used, and after confirming the movement distance is correct, the target virtual image is moved. It is worth noting that for special application scenarios (such as...), Figure 6 In step 3-4, it is necessary to determine whether the target virtual image is moving left or right, or up or down.

[0086] In some embodiments, to improve stitching stability and accuracy, collision detection is performed during the stitching process of the target virtual image into the connected component of the second image. When a collision is detected between the target virtual image and another virtual image, the movement of the target virtual image is stopped, indicating that the target virtual image has been successfully stitched with the connected component of the second image.

[0087] If the target virtual image, after being moved, results in multiple second image connected regions, the image region formed by connecting the target virtual image with its nearest second image is defined as the third image connected region. The connectivity between each second and third image connected region is then checked. Each second image connected region that is not connected to the third virtual image connected region is then concatenated with the third image connected region.

[0088] Specifically, for each second image connected component, it is determined whether at least one of the virtual images included in it is connected to any virtual image in the third image connected component. If so, it indicates that the second image connected component is connected to the third image connected component. In other words, as long as at least one virtual image in the second image connected component is connected to the third image connected component, it indicates that the second image connected component is connected to the third image connected component.

[0089] If there is a second image connected domain that is not connected to the third image connected domain, then based on the position information of each virtual image, a second stitching path is determined between the second image connected domain and the third image connected domain, and according to the second image stitching path, the second image connected domain is moved to be connected to the third image connected domain.

[0090] In a specific example, firstly, based on the connectivity of each virtual image, the connected components of the second images that are not connected to the connected component of the third image are identified. Then, the DX and DY values ​​(i.e., the distance to be moved) of each virtual image in these second image connected components to the third virtual image are calculated. The required DX and DY values ​​for each virtual image are sorted by their minimum values, and the minimum value is taken as the DX and DY value for moving the corresponding second image connected component to the third image connected component. Then, the second image connected component is moved to the third image connected component based on the lowest DX and DY value to achieve concatenation with the third image connected component. This process is repeated for all second image connected components.

[0091] refer to Figure 8 , Figure 8 A schematic diagram showing the splicing of a second image connected region and a third image connected region according to an embodiment of the present invention is shown.

[0092] like Figure 8 As shown, in the first state, the second image connected component formed by the target virtual image 2, virtual image 3, and virtual image 5 is concatenated to form the third image connected component. The second image connected component is formed by virtual image 1 and virtual image 4. The second and third image connected components are not connected. After automatically adjusting the second and third image connected components, the second and third image connected components become connected (in the second state).

[0093] For the specific assembly process, please refer to the above. Figure 7 The splicing process shown will not be described in detail here.

[0094] The method 200 provided by this invention can automatically stitch together virtual screens, automatically snap them together when moved, and automatically adjust them after movement. Specifically, the automatic stitching and adjustment functions allow users to customize the stitching of multiple screens on their desktop, giving them maximum freedom. The automatic snapping function allows users to complete the stitching process faster and more accurately, reducing the complexity of the stitching process.

[0095] Figure 9A schematic diagram of a multi-screen control device 900 according to an embodiment of the present invention is shown. The device 900 resides in a computing device (as described above in the computing device 100), which can be externally connected to multiple displays. The device 900 includes a display module 920, a detection module 940, and a splicing module 960 coupled sequentially. The display module 920 is adapted to provide virtual images indicating each display on a display management interface. These virtual images have a first arrangement order, and each virtual image is connected to at least one other virtual image, forming a first image connectivity region. The detection module 940 is adapted to detect, in response to a movement operation on a target virtual image, whether the moved target virtual image is connected to at least one second image connectivity region formed by other virtual images. The splicing module 960 is adapted to, when detecting that the moved target virtual image is not connected to at least one second image connectivity region formed by other virtual images, splice the target virtual image with its nearest second image connectivity region and generate a second virtual image arrangement order, so that the user can control each display based on the second virtual image arrangement order.

[0096] It should be noted that the working principle and process of the device 900 provided in this embodiment are similar to those of the aforementioned method 200. For relevant details, please refer to the description of the aforementioned method 200. Here, they will not be repeated.

[0097] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0098] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the method of the present invention according to instructions in the program code stored in the memory.

[0099] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0100] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of the invention.

[0101] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0102] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0103] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0104] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0105] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0106] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0107] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0108] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative rather than restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A multi-screen control method, executed in a computing device connected to multiple displays, the method comprising: The display management interface provides virtual images that indicate each display screen, wherein each virtual image has a first arrangement order and any virtual image is connected to at least one other virtual image, such that the virtual images form a first image connectivity domain. In response to a movement operation on a target virtual image, it is detected whether the moved target virtual image is connected to at least one second image connected component formed by other virtual images; If not, the target virtual image is concatenated with the nearest connected component of the second image to generate a second virtual image arrangement order, so that the user can control each display based on the second virtual image arrangement order.

2. The method as described in claim 1, wherein, Prior to the step of responding to a movement operation on the target virtual image, the method further includes: Obtain the position information of each virtual image on the display management interface; Based on the location information, an identifier is formed for each virtual image.

3. The method as described in claim 1 or 2, wherein, It also includes the following steps: For each virtual image, record the connectivity relationships between it and all other virtual images connected to it.

4. The method of claim 3, wherein, The step of detecting whether the moved target virtual image is connected to at least one second image connected component formed by other virtual images includes: Based on the connectivity of the target virtual image, it is determined whether the target virtual image is connected to other virtual images.

5. The method of claim 2, wherein, The step of concatenating the target virtual image with its nearest connected component of the second image includes: Based on the location information, the second connected component of the image that is closest to the target virtual image is determined; Determine the first stitching path between the target virtual image and its nearest connected component of the second image; Based on the first stitching path, the target virtual image is moved to the second image connectivity domain that is closest to it.

6. The method of claim 2, wherein, Before the step of detecting whether the moved target virtual image is connected to at least one second image connected component formed by other virtual images, the method further includes: Based on the location information, the distance between the target virtual image and each connected component of the second image is obtained during the movement. When the distance is less than a distance threshold, the target virtual image is moved to the edge of the second image connected region that is closest to it, so that the target virtual image moves along the edge.

7. The method of claim 2, wherein, After the step of concatenating the target virtual image with its nearest connected component of the second image, the method further includes: The image region formed by connecting the target virtual image with its nearest second image connected component is determined as the third image connected component; Detect whether each of the second image connected components and the third image connected components is connected; Each of the second image connected components that is not connected to the third image connected component is concatenated with the third image connected component.

8. The method of claim 7, wherein, The step of detecting whether each of the second image connected components and the third image connected components are connected includes: For each second image connected component, determine whether at least one of the virtual images included in it has a connection relationship with any virtual image in the third image connected component; If so, the second image connected component is connected to the third image connected component.

9. The method of claim 3, wherein, It also includes the following steps: The connection relationships of each virtual image are sorted and deduplicated.

10. The method of claim 7, wherein, The step of concatenating each of the second image connected components that is not connected to the third image connected component with the third image connected component includes: Based on the location information, a second stitching path is determined between the second image connected component and the third image connected component; According to the second stitching path, the second image connected region is moved to be connected with the third image connected region.

11. A multi-screen control device residing in a computing device connected to multiple displays, the device comprising: The display module is adapted to provide virtual images that indicate each display in a display management interface, wherein each of the virtual images has a first arrangement order and any virtual image is connected to at least one other virtual image, such that the virtual images form a first image connectivity domain. The detection module is adapted to detect, in response to a movement operation on a target virtual image, whether the moved target virtual image is connected to at least one second image connected component formed by other virtual images; The stitching module is adapted to stitch the target virtual image with its nearest second image connected component when it detects that the moved target virtual image is not connected to at least one second image connected component formed by other virtual images, and generate a second virtual image arrangement order so that the user can control each display based on the second virtual image arrangement order.

12. A computing device, comprising: At least one processor; and A memory storing program instructions configured to be executed by the at least one processor, the program instructions including instructions for performing the method as claimed in any one of claims 1-10.

13. A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-10.

Images