Cloud application window drag area identification method, window drag method and device

Abstract

Embodiments of the present application provide a method for identifying a drag zone for a cloud application window, a window dragging method, and a device. The method includes: obtaining a top image of the application window; performing contour detection on the top image to determine the largest outer contour in the top image and secondary contours within the largest outer contour; determining the enclosed area formed by adjacent edges of adjacent secondary contours and the largest outer contour; and determining the drag zone of the application window based on at least one enclosed area to prevent the drag operation from affecting the native functions of the application window.

Description

Technical Field

[0001] The present application relates to the field of cloud applications, and more specifically, to a method for identifying a window drag area of ​​a cloud application, a window dragging method, and a device. Background Art

[0002] Virtualized cloud applications are called cloud applications and can be connected to through clients. Unlike standard applications, cloud applications have native application windows on the cloud side. Clients create client-side windows to display cloud applications and receive user input. These client-side windows directly correspond to the native application windows on the cloud side. Users perceive and operate cloud applications directly within the client-side windows, but these windows are not truly native application windows. A core issue with cloud applications is how to make the functionality of client-side windows more similar to that of native application windows.

[0003] Window movement is a basic function and a high-frequency operation of application windows. Users drag the client-side window. After the dragging operation is completed, the window position is reported to the cloud side. During this dragging operation, because the client-side window is not a true native application window, the cloud side needs to configure a drag zone for the client-side window so that users can drag the drag zone to move the client-side window.

[0004] In general cloud application solutions, the entire title bar area at the top of the window is set as the drag zone. However, since the title bar at the top of the window usually contains functional components such as menu buttons and tabs, dragging operations will affect the native functions of the application window. Summary of the Invention

[0005] This application provides a method for identifying the window drag area of ​​a cloud application, a window dragging method and a device, in order to prevent the drag operation from affecting the native functions of the application window.

[0006] In a first aspect, the present application provides a method for identifying a window drag area of ​​a cloud application, which is applied to the cloud side of the cloud application. The method includes:

[0007] Get the top image of the application window;

[0008] Performing contour detection on the top image to determine the largest outer contour in the top image and secondary contours in the largest outer contour;

[0009] A closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour is determined, and a drag area of ​​the application window is determined based on at least one of the closed areas.

[0010] In one implementation, performing contour detection on the top image includes:

[0011] converting the top image into a grayscale image;

[0012] Determining a background color pixel value of the grayscale image, and performing binarization processing on the grayscale image based on the background color pixel value to obtain a binary image;

[0013] Perform contour detection on the binary image.

[0014] In one implementation, determining the background color pixel value of the grayscale image and binarizing the grayscale image based on the background color pixel value to obtain a binary image includes:

[0015] Determine the pixel value with the largest number among the pixel values ​​of the grayscale image, and determine the pixel value with the largest number as the background color pixel value;

[0016] The pixels corresponding to the background color pixel values ​​are set to white, and the pixels corresponding to other pixel values ​​other than the background color pixel values ​​are set to black, to obtain the binary image.

[0017] In one implementation, performing contour detection on the binary image includes:

[0018] performing a closing operation on the binary image;

[0019] Perform contour detection on the binary image after closing operation.

[0020] In one implementation, determining a closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour includes:

[0021] A rectangular area between adjacent sides of adjacent secondary contours and the largest outer contour is determined as the closed area.

[0022] In one implementation, determining the drag area of ​​the application window according to at least one of the closed areas includes:

[0023] The closed area in at least one of the closed areas, the closed area whose size exceeds a preset value, is determined as the drag area of ​​the application window.

[0024] In one implementation, obtaining the top image of the application window includes:

[0025] Get the application window image;

[0026] A screenshot is taken of an area of ​​a preset height at the top of the application window image to obtain the top image.

[0027] In one implementation, obtaining the top image of the application window includes:

[0028] Get the application window image;

[0029] Perform image recognition on the application window to obtain a top image corresponding to the title bar of the application window.

[0030] In a second aspect, the present application provides a cloud application window dragging method, which is applied to the cloud side and client of the cloud application, and the method includes:

[0031] The cloud side obtains the top image of the application window in response to the change in the form of the application window;

[0032] The cloud side performs contour detection on the top image to determine the largest outer contour in the top image and secondary contours in the largest outer contour;

[0033] The cloud side determines a closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour, determines a drag area of ​​the application window based on at least one of the closed areas, and sends the drag area to the client;

[0034] In response to the drag operation on the drag area, the client reports the window position after the drag operation to the cloud side.

[0035] In a third aspect, the present application provides a cloud device, comprising: a memory and a processor;

[0036] The memory is used to store computer programs;

[0037] The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor executes the method according to the first aspect.

[0038] In a fourth aspect, the present application provides a cloud application system, comprising the cloud device and client as described in the third aspect.

[0039] In a fifth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the method described in the first aspect or the second aspect.

[0040] The cloud application window drag area recognition method, window dragging method and device provided in this application perform contour detection on the top image of the application window based on an image processing method, and then exclude the top area that may have other functions based on the detected contour, and retain the blank area as the drag area, thereby ensuring that the dragging behavior of the end-side window does not affect the native function of the application window, thereby optimizing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 This is a flow chart of a method for identifying a window drag area of ​​a cloud application provided in an embodiment of the present application;

[0043] Figure 2 This is a schematic diagram of an application window image provided by an embodiment of the present application;

[0044] Figure 3 This is a schematic diagram of a top image provided in an embodiment of the present application;

[0045] Figure 4 1 is a schematic diagram of a binary image of a top image provided in an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of a binary image after a closing operation provided in an embodiment of the present application;

[0047] Figure 6 This is a schematic diagram of a top image provided in an embodiment of the present application. Figure 1 ;

[0048] Figure 7 This is a schematic diagram of a top image provided in an embodiment of the present application. Figure 2 ;

[0049] Figure 8 This is a flowchart of a window dragging method for a cloud application provided in an embodiment of the present application;

[0050] Figure 9 This is a structural diagram of a device for identifying a window drag area of ​​a cloud application provided by an embodiment of the present application;

[0051] Figure 10 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0053] The drag zone is the area within an application window that can be moved by dragging. It's typically located at the top of the window. The drag zone for client-side windows is identified by the cloud and sent to the client. In typical cloud application solutions, the entire title bar area at the top of the window is set as the drag zone. However, because the title bar at the top of the window typically contains functional components such as menu buttons and tabs, dragging the client-side window can affect the native functionality of the title bar. For example, if a user clicks on a menu button during a drag operation, the window may respond to both menu pop-up events and drag events, affecting the user experience. For another example, if a button at the top of a window is set to native drag behavior—for example, a browser tab can be dragged outside the window to form a new independent window—setting the drag zone for the moving window will disable the tab's dragging behavior.

[0054] In order to solve the above problems, an embodiment of the present application provides a method for identifying the drag area of ​​a cloud application window. The cloud side uses an image processing method to exclude areas in the top area of ​​the window that may have other functions when determining the drag area, thereby ensuring that the dragging behavior of the terminal side window does not affect the native function of the window menu and optimizes the user experience.

[0055] The method for identifying the drag area of ​​a cloud application window provided in an embodiment of the present application can be triggered and executed when the window shape changes. For example, when the user moves, scales, maximizes or minimizes the end-side window, the cloud side identifies the drag area of ​​the end-side window and sends the identified drag area to the client, so that the user can perform drag operations based on the drag area to move the window.

[0056] Figure 1 This is a flow chart of a method for identifying a window drag area of ​​a cloud application provided by an embodiment of the present application. The method is applied to the cloud side of the cloud application. Figure 1 As shown, the method includes:

[0057] S101: Acquire the top image of the application window.

[0058] The top of the application window is typically where the title bar is located and where the drag zone is typically set. In this step, the top image of the application window refers to an image of a certain area at the top of the application window. The extent of this area can be determined based on actual circumstances. In one implementation, the application window image can be first obtained, and then a screenshot of an area at a preset height from the top of the application window image can be taken to obtain the top image. The application window image is the complete image of the client-side window, and the preset height can be set based on empirical values.

[0059] In another implementation, after acquiring the application window image, image recognition is performed on the application window image to obtain a top image corresponding to the application window title bar. For some cloud applications, the title bar in their application window is clearly distinguishable from the rest of the application window, such as a different color or a clear dividing line between the title bar and the rest of the application window. In such cases, image recognition methods can be used to accurately identify the title bar in the application window and determine the image of the title bar as the top image of the application window.

[0060] In addition, the above two implementation methods can be used in combination. For example, the image recognition method is first used to identify the title bar in the application window image. If it cannot be identified, a method of taking a screenshot of the area at a preset height at the top of the application window image is used to obtain the top image.

[0061] S102 : Perform contour detection on the top image to determine the largest outer contour in the top image and secondary contours in the largest outer contour.

[0062] The top image may contain functional components such as menu buttons and tabs, and may also contain information such as text and icons. It may also include some blank areas. The blank areas here refer to areas without functional components, text or icons, and do not specifically refer to white areas. It can be understood that whether it is a functional component, text or icon, its contour will be detected during contour detection. By performing contour detection on the top image, all the contour information in the top image is obtained, and then the largest outer contour is determined. Among them, if there are no other contours surrounding a contour, then the contour is the outer contour, and the one with the largest area among all the outer contours is the aforementioned largest outer contour. The largest outer contour will contain more blank areas. In addition, the outer contour also includes smaller contours, namely secondary contours. The secondary contour may be the contours of some functional components, text or icons in the area contained in the outer contour. There may be multiple secondary contours.

[0063] S103: Determine a closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour, and determine a drag area of ​​the application window based on at least one closed area.

[0064] The closed area formed by the adjacent edges of adjacent secondary contours and the largest outer contour can be a closed area formed by extending the adjacent edges of adjacent secondary contours toward the largest outer contour. The closed area is the area between adjacent secondary contours. The closed area is a blank area and does not include components, text, or icons, etc. Therefore, these closed areas are determined as drag areas. When performing drag operations, the native functions of the application window will not be affected, which provides a better user experience.

[0065] The method of the embodiment of the present application performs contour detection on the top image of the application window based on an image processing method, and thereby excludes the top area that may have other functions based on the detected contour, and retains the blank area therein as the drag area, thereby ensuring that the dragging behavior of the end-side window does not affect the native function of the application window, thereby optimizing the user experience.

[0066] Based on the above embodiments, how to perform contour detection and how to determine the drag area are described.

[0067] After obtaining the top image, the top image is converted into a grayscale image, the background color pixel value of the grayscale image is determined, and the grayscale image is binarized based on the background color pixel value to obtain a binary image; and contour detection is performed on the binary image.

[0068] The background color pixel value can be determined by determining the most numerous pixel value in the grayscale image and determining the most numerous pixel value as the background color pixel value. Binarization processing can include setting pixels corresponding to the background color pixel value to white and pixels corresponding to other pixel values ​​other than the background color pixel value to black, thereby obtaining a binary image. It can be seen that after binarization of the grayscale image, the functional components, text, icons, etc., in the resulting binary image appear black, while the background appears white. Binarization converts the top image to black and white, facilitating contour detection.

[0069] Optionally, a closing operation is performed on the binary image, and then contour detection is performed on the binary image after the closing operation.

[0070] The closing operation involves dilating and then eroding a binary image. This operation eliminates small holes, smoothes object contours, connects narrow discontinuities and gullies, and fills broken contour lines. By performing a closing operation on a binary image, the outlines of black blocks, such as functional components, text, or images, are smoothed. Small discontinuities between these black blocks are connected, small black blocks are connected, or small holes between black blocks are filled, making the local black blocks more cohesive and facilitating subsequent contour detection.

[0071] In the results of contour detection, the largest outer contour includes a large amount of white area and some secondary contours. These secondary contours may be the contours of functional components, text, or images. For two adjacent secondary contours, their adjacent edges are extended toward the largest outer contour, and a closed area can be formed between them and the largest outer contour. This closed area is a blank area, that is, the closed area does not contain any functional components, text, icons, etc. Using it as a drag area will not affect the native functions of the application window.

[0072] Optionally, since the adjacent edges of adjacent secondary contours and the largest outer contour may be curves or straight lines, the closed area formed by the adjacent edges of adjacent secondary contours and the largest outer contour may be an irregularly shaped area. In order to make the drag area more standardized, for every two adjacent secondary contours, the rectangular area between the adjacent edges of the adjacent secondary contours and the largest outer contour can be determined as the aforementioned closed area that can be used as the drag area, that is, a regular rectangular area is determined in the irregularly shaped area formed by the adjacent edges of the adjacent secondary contours and the largest outer contour as a possible drag area.

[0073] Among the secondary contours in the largest outer contour, there may be some adjacent secondary contours whose distances are very close to each other. The area formed by the adjacent edges of these very close adjacent secondary contours and the largest outer contour is very narrow. If it is used as the drag area, it is not convenient to operate. Therefore, the smaller closed areas in the closed areas formed by the adjacent edges of the adjacent secondary contours and the largest outer contour can be eliminated, and only the larger areas are retained as the drag area, that is, the closed area in at least one closed area whose size exceeds the preset value is determined as the drag area of ​​the application window.

[0074] The method of the embodiment of the present application is based on an image processing method, which performs grayscale conversion, binarization and closing operation on the top image of the application window, so as to accurately detect the contour in the top image. Based on the detected contour, the top area that may have other functions, such as functional components, text or icons, is excluded, and the blank area of ​​the top image is retained as the drag area, thereby ensuring that the dragging behavior of the terminal side window does not affect the native function of the application window, thereby optimizing the user experience.

[0075] The following is an example description with reference to the accompanying drawings.

[0076] 1. Get the application window image, such as Figure 2 shown.

[0077] The application window image is the device-side window image generated by the client connecting to the cloud application. When the application window changes its form, such as when the user moves, resizes, maximizes, or minimizes the device-side window, the cloud obtains the application window image to identify the drag area.

[0078] 2. Take a screenshot of the area at the preset height at the top of the application window image to get the top image, such as Figure 3 shown.

[0079] Figure 3 The top image shown in the figure is obtained by taking a screenshot of the area at a preset height at the top of the application window image. It can be understood that the image recognition method can also be used in this step to obtain the top image of the application window image. Figure 2 The application window image shown is identified to obtain a top image corresponding to the title bar of the application window.

[0080] Furthermore, it is understandable that the top image obtained by the screenshot or image recognition method still has the original color of the application window image. Figure 3 A grayscale image is used for illustration, but no grayscale processing is actually performed in this step.

[0081] 3. Convert the top image to grayscale image, Figure 3 Grayscale images have been used for illustration, so the grayscale images are still used here. Figure 3 To give a hint.

[0082] In the embodiment of the present application, the method for grayscale image conversion is not described in detail, for example, the top image is converted into a 256-level grayscale image.

[0083] 4. Determine the background color pixel value of the grayscale image, e.g. Figure 3 The blank area between the menu button and the text is the background color of the top image, and its pixel value is the largest. It is determined as the background color pixel value. Based on the background color pixel value, the grayscale image is binarized. The pixels corresponding to the background color pixel value are set to white, and the pixels corresponding to other pixel values ​​are set to black to obtain a binary image, as shown in the figure. Figure 4 As shown, after this step, the functional components, text or icon areas in the top image are set to black, and the blank area, that is, the background area, is set to white to facilitate accurate contour detection.

[0084] 5. Perform a closing operation on the binary image to obtain Figure 5 The image shown.

[0085] By performing a closing operation on the binary image, the outlines of the black blocks of functional components, text or images can be made smoother, and the small discontinuities between these black blocks are connected, the adjacent black blocks are connected, and the small holes between the black blocks are filled, for example Figure 5 The numbers and letters in the figure are connected, and the color blocks indicated by the circled parts are connected, so that the local black blocks become more integrated, so that the contour obtained by contour detection can more completely include the functional components, text or image areas, and the contour is clearer.

[0086] 6. Perform contour detection on the binary image after the closing operation to obtain the largest outer contour and the secondary contours in the largest outer contour.

[0087] Figure 6 The result of contour detection is shown in FIG. 1 . It should be noted that, in order to clearly illustrate the relationship between the contour line and each part in the top image, Figure 6 The figure shows the contours detected in the top image based on the top image, but not the binary image after the closing operation. In fact, this step is to perform contour detection on the binary image after the closing operation. Figure 6 This is only used to illustrate the contour range. Contour lines other than rectangle 600 are the results of contour detection. Contour line 601 is the largest outer contour line, and the contour lines within contour line 601 are secondary contour lines. It can be seen that secondary contour lines are the outlines of areas where text, icons, and menu buttons are located.

[0088] The result of contour detection on the binary image after closing operation is as follows Figure 7 As shown in Figure 7 The figure does not show all the detected contours, but only shows the Figure 6 The largest outer contour and the secondary contour in the largest outer contour, wherein the contour line 701 is Figure 6 The largest outer contour line 601 in the image, and the secondary contour inside the contour line 701 are Figure 6 The secondary contour inside the middle contour line 601, Figure 7 In the figure, the secondary contour lines are simplified as rectangular frames.

[0089] 7. A rectangular area between adjacent sides of adjacent secondary contours and the largest outer contour is determined as a drag area, and the size of the rectangular area is greater than a preset value.

[0090] It can be seen that Figure 7The distance between the two secondary contours on the left is closer, and the distance between the three secondary contours on the right is also closer, while the distance between the second and third secondary contours on the left is farther. Finally, the rectangular area between the adjacent edges of the second and third secondary contours on the left and the largest outer contour is determined as the drag area. This rectangular area corresponds to Figure 6 Rectangle 600 is shown in FIG.

[0091] Using the above method, functional components, text, icons and other areas are excluded from the drag area, and the identified drag area is the blank area at the top of the window, so that the terminal side window can be moved without affecting the use of the native functions of the application window.

[0092] Figure 8 This is a flow chart of a window dragging method for a cloud application provided by an embodiment of the present application. Figure 8 As shown, the method includes:

[0093] S801. The cloud side obtains the top image of the application window in response to the change in the form of the application window.

[0094] S802: The cloud side performs contour detection on the top image to determine the largest outer contour in the top image and the secondary contours in the largest outer contour.

[0095] S803: The cloud side determines a closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour, determines a drag area of ​​the application window based on at least one closed area, and sends the drag area to the client.

[0096] S804: In response to the drag operation on the drag area, the client reports the window position after the drag operation to the cloud side.

[0097] In this embodiment, when the shape of the application window changes on the cloud side, for example, when the user moves, scales, maximizes or minimizes the terminal side window, the method of the aforementioned embodiment is used to identify the window drag area, and the identified drag area is sent to the client. When the user drags the drag area to move the terminal side window, the client reports the window position after the drag operation to the cloud side, thereby completing the window movement.

[0098] Figure 9 This is a schematic diagram of the structure of a device for identifying a window drag area of ​​a cloud application provided by an embodiment of the present application. Figure 9 As shown, the device includes:

[0099] An acquisition module 901 is used to acquire a top image of an application window;

[0100] A detection module 902 is configured to perform contour detection on the top image to determine the largest outer contour in the top image and secondary contours in the largest outer contour;

[0101] The determination module 903 is configured to determine a closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour, and determine the drag area of ​​the application window based on at least one of the closed areas.

[0102] In one implementation, the detection module 902 is configured to:

[0103] converting the top image into a grayscale image;

[0104] Determining a background color pixel value of the grayscale image, and performing binarization processing on the grayscale image based on the background color pixel value to obtain a binary image;

[0105] Perform contour detection on the binary image.

[0106] In one implementation, the detection module 902 is configured to:

[0107] Determine the pixel value with the largest number among the pixel values ​​of the grayscale image, and determine the pixel value with the largest number as the background color pixel value;

[0108] The pixels corresponding to the background color pixel values ​​are set to white, and the pixels corresponding to other pixel values ​​other than the background color pixel values ​​are set to black, to obtain the binary image.

[0109] In one implementation, the detection module 902 is configured to:

[0110] performing a closing operation on the binary image;

[0111] Perform contour detection on the binary image after closing operation.

[0112] In one implementation, the determining module 903 is configured to:

[0113] A rectangular area between adjacent sides of adjacent secondary contours and the largest outer contour is determined as the closed area.

[0114] In one implementation, the determining module 903 is configured to:

[0115] The closed area in at least one of the closed areas, the closed area whose size exceeds a preset value, is determined as the drag area of ​​the application window.

[0116] In one implementation, the acquisition module 901 is configured to:

[0117] Get the application window image;

[0118] A screenshot is taken of an area of ​​a preset height at the top of the application window image to obtain the top image.

[0119] In one implementation, the acquisition module 901 is configured to:

[0120] Get the application window image;

[0121] Perform image recognition on the application window to obtain a top image corresponding to the title bar of the application window.

[0122] The device for identifying the window drag area of ​​a cloud application provided in an embodiment of the present application can be used to implement the method for identifying the window drag area of ​​a cloud application in the aforementioned embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0123] Figure 10 This is a schematic block diagram of the cloud device provided in the embodiment of the present application. Figure 10 As shown, the electronic device 1000 may include at least one processor 1001 for implementing the method for identifying a window drag area of ​​a cloud application provided in an embodiment of the present application. The cloud device 1000 is the cloud side of the cloud application in the aforementioned embodiment.

[0124] Optionally, the electronic device 1000 further includes at least one memory 1002 for storing program instructions and / or data. The memory 1002 is coupled to the processor 1001. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 1001 may operate in conjunction with the memory 1002. The processor 1001 may execute program instructions stored in the memory 1002. At least one of the at least one memory may be included in the processor.

[0125] Optionally, the electronic device 1000 also includes a communication interface 1003 for communicating with other devices through a transmission medium, so that the electronic device 1000 can communicate with other devices. The communication interface 1003 can be, for example, a transceiver, an interface, a bus, a circuit, or a device capable of implementing a transceiver function. The processor 1001 can use the communication interface 1003 to send and receive data and / or information, and is used to implement: obtaining the top image of the application window; performing contour detection on the top image to determine the largest outer contour in the top image and the secondary contour in the largest outer contour; determining the closed area formed by the adjacent edges of adjacent secondary contours and the largest outer contour, and determining the drag area of ​​the application window based on at least one closed area. Please refer to the detailed description in the previous embodiment for details, which will not be repeated here.

[0126] The specific connection medium between the processor 1001, the memory 1002 and the communication interface 1003 is not limited in the embodiment of the present application. Figure 10 The processor 1001, the memory 1002 and the communication interface 1003 are connected via a bus 1004. The bus 1004 is connected to the processor 1001, the memory 1002 and the communication interface 1003. Figure 10 The connections between the other components are shown in bold lines, which are only for illustration and not intended to be limiting. The bus can be divided into address bus, data bus, control bus, etc. Figure 10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0127] It should be understood that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-mentioned method embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-mentioned method.

[0128] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0129] The present application also provides a cloud application system, including the cloud device and client as described in the aforementioned embodiment.

[0130] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method in any of the aforementioned embodiments.

[0131] The terms "unit," "module," and the like used in this specification may be used to refer to a computer-related entity, hardware, firmware, a combination of hardware and software, software, or software in execution.

[0132] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. In the several embodiments provided in this application, it should be understood that the disclosed devices, equipment, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] In the above-described embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (program) are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0136] If this function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0137] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0138] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for identifying a window drag area of ​​a cloud application, characterized in that: Applied to the cloud side of a cloud application, the method includes: Get the top image of the application window; Performing contour detection on the top image to determine the largest outer contour in the top image and the secondary contour in the largest outer contour; performing contour detection on the top image, including: after obtaining the top image, converting the top image into a grayscale image, determining the background color pixel value of the grayscale image, and binarizing the grayscale image based on the background color pixel value to obtain a binary image; performing contour detection on the binary image; wherein, determining the background color pixel value of the grayscale image and binarizing the grayscale image based on the background color pixel value to obtain a binary image; performing contour detection on the binary image, including: determining the pixel value with the largest number among the pixel values ​​of the grayscale image, determining the pixel value with the largest number as the background color pixel value; setting the pixels corresponding to the background color pixel value to white, and setting the pixels corresponding to other pixel values ​​other than the background color pixel value to black, to obtain the binary image; Determining a closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour, determining a drag area of ​​the application window based on at least one of the closed areas, and sending the drag area to the client; When the user drags the drag area to move the application window, a report of the window position after the drag operation is received from the client.

2. The method according to claim 1, characterized in that The performing contour detection on the binary image includes: performing a closing operation on the binary image; Perform contour detection on the binary image after closing operation.

3. The method according to any one of claims 1-2, characterized in that The determining of the closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour includes: A rectangular area between adjacent sides of adjacent secondary contours and the largest outer contour is determined as the closed area.

4. The method according to any one of claims 1 to 2, characterized in that The determining the drag area of ​​the application window according to at least one of the closed areas includes: The closed area in at least one of the closed areas, the closed area whose size exceeds a preset value, is determined as the drag area of ​​the application window.

5. The method according to any one of claims 1-2, characterized in that The obtaining of the top image of the application window includes: Get the application window image; A screenshot is taken of an area of ​​a preset height at the top of the application window image to obtain the top image.

6. The method according to any one of claims 1-2, characterized in that The obtaining of the top image of the application window includes: Get the application window image; Perform image recognition on the application window to obtain a top image corresponding to the title bar of the application window.

7. A cloud application window dragging method, characterized in that: Applied to the cloud side and client of a cloud application, the method includes: The cloud side obtains the top image of the application window in response to the change in the form of the application window; The cloud side performs contour detection on the top image to determine the largest outer contour in the top image and the secondary contour in the largest outer contour; the cloud side performs contour detection on the top image, including: after obtaining the top image, converting the top image into a grayscale image, determining the background color pixel value of the grayscale image, and binarizing the grayscale image based on the background color pixel value to obtain a binary image; performing contour detection on the binary image; wherein, determining the background color pixel value of the grayscale image and binarizing the grayscale image based on the background color pixel value to obtain a binary image; performing contour detection on the binary image, including: determining the pixel value with the largest number among the pixel values ​​of the grayscale image, determining the pixel value with the largest number as the background color pixel value; setting the pixels corresponding to the background color pixel value to white, and setting the pixels corresponding to other pixel values ​​other than the background color pixel value to black, to obtain the binary image; The cloud side determines a closed area formed by adjacent edges of adjacent secondary contours and the largest outer contour, determines a drag area of ​​the application window based on at least one of the closed areas, and sends the drag area to the client; In response to the drag operation on the drag area, the client reports the window position after the drag operation to the cloud side.

8. A cloud device, characterized in that: include: memory and processor; The memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory, wherein when the computer program is executed, the processor executes the method according to any one of claims 1 to 6.

9. A cloud application system, characterized in that: Comprises the cloud device and client as described in claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images