Mouse pointer image display method, device, equipment, medium and program product

Abstract

The invention relates to a mouse pointer image display method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining a target mouse pointer object through a first application program running in a native operation system of electronic equipment, the target mouse pointer object being generated by the first application program based on obtained mouse pointer data, and the target mouse pointer object being generated by the first application program based on the obtained mouse pointer data; the mouse pointer data is image data corresponding to a mouse pointer image in a second application program, and the second application program is an application program installed in a virtualized operating system operated by the electronic equipment; obtaining a current display position of the target mouse pointer object through the first application program; and rendering the target mouse pointer object based on the current display position through the first application program so as to display a corresponding mouse pointer image. According to the method, the mouse pointer image function of the second application program in the virtualized operating system is displayed in the first application program, and meanwhile, the situation that the mouse pointer image is not displayed can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of software technology, and in particular to a method, apparatus, device, medium, and program product for displaying a mouse pointer image. Background Art

[0002] In the Android system, container virtualization technology can be used to run the Linux system and corresponding applications, and the graphical interface data of each window running in Linux can be passed to the Android application for rendering and display through the Wayland+FreeRDP protocol, thereby realizing the function of using applications in the Linux system in the Android system.

[0003] Among them, the graphical interface data of each window running in Linux includes the mouse window graphical interface data. In the related technology, when the mouse window graphical interface data is passed to the Android application for rendering and display, there are problems such as mouse display freeze and image retention, which reduces the user experience in the above usage scenarios. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a mouse pointer image display method, device, equipment, medium and program product.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for displaying a mouse pointer image is provided, the method comprising:

[0006] Obtaining, by a first application running in a native operating system of the electronic device, a target mouse pointer object, where the target mouse pointer object is generated by the first application based on acquired mouse pointer data, where the mouse pointer data is image data corresponding to a mouse pointer image in a second application, where the second application is an application installed in a virtualized operating system running on the electronic device;

[0007] Obtaining, through the first application, a current display position of the target mouse pointer object;

[0008] The target mouse pointer object is rendered based on the current display position through the first application to display a corresponding mouse pointer image.

[0009] Optionally, the mouse pointer data includes mouse pointer style data and mouse pointer image parameter data, and the method further includes:

[0010] generating, by the first application, a target bitmap corresponding to a preset bitmap format based on a preset bitmap generation function, the mouse pointer style data, and a preset bitmap format, wherein the preset bitmap format is a bitmap format in the native operating system;

[0011] The target mouse pointer object is obtained through the first application based on the target bitmap and the mouse pointer image parameter data.

[0012] Optionally, the mouse pointer image parameter data includes an image size parameter and first hotspot coordinates, and obtaining the target mouse pointer object based on the target bitmap and the mouse pointer image parameter data by the first application includes:

[0013] Obtaining, through the first application, an offset value of second hotspot coordinates compared to the first hotspot coordinates, where the second hotspot coordinates are hotspot coordinates of a mouse pointer object in the native operating system;

[0014] Expanding the image area corresponding to the target bitmap based on the offset value and the image size parameter by the first application to obtain an expanded bitmap image area;

[0015] By means of the first application, based on the offset value, in the expanded bitmap image area, pixels corresponding to the mouse pointer image in the target bitmap are moved to obtain an intermediate bitmap;

[0016] The target mouse pointer object is obtained through the first application based on the intermediate bitmap and the second hotspot coordinates.

[0017] Optionally, obtaining the target mouse pointer object through a first application running in a native operating system of the electronic device includes:

[0018] In response to the mouse moving to a target view in the first application, obtaining the target mouse pointer object through the first application;

[0019] The method further comprises:

[0020] The target mouse pointer object is set as the mouse pointer object of the target view through the first application.

[0021] Optionally, setting the target mouse pointer object as the mouse pointer object of the target view by the first application includes:

[0022] By calling the set mouse pointer object method in the view component through the first application, the target mouse pointer object is set as the mouse pointer object of the target view; or

[0023] Through the first application, the determine mouse pointer image method in the view component is called to obtain a return value, and the return value is used as the mouse pointer object of the target view, wherein the return value of the determine mouse pointer image method is the target mouse pointer object.

[0024] Optionally, the method further includes:

[0025] The mouse pointer data is obtained by monitoring the window display service of the virtualized operating system through the virtualized operating system.

[0026] Optionally, the method further includes:

[0027] The mouse pointer data is sent to the first application program via the virtualized operating system based on a fast path channel in a free remote desktop service.

[0028] According to a second aspect of an embodiment of the present disclosure, a mouse pointer image display device is provided, the device comprising:

[0029] a first acquisition module configured to acquire, through a first application running in a native operating system of the electronic device, a target mouse pointer object, the target mouse pointer object being generated by the first application based on acquired mouse pointer data, the mouse pointer data being image data corresponding to a mouse pointer image in a second application, the second application being an application installed in a virtualized operating system running on the electronic device;

[0030] A second acquisition module is configured to acquire the current display position of the target mouse pointer object through the first application;

[0031] The rendering and display module is configured to render the target mouse pointer object based on the current display position through the first application program to display a corresponding mouse pointer image.

[0032] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the mouse pointer image display method provided in the first aspect of the present disclosure are implemented.

[0033] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the steps of the mouse pointer image display method mentioned in the first aspect of the present disclosure.

[0034] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the mouse pointer image display method mentioned in the first aspect of the present disclosure.

[0035] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0036] The electronic device obtains a target mouse pointer object and the current display position of the target mouse pointer object through a first application running in the native operating system of the electronic device. The electronic device can then render the target mouse pointer object based on the current display position through the first application. Since the target mouse pointer image is generated based on the image data corresponding to the mouse pointer image in the second application, the mouse pointer image style of the second application can be rendered and displayed in the first application. In addition, since the mouse pointer image display is achieved by rendering the created mouse pointer object, on the one hand, compared with the method of synthesizing the entire mouse window and displaying it in the related art, only synthesizing the mouse pointer image can reduce resource consumption during the mouse pointer image display process. On the other hand, since the mouse pointer object remains unchanged when the mouse pointer image style is not switched, it is not necessary to transmit mouse pointer data in real time and synthesize the target mouse pointer object during the mouse movement, which can further reduce resource consumption during the mouse pointer image display process, thereby improving the situation of mouse pointer image display jamming. It can be seen that the method of the embodiment of the present disclosure can improve the situation of mouse pointer image display jamming while realizing the function of displaying the mouse pointer image of the second application in the virtualized operating system in the first application.

[0037] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0039] Figure 1 The figure is a flowchart of a method for displaying a mouse pointer image according to an exemplary embodiment of the present disclosure.

[0040] Figure 2 The figure is a flowchart of a method for displaying a mouse pointer image according to an exemplary embodiment of the present disclosure.

[0041] Figure 3 The figure is a flowchart of another method for displaying a mouse pointer image according to an exemplary embodiment of the present disclosure.

[0042] Figure 4 The figure is a structural block diagram of a mouse pointer image display device according to an exemplary embodiment of the present disclosure.

[0043] Figure 5 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0045] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0046] like Figure 1 As shown, in the solution of transmitting Linux application images to Android applications based on the Wayland+FreeRDP protocol in the related technology, the mouse pointer image of the mouse pointer layer of the Linux operating system (RDP Server) is saved in the shared memory (ShareMemory) in the form of graphic data of a Linux application window through the xserver+weston architecture, and the window parameters (including number, width, height and other attributes) are passed to the Android operating system (RDP Client) through the dynamic channel (gfxredir channel), which is synthesized into a picture (Bitmap) of the entire window by the Android application (Android APP), and then rendered by the rendering service (SurfaceFlinger service), and finally presented to the user on the application side.

[0047] Wayland is a modern, secure, and flexible display server protocol. The FreeRDP protocol is an open-source remote desktop protocol implementation that allows users to remotely connect to another computer and control its desktop. Xserver is server software for graphical interface displays. It provides abstraction and management of graphics hardware, enabling graphical applications to run on different operating systems. It is a new generation of display servers that supports the Wayland display server protocol. RDP stands for Remote Desktop Protocol.

[0048] In the above solution, the mouse image displayed in the application of the Android operating system is implemented by applying the window graphical interface display solution. As a result, whenever the mouse moves, the graphics data and window parameters of the application window are transmitted through shared memory and the gfxredir dynamic channel, and the bitmap data is refreshed. As a result, when the mouse moves at a high frame rate, it will cause significant visual lag and image retention, seriously affecting the user experience.

[0049] Furthermore, the applicant discovered through long-term research that the only valid data in the entire mouse layer is the mouse cursor, which has the characteristics of small data volume and high refresh rate. In view of this, the embodiments of the present disclosure provide a mouse pointer image display method, apparatus, electronic device, medium, and program, which provide a method for generating a target mouse pointer object based on the image data corresponding to the mouse pointer image in the second application, and then rendering the target mouse pointer object to realize the function of displaying the mouse pointer image in the second application in the first application. In this way, the problem of mouse pointer image display lag can be improved.

[0050] It should be noted that, for ease of description, the embodiments of this disclosure use the example of transferring an application running in a Linux operating system to an Android operating system for display. In some implementations, this can also be applied to transferring an application running in an operating system other than the Linux operating system to an operating system other than the Android operating system for display. That is, in subsequent embodiments, the virtualized operating system can be a Linux operating system or another operating system, and the native operating system of the electronic device can be an Android operating system or another operating system. This disclosure does not limit this.

[0051] Figure 2 This is a flowchart of a method for displaying a mouse pointer image according to an exemplary embodiment. The method for displaying a mouse pointer image can be used in an electronic device, such as a mobile phone, a desktop computer, a laptop computer, a tablet computer, etc. Figure 2 As shown, the mouse pointer image display method includes the following steps:

[0052] S210, obtain the target mouse pointer object through the first application running in the native operating system of the electronic device, the target mouse pointer object is generated by the first application based on the obtained mouse pointer data, the mouse pointer data is the image data corresponding to the mouse pointer image in the second application, and the second application is an application installed in the virtualized operating system running on the electronic device.

[0053] In some exemplary scenarios, the functionality of an application product under a native operating system and an application product under a virtualized operating system may differ. For example, the functionality of an application product under a virtualized operating system is more comprehensive than that of an application product under a native operating system. In this case, in order to use the more comprehensive functionality under the native operating system, it is necessary to create a virtualized operating system under the native operating system to use the application product under the virtualized operating system. In this scenario, the display interface of a first application can be used to synchronously operate and display a second application installed in the virtualized operating system, thereby enabling the use of the functionality of the application product under the virtualized operating system in the native operating system. This includes the synchronous operation and display of the mouse pointer image.

[0054] In the embodiment of the present disclosure, the target pointer object is a PointerIcon object, and the corresponding mouse pointer image can be displayed by rendering the PointerIcon object.

[0055] In the embodiment of the present disclosure, the electronic device can run a native operating system. In the native operating system, an independent and isolated environment can be created through container virtualization technology, and the virtualized operating system can be run in the created environment. Subsequently, a second application to be used can be further installed in the virtualized operating system and the second application can be run.

[0056] In some implementations, in order to enable the first application to generate a target mouse pointer object based on the acquired mouse pointer data, the method of the embodiment of the present disclosure may further include the following steps:

[0057] The mouse pointer data is sent to the first application program via the virtualized operating system based on a fast path channel in the free remote desktop service.

[0058] In an embodiment of the present disclosure, after the second application is installed and run in the virtualized operating system, the virtualized operating system can obtain image data corresponding to the mouse pointer image in the running second application, that is, mouse pointer data, and then the virtualized operating system can further send the mouse pointer data to the first application based on the Fastpath channel in the Free Remote Desktop Service (FreeRDP protocol), so that the first application can further generate a target mouse pointer object based on the obtained mouse pointer data.

[0059] In some implementations, to obtain mouse pointer data, the method of the embodiment of the present disclosure may further include the following steps:

[0060] Through the virtualized operating system, the window display service of the virtualized operating system is monitored to obtain the mouse pointer data.

[0061] In the embodiment of the present disclosure, the virtualized operating system may monitor the window display service of the virtualized operating system to obtain the graphic data of the window, which may include the graphic data of the mouse window, that is, the mouse pointer data.

[0062] In some implementations, the virtualized operating system may start a weston process in the Wayland service to monitor the window display service of the virtualized operating system and obtain mouse pointer data.

[0063] It should be noted that, considering that the mouse pointer image used by the second application running in the virtualized operating system may not be modified, in the above embodiment, the process of generating the target mouse pointer object based on the acquired mouse pointer data by the first application can be executed once, that is, the first application can store the generated target mouse pointer object, and can subsequently directly use the stored target mouse pointer object during the mouse movement process.

[0064] S220: Obtain the current display position of the target mouse pointer object through the first application.

[0065] In the embodiment of the present disclosure, in order to accurately render the mouse pointer image in the first application, the first application can also obtain the current display position of the target mouse pointer object. There are many ways for the first application to obtain the current display position of the target mouse pointer object.

[0066] In some embodiments, the virtual operating system can collect the position of the mouse pointer image displayed in the installed second application, and then send the collected position information to the first application, so that the first application can obtain the current display position of the target mouse pointer object based on the position information sent by the virtual operating system.

[0067] In other embodiments, since the mouse position displayed in the display interface of the second application is the same as the mouse position in the first application, the first application can directly collect the position information of the mouse pointer image in the display interface of the first application and use it as the position information of the current display position of the target mouse pointer object.

[0068] S230 , rendering the target mouse pointer object based on the current display position through the first application program to display a corresponding mouse pointer image.

[0069] In the embodiment of the present disclosure, after obtaining the target mouse pointer object and the current display position, the first application may render the target mouse pointer object based on the current display position to display the corresponding mouse pointer image.

[0070] Using the above method, an electronic device obtains a target mouse pointer object and the current display position of the target mouse pointer object through a first application running in the native operating system of the electronic device. Then, the electronic device can render the target mouse pointer object based on the current display position through the first application. Since the target mouse pointer image is generated based on the image data corresponding to the mouse pointer image in the second application, the mouse pointer image style of the second application can be rendered and displayed in the first application. In addition, since the mouse pointer image display is achieved by rendering the created mouse pointer object, on the one hand, compared with the method of synthesizing the entire mouse window and displaying it in the related art, only synthesizing the mouse pointer image can reduce resource consumption during the mouse pointer image display process. On the other hand, since the mouse pointer object remains unchanged when the mouse pointer image style is not switched, it is not necessary to transmit mouse pointer data in real time and synthesize the target mouse pointer object during the mouse movement, which can further reduce resource consumption during the mouse pointer image display process, thereby improving the situation of mouse pointer image display lag. It can be seen that using the method of the embodiment of the present disclosure, while realizing the function of displaying the mouse pointer image of the second application in the virtualized operating system in the first application, the situation of mouse pointer image display lag can be improved.

[0071] In some implementations, the mouse pointer data includes mouse pointer style data and mouse pointer image parameter data. In this case, the mouse pointer image display method of the embodiment of the present disclosure may further include the following steps:

[0072] Generate, by the first application, a target bitmap corresponding to a preset bitmap format based on a preset bitmap generation function, mouse pointer style data, and a preset bitmap format, where the preset bitmap format is a bitmap format in a native operating system;

[0073] A target mouse pointer object is obtained through the first application based on the target bitmap and the mouse pointer image parameter data.

[0074] In the embodiment of the present disclosure, the mouse pointer style data may be understood as the original data of the mouse style image, for example, may include data such as xorMaskData, andMaskData.

[0075] In the embodiment of the present disclosure, the mouse pointer image parameter data may include image size parameters and hotspot coordinates of the mouse pointer image in the second application program, wherein the image size parameters may include width and height, and the hotspot coordinates of the mouse pointer image in the second application program are the first hotspot coordinates.

[0076] In the embodiment of the present disclosure, after the first application obtains the mouse pointer data, the mouse pointer style data contained therein may be converted into a bitmap according to a preset bitmap format.

[0077] In an embodiment of the present disclosure, a first application may use mouse pointer style data and a preset bitmap format as input to a preset bitmap generation function, and obtain a bitmap in the preset bitmap format output by the preset bitmap generation function, i.e., a target bitmap. The first application may then further create a target mouse pointer object based on the target bitmap and the mouse pointer image parameter data.

[0078] In addition, considering that for different mouse pointer images of the second application, the hotspot coordinates may be in different positions, for example, for the arrow mouse style, the hotspot coordinates are located at the tip of the upper left corner of the arrow, and for the cross cursor, the hotspot coordinates are located at the center of the cursor. At the same time, in different operating systems, the hotspot coordinates of the mouse pointer bitmap may be inconsistent. For example, in the native Android operating system of the electronic device, the hotspot coordinates of the mouse pointer bitmap can be uniformly set at the center point of the bitmap. At this time, it is possible that the hotspot coordinates of the mouse pointer object in the native operating system are different from the hotspot coordinates included in the mouse pointer image parameter data, that is, the second hotspot coordinates are different from the first hotspot coordinates. In this case, if the target mouse pointer object is created directly according to the target bitmap generated by the mouse pointer style data, and the target mouse pointer object is rendered based on the current display position, it may cause the timing of the mouse pointer image rendered and displayed in the first application to deviate, and the mouse pointer image will jump in visual effect, and the second application cannot be synchronously operated and displayed through the display interface of the first application.

[0079] For example, taking the bitmap size corresponding to the mouse pointer image as 64*64 pixel units as an example, assuming that the mouse pointer image parameter data includes the first hotspot coordinates, that is, the hotspot coordinates corresponding to the mouse pointer image in the second application are the coordinate points of the upper left corner of the bitmap, for example (0, 0), and assuming that the hotspot coordinates of the mouse pointer object in the native operating system are the coordinates of the center point of the bitmap, for example, the second hotspot coordinates are (32, 32). At this time, if the arrow-style mouse pointer image is rendered and displayed directly at the current display position, the click detection point of the mouse pointer image actually rendered and displayed is at the center position of the target bitmap, not at the tip of the upper left corner of the mouse. Therefore, according to the user's usage habits, clicking or selecting according to the tip of the arrow is different from the position the user actually wants to click or select, resulting in a visual jump of the mouse pointer image, and the second application cannot be displayed synchronously through the display interface of the first application.

[0080] Therefore, in order to improve the accuracy of the mouse pointer image display position, in some embodiments, the mouse pointer image parameter data includes an image size parameter and a first hotspot coordinate, wherein the hotspot coordinate included in the mouse pointer image parameter data can be understood as the first hotspot coordinate. In this case, obtaining the target mouse pointer object based on the target bitmap and the mouse pointer image parameter data by the first application may include the following steps:

[0081] Obtaining, through the first application, an offset value of second hotspot coordinates compared to the first hotspot coordinates, where the second hotspot coordinates are hotspot coordinates of a mouse pointer object in a native operating system;

[0082] Expanding the image area corresponding to the target bitmap based on the offset value and the image size parameter by the first application program to obtain an expanded bitmap image area;

[0083] By means of the first application, based on the offset value, in the expanded bitmap image area, pixels corresponding to the mouse pointer image in the target bitmap are moved to obtain an intermediate bitmap;

[0084] The target mouse pointer object is obtained through the first application based on the intermediate bitmap and the second hotspot coordinates.

[0085] In the embodiment of the present disclosure, in order to make the user experience of the mouse pointer image in the application program in the virtualized operating system consistent with the user experience in the first application program, the pixel points corresponding to the mouse pointer image in the target bitmap can be offset as a whole based on the offset value. Considering that the overall offset of the pixel points corresponding to the mouse pointer image may exceed the image area of ​​the target bitmap, the image area corresponding to the target bitmap can be expanded based on the offset value and the image size parameter to obtain an expanded bitmap image area. Then, based on the offset value, the pixel points corresponding to the mouse pointer image in the target bitmap can be moved in the expanded bitmap image area to obtain an intermediate bitmap. Subsequently, the target mouse pointer object can be obtained based on the intermediate bitmap and the second hotspot coordinates.

[0086] Continuing with the previous example, let's assume that the bitmap size corresponding to the mouse pointer image is 64*64 pixels. The offset value of the second hotspot coordinate (32, 32) compared to the first hotspot coordinate (0, 0) is 32*32. At this time, the pixel point corresponding to the mouse pointer image is moved 32 pixels to the right and 32 pixels downward respectively to obtain the intermediate bitmap.

[0087] For some cases where the mouse pointer image occupies more than 1 / 4 of the original image area, some pixels may exceed the area after moving 32*32. Therefore, to fully include the mouse pointer image in the expanded bitmap image area, the image area corresponding to the target bitmap can be expanded by n pixels to the right and below, respectively, according to the actual situation. The expanded bitmap image area is a (64+n)*(64+n) pixel area, and the coordinates of the expanded second hotspot are the new center position (32+n / 2,32+n / 2). Then, in the expanded bitmap image area, the pixels corresponding to the mouse pointer image are moved to the right and below by (32+n / 2) pixels, respectively, to obtain the intermediate bitmap.

[0088] In addition, considering that the mouse is expanded in the image area corresponding to the target bitmap, the mouse pointer image may be enlarged during subsequent rendering and display. Therefore, in order to achieve a better visual display effect, in some embodiments, after obtaining the intermediate bitmap, the intermediate bitmap can be scaled to obtain a scaled bitmap, and then, based on the scaled bitmap and the second hotspot coordinates, the target mouse pointer object is created.

[0089] In some embodiments, step S110, obtaining the target mouse pointer object through the first application running in the native operating system of the electronic device, may include the following steps:

[0090] In response to the mouse moving to a target view in the first application, a target mouse pointer object is obtained by the first application.

[0091] In this case, the mouse pointer image display method according to the embodiment of the present disclosure may further include the following steps:

[0092] By the first application, a target mouse pointer object is set to the mouse pointer object of the target view.

[0093] In an embodiment of the present disclosure, the target view can be any view in the first application. Each time the mouse is moved, the first application can set the target mouse pointer object to the mouse pointer object of the target view in response to the mouse moving to the target view in the first application, so as to display the mouse pointer image in the target view.

[0094] In some implementations, setting the target mouse pointer object as the mouse pointer object of the target view by the first application may include the following steps:

[0095] By calling the set mouse pointer object method in the view component through the first application, the target mouse pointer object is set as the mouse pointer object of the target view; or

[0096] The first application program is used to call a method for determining a mouse pointer image in a view component to obtain a return value, which is used as a mouse pointer object of a target view. The return value of the method for determining a mouse pointer image is a target mouse pointer object.

[0097] In the disclosed embodiment, the first application can call a method in the view component that sets a mouse pointer object, such as the setPointIcon method, to set the target mouse pointer object as the mouse pointer object of the target view. Alternatively, the first application can call a method in the view component that determines a mouse pointer image, such as the onResolvePointIcon method, to obtain a return value. Since the return value is the target mouse pointer object, the target mouse pointer object can also be set as the mouse pointer object of the target view.

[0098] That is, in the disclosed embodiment, each time the mouse is moved, the first application can call the SetMousePointerObject method in the target view based on the pre-generated target mouse pointer object to replace the system mouse pointer object with the target mouse pointer object. It should be noted that this call must be synchronous, not asynchronous. Alternatively, the first application can override the DetermineMousePointerImage method in the target view so that the return value of this method is the target mouse pointer object.

[0099] The following combination Figure 3 The flowchart of the method for displaying a mouse pointer image is shown in FIG. 1 , which uses an example to illustrate the execution process of the method for displaying a mouse pointer image according to an embodiment of the present disclosure. Figure 3 As shown:

[0100] The user can start the first application installed in the electronic device, which includes a mouse pointer synthesis and display module. When the mouse pointer synthesis and display module is initialized, it will create an rdpPointer structure data to receive the mouse pointer data structure passed from the Fastpath channel. The data structure includes mouse pointer style data, image width and height dimensions, and the first hotspot coordinates of the pointer.

[0101] In addition, after the first application is started, the first application can create an independent, isolated container through container virtualization technology, and then run the virtualized Linux operating system in the created container, and run the second application in the virtualized Linux operating system.

[0102] The virtualized Linux operating system can start the xserver process, the weston process and the FreeRDP protocol to monitor the Linux window display service and send image data to the first application in the Android operating system, and the image data includes the mouse pointer data of the mouse pointer layer of the second application.

[0103] The virtualized Linux operating system configures and opens a Fastpath channel in the FreeRDP protocol to pass the image data information (struct_POINTER_LARGE_UPDATE*pointer) encapsulated as a structure to the client, that is, to the first application. Specifically, the update_send_pointer_large function of RDP RAIL is called to send the mouse pointer data to the pre-created rdpPointer structure data through the Fastpath channel.

[0104] After receiving the mouse pointer style data, the first application will synthesize the mouse pointer style data into a target bitmap according to the preset bitmap format requirements, and then perform a hotspot coordinate calibration process. That is, the first application can obtain the offset value of the second hotspot coordinates compared to the first hotspot coordinates, and based on the offset value and the image size parameter, expand the image area corresponding to the target bitmap to obtain an expanded bitmap image area, and then further based on the offset value, in the expanded bitmap image area, move the pixel points corresponding to the mouse pointer image in the target bitmap to obtain an intermediate bitmap, and then obtain the target mouse pointer object based on the intermediate bitmap and the second hotspot coordinates, and store the target mouse pointer object.

[0105] Subsequently, each time the mouse is moved, the first application calls the setPointIcon method in the target view (VIEW) to replace the system's PointerIcon with the target mouse pointer object. Alternatively, the first application can override the return value of the onResolvePointIcon method to set the target mouse pointer object. In this way, each time the mouse is moved, the first application can call onResolvePointIcon in the target view to obtain the target mouse pointer object.

[0106] Next, the first application may display the mouse pointer image through the window manager and the mouse pointer service in sequence.

[0107] Figure 4 is a structural block diagram of a mouse pointer image display device 400 according to an exemplary embodiment. Figure 4 , the device comprises:

[0108] A first acquisition module 410 is configured to acquire a target mouse pointer object through a first application running in a native operating system of the electronic device, where the target mouse pointer object is generated by the first application based on acquired mouse pointer data, where the mouse pointer data is image data corresponding to a mouse pointer image in a second application, where the second application is an application installed in a virtualized operating system running on the electronic device;

[0109] A second acquisition module 420 is configured to acquire the current display position of the target mouse pointer object through the first application;

[0110] The rendering and display module 430 is configured to render the target mouse pointer object based on the current display position through the first application program to display a corresponding mouse pointer image.

[0111] In some embodiments, the mouse pointer data includes mouse pointer style data and mouse pointer image parameter data, and the mouse pointer image display device 400 further includes:

[0112] a target bitmap generation module configured to generate, through the first application, a target bitmap corresponding to a preset bitmap format based on a preset bitmap generation function, the mouse pointer style data, and a preset bitmap format, wherein the preset bitmap format is a bitmap format in the native operating system;

[0113] The target mouse pointer object generating module is configured to obtain the target mouse pointer object based on the target bitmap and the mouse pointer image parameter data through the first application program.

[0114] In some embodiments, the mouse pointer image parameter data includes image size parameters and first hotspot coordinates, and the target mouse pointer object generation module includes:

[0115] an offset value acquisition submodule configured to acquire, through the first application, an offset value of second hotspot coordinates compared to the first hotspot coordinates, where the second hotspot coordinates are hotspot coordinates of a mouse pointer object in the native operating system;

[0116] a region expansion submodule configured to expand the image region corresponding to the target bitmap based on the offset value and the image size parameter through the first application program to obtain an expanded bitmap image region;

[0117] a moving submodule configured to move, through the first application program, the pixel points corresponding to the mouse pointer image in the target bitmap in the expanded bitmap image area based on the offset value to obtain an intermediate bitmap;

[0118] The target mouse pointer object generation submodule is configured to obtain the target mouse pointer object based on the intermediate bitmap and the second hotspot coordinates through the first application.

[0119] In some embodiments, the first acquisition module 410 is further configured to acquire the target mouse pointer object through the first application in response to the mouse moving to the target view in the first application. In this case, the mouse pointer image display device 400 further includes:

[0120] A setting module is configured to set the target mouse pointer object as the mouse pointer object of the target view through the first application.

[0121] In some embodiments, the setting module includes:

[0122] A first calling submodule is configured to call a method of setting a mouse pointer object in a view component through the first application program to set the target mouse pointer object as the mouse pointer object of the target view;

[0123] The second calling submodule is configured to call the determine mouse pointer image method in the view component through the first application to obtain a return value, and the return value is used as the mouse pointer object of the target view, wherein the return value of the determine mouse pointer image method is the target mouse pointer object.

[0124] In some embodiments, the mouse pointer image display device 400 further includes:

[0125] The monitoring module is configured to monitor the window display service of the virtualized operating system through the virtualized operating system to obtain the mouse pointer data.

[0126] In some embodiments, the mouse pointer image display device 400 further includes:

[0127] The sending module is configured to send the mouse pointer data to the first application program through the virtualized operating system based on a fast path channel in the free remote desktop service.

[0128] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0129] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the steps of the mouse pointer image display method provided by the present disclosure are implemented.

[0130] Figure 5 FIG1 is a block diagram of an electronic device 500 according to an exemplary embodiment. For example, the electronic device 500 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, etc.

[0131] Reference Figure 5 , the electronic device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output interface 512 , a sensor component 514 , and a communication component 516 .

[0132] The processing component 502 generally controls the overall operation of the electronic device 500, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the above-described method for displaying a mouse pointer image. In addition, the processing component 502 may include one or more modules to facilitate interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.

[0133] The memory 504 is configured to store various types of data to support operations on the electronic device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0134] The power supply assembly 506 provides power to the various components of the electronic device 500. The power supply assembly 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 500.

[0135] The multimedia component 508 includes a screen that provides an output interface between the electronic device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the electronic device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0136] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 500 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 504 or transmitted via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0137] The input / output interface 512 provides an interface between the processing component 502 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0138] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 can detect the open / closed state of the electronic device 500, the relative positioning of components, such as the display and keypad of the electronic device 500. The sensor assembly 514 can also detect changes in the position of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, the orientation or acceleration / deceleration of the electronic device 500, and temperature changes of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0139] The communication component 516 is configured to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0140] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned mouse pointer image display method.

[0141] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory 504 including instructions. The instructions can be executed by the processor 520 of the electronic device 500 to perform the above-mentioned mouse pointer image display method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0142] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be a single IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned mouse pointer image display method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction can be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned mouse pointer image display method is implemented; alternatively, the integrated circuit or chip can receive the executable instruction through the interface and transmit it to the processor for execution to implement the above-mentioned mouse pointer image display method.

[0143] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned mouse pointer image display method when executed by the programmable device.

[0144] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0145] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for displaying a mouse pointer image, characterized in that: The method comprises: Obtaining, by a first application running in a native operating system of the electronic device, a target mouse pointer object, where the target mouse pointer object is generated by the first application based on acquired mouse pointer data, where the mouse pointer data is image data corresponding to a mouse pointer image in a second application, where the second application is an application installed in a virtualized operating system running on the electronic device; Obtaining, through the first application, a current display position of the target mouse pointer object; The target mouse pointer object is rendered based on the current display position through the first application to display a corresponding mouse pointer image.

2. The method according to claim 1, characterized in that The mouse pointer data includes mouse pointer style data and mouse pointer image parameter data, and the method further includes: generating, by the first application, a target bitmap corresponding to a preset bitmap format based on a preset bitmap generation function, the mouse pointer style data, and a preset bitmap format, wherein the preset bitmap format is a bitmap format in the native operating system; The target mouse pointer object is obtained through the first application based on the target bitmap and the mouse pointer image parameter data.

3. The method according to claim 2, characterized in that The mouse pointer image parameter data includes an image size parameter and a first hotspot coordinate. The obtaining, by the first application program, of the target mouse pointer object based on the target bitmap and the mouse pointer image parameter data includes: Obtaining, through the first application, an offset value of second hotspot coordinates compared to the first hotspot coordinates, where the second hotspot coordinates are hotspot coordinates of a mouse pointer object in the native operating system; Expanding the image area corresponding to the target bitmap based on the offset value and the image size parameter by the first application to obtain an expanded bitmap image area; By means of the first application, based on the offset value, in the expanded bitmap image area, pixels corresponding to the mouse pointer image in the target bitmap are moved to obtain an intermediate bitmap; The target mouse pointer object is obtained through the first application based on the intermediate bitmap and the second hotspot coordinates.

4. The method according to claim 1, wherein The step of obtaining the target mouse pointer object by using the first application running in the native operating system of the electronic device includes: In response to the mouse moving to a target view in the first application, obtaining the target mouse pointer object through the first application; The method further comprises: The target mouse pointer object is set as the mouse pointer object of the target view through the first application.

5. The method according to claim 4, characterized in that The step of setting the target mouse pointer object as the mouse pointer object of the target view by the first application program includes: By calling the set mouse pointer object method in the view component through the first application, the target mouse pointer object is set as the mouse pointer object of the target view; or Through the first application, the determine mouse pointer image method in the view component is called to obtain a return value, and the return value is used as the mouse pointer object of the target view, wherein the return value of the determine mouse pointer image method is the target mouse pointer object.

6. The method according to claim 1, characterized in that The method further comprises: The mouse pointer data is obtained by monitoring the window display service of the virtualized operating system through the virtualized operating system.

7. The method according to claim 1, characterized in that The method further comprises: The mouse pointer data is sent to the first application program via the virtualized operating system based on a fast path channel in a free remote desktop service.

8. A mouse pointer image display device, characterized in that: The device comprises: a first acquisition module configured to acquire, through a first application running in a native operating system of the electronic device, a target mouse pointer object, the target mouse pointer object being generated by the first application based on acquired mouse pointer data, the mouse pointer data being image data corresponding to a mouse pointer image in a second application, the second application being an application installed in a virtualized operating system running on the electronic device; A second acquisition module is configured to acquire the current display position of the target mouse pointer object through the first application; The rendering and display module is configured to render the target mouse pointer object based on the current display position through the first application program to display a corresponding mouse pointer image.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images