Method and device for displaying image object
By judging and hiding inconsistent image objects in a hybrid operating system, and using the inference model to train the Windows Manager, the problem of inconsistent experience during runtime of different operating systems is solved, and the user experience is unified and improved.
Patent Information
- Application Number
- CN202310655270.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-05
- Publication Date
- 2024-12-06
AI Technical Summary
In a hybrid operating system, users cannot hide the work column, resulting in inconsistent experience when running different operating systems, affecting the user experience.
By starting the second operating system on an electronic device, we can determine whether the image object comes from the window synthesis manager, and determine whether the image object is hidden based on the consistency between the image content and the window synthesis manager, and use inference models (such as convolutional neural networks) ) to train CPU model, memory capacity, and desktop menus to map the window manager to ensure consistent display of image objects.
It realizes the consistency of user experience when different operating systems are running, reduces the gap in cognition and impression, and improves the user experience of hybrid operating systems.
Smart Images

Figure CN119088274A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a method and device for displaying image objects, and more particularly to a method and device for displaying image objects when different operating systems are running simultaneously. Background Art
[0002] A hybrid operating system (Hybrid OS) refers to a system that uses the same system kernel, system settings, and driver programs, and runs two or more different operating systems simultaneously on the same electronic device.
[0003] Starting from Windows 11, the taskbar will not be open to users to set to hide. When a user runs a hybrid operating system, the taskbar 100 cannot be set to hide (such as Figure 1 As shown in the figure, users will not be able to get the full user experience when running operating systems other than Windows 11.
[0004] Therefore, there is a need for a method and device for displaying image objects that can be consistent with the user's existing impression when running on two different operating systems and improve the gap between cognition and impression during use. Summary of the invention
[0005] The following disclosures are exemplary only and are not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features, other aspects, embodiments, and features will also be apparent by reference to the accompanying drawings and the following detailed description. That is, the following disclosures are provided to introduce concepts, key points, benefits, and novel and non-obvious technical advantages described herein. Selected, but not all, embodiments will be further described in detail as follows. Therefore, the following disclosures are not intended to be essential features of the claimed subject matter, nor are they intended to be used in determining the scope of the claimed subject matter.
[0006] Therefore, the main purpose of the present disclosure is to provide a method and device for displaying image objects.
[0007] The present disclosure provides a method for displaying an image object, which is used in a device, comprising: starting a second operating system when executing a first operating system; determining whether an image object in a register comes from a window composition manager of the second operating system; when the image object comes from the window composition manager of the second operating system, determining whether an image content of the image object is consistent with the window composition manager; and when the image content is inconsistent with the window composition manager, setting the image object to be hidden.
[0008] In some embodiments, the method further comprises: when the image content is consistent with the window composition manager, displaying the image object.
[0009] In some embodiments, the above-mentioned image content includes at least: pixels, ARGB (Alpha, Red, Green, Blue) values and application programming interface (Application Programming Interface, API).
[0010] In some embodiments, before starting the second operating system, the method further includes: training an inference model; and inputting a CPU model, a memory capacity and a desktop menu of the device into the inference model to obtain the window synthesis manager.
[0011] In some embodiments, the inference model is based on a Convolutional Neural Network (CNN) model or a Deep Neural Network (DNN) model.
[0012] The present disclosure provides a device for displaying an image object, comprising: one or more processors; and one or more computer storage media storing computer-readable instructions, wherein the processor uses the computer storage media to execute: when executing a first operating system, starting a second operating system; determining whether an image object in a register comes from a window synthesis manager of the second operating system; when the image object comes from the window synthesis manager of the second operating system, determining whether an image content of the image object is consistent with the window synthesis manager; and when the image content is inconsistent with the window synthesis manager, setting the image object to be hidden. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 This is a diagram showing that the taskbar cannot be hidden according to the settings in the operating system.
[0014] Figure 2 It is a schematic diagram showing the architecture of the hybrid operating system for image objects and display according to an embodiment of the present disclosure.
[0015] Figure 3 The flowchart of the method for displaying an image object according to an embodiment of the present disclosure is shown.
[0016] Figure 4 It is a flow chart showing a method for intelligent deployment of a hybrid operating system according to an embodiment of the present disclosure.
[0017] Figure 5is a schematic diagram showing a training inference model according to an embodiment of the present disclosure.
[0018] Figure 6 FIG. 1 is a diagram showing an exemplary operating environment for implementing an embodiment of the present disclosure.
[0019] The reference numerals are described as follows:
[0020] 100: Task list
[0021] 200: Hybrid Operating System
[0022] 210: The First Operating System
[0023] 212: Desktop Environment
[0024] 214: DirectX
[0025] 216: Desktop Window Manager
[0026] 218:D3D
[0027] 220: Second operating system
[0028] 222: Desktop Environment
[0029] 224: Xorg / Wayland
[0030] 226: Window Manager
[0031] 228: Display Server
[0032] 230: Synthesizer
[0033] 232: Graphical Application Programming Interface
[0034] 234: Client kernel
[0035] 2342: Virtual graphics processor
[0036] 236: Main operating system
[0037] 2362: Graphical Application Programming Interface
[0038] 240: Device driver
[0039] 250: Hardware
[0040] 300: Methods
[0041] S305, S310, S315, S320, S325: Steps
[0042] 400: Method Flowchart
[0043] S405, S410, S415, S420, S425: Steps
[0044] 500: Inference Model
[0045] 510: Desktop Menu
[0046] 520: Desktop Environment
[0047] 530: Window Composition Manager
[0048] 540: CPU model and memory capacity
[0049] 600: Electronic devices
[0050] 610: Bus
[0051] 612: Memory
[0052] 614: Processor
[0053] 616: Display components
[0054] 618: I / O port
[0055] 620: I / O components
[0056] 622: Power supply DETAILED DESCRIPTION
[0057] Various aspects of the present disclosure will be described more fully below with reference to the accompanying drawings. However, the present disclosure can be embodied in many different forms and should not be construed as being limited to any specific structure or function presented throughout the present disclosure. On the contrary, providing these aspects will make the present disclosure comprehensive and complete, and the present disclosure will fully convey the scope of the present disclosure to those skilled in the art. Based on the content taught herein, those skilled in the art will appreciate that, whether it is implemented alone or in combination with any other aspect of the present disclosure, any aspect disclosed herein is intended to be covered by the scope of the present disclosure. For example, any number of devices or execution methods proposed herein can be used to implement. In addition, in addition to the multiple aspects of the present disclosure proposed herein, the scope of the present disclosure is more intended to cover devices or methods implemented using other structures, functions, or structures and functions. It should be understood that any aspect disclosed herein can be embodied by one or more elements of the claims.
[0058] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect of the disclosure or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects of the disclosure or design. In addition, like numbers refer to like elements throughout the several figures, and the articles "a," "an," and "above" include plural references unless otherwise specified in the description.
[0059] It is understood that when an element is referred to as being "connected" or "coupled" to another element, the element may be directly connected or coupled to the other element or there may be an intermediate element. Conversely, when the element is referred to as being "directly connected" or "directly coupled" to another element, there are no intermediate elements. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., "between" versus "directly between," "adjacent" versus "directly adjacent," etc.).
[0060] The disclosed embodiments provide a method and device for displaying image objects, so as to achieve the same experience as the native operating system when running different operating systems on the same electronic device.
[0061] Figure 2 2 is a schematic diagram showing the architecture of the hybrid operating system 200 for image objects and display according to an embodiment of the present disclosure. The hybrid operating system 200 is running in an electronic device, and includes a first operating system 210 and a second operating system 220 .
[0062] The first operating system 210 is the main operating system of the electronic device. In this embodiment, the first operating system 210 is a Windows operating system. In the Windows operating system, the desktop environment 212 is a Windows desktop. The window composition manager is a desktop window manager (Desktop Window Manager, DWM) 216. The display-related application programming interface (Application Programming Interface, API) is DirectX 214, and the drawing-related application programming interface is Direct3D (D3D) 218.
[0063] The second operating system 220 is an operating system completely different from the first operating system 210. In this embodiment, the second operating system 220 is a Linux operating system. In the Linux operating system, the top layer is a desktop environment 222, and the display-related application program interface is Xorg or Wayland 224. The window composition manager may include a window manager 226, a display server 228, and a compositor 230. The following architecture is a virtual graphics processing unit (GPU) 2342 in a graphical application program interface 232 and a guest kernel 234.
[0064] The subsequent architecture layers are the host operating system (Host OS) 236, the device driver 240 and the hardware 250. The virtual graphics processor 234 accesses the graphical application program interface 2362 in the host operating system 236. The device driver 240 and the hardware 250 can be used by the first operating system 210 and the second operating system 220.
[0065] In order to maintain the same user experience as the native second operating system 220, for example, the location and coordinates of the start menu and control panel must be consistent with the second operating system 220. The window manager 226 in the second operating system 220 must be deployed. Since the second operating system 220 is deployed based on the first operating system 210, there must be a mapping relationship between the graphical application interface 232 of the second operating system 220 and the D3D 218 of the first operating system 210, so that signal transmission and application interface mapping can be supported.
[0066] It should be understandable. Figure 2 The electronic device for running the hybrid operating system 200 may be implemented by any type of electronic device, such as Figure 6 The electronic device 600 described, as Figure 6 shown.
[0067] Figure 3 FIG. 3 is a flow chart showing a method 300 for displaying an image object according to an embodiment of the present disclosure. The method may be performed as follows: Figure 2 The electronic device running the hybrid operating system 200 is executed.
[0068] In step S305, the electronic device starts a second operating system while executing a first operating system. Figure 2 Taking the architecture as an example, when the second operating system is a Linux operating system, after starting the second operating system, a virtual desktop will be generated, and the synthesizer of the second operating system will be displayed in full screen mode in the virtual desktop, so that after the second operating system is started, the user can also maintain the complete experience of using the first operating system.
[0069] In step S310, the electronic device determines whether a graphic object in a register is from a window composition manager of the second operating system, wherein the graphic object in the register is the graphic object to be displayed on the screen. In other words, the electronic device confirms whether the graphic object complies with the graphic composition mechanism of the window composition manager.
[0070] When the image object is not from the window composition manager of the second operating system (“No” in step S310 ), the process ends. In other words, the electronic device may not be executing the action of starting the second operating system.
[0071] When the above-mentioned image object comes from the above-mentioned window synthesis manager of the above-mentioned second operating system ("Yes" in step S310), in step S315, the electronic device determines whether an image content of the above-mentioned image object is consistent with the above-mentioned window synthesis manager, wherein the above-mentioned image content at least includes: pixels, ARGB (Alpha, Red, Green, Blue) values and application programming interface (Application Programming Interface, API).
[0072] When the image content is inconsistent with the window composition manager ("No" in step S315), in step S320, the electronic device sets the image object to be hidden. In more detail, the image content is inconsistent with the window composition manager, which means that an image object different from the window composition manager will be displayed on the screen. In order to avoid the different image objects being displayed on the screen, the different image objects need to be set to be hidden. In one embodiment, the electronic device can set the display coordinates of the image object to be hidden by setting them to negative values.
[0073] When the image content is consistent with the window composition manager ("Yes" in step S315), in step S325, the electronic device displays the image object on the screen. Figure 2 For example, when the second operating system is a Linux operating system, the graphical user interface (GUI) of the Linux operating system, such as the desktop environment or built-in applications, will be displayed on the synthesizer. In another example, when the second operating system is a Windows operating system, the desktop environment or built-in applications of the Windows operating system will be displayed on the desktop window manager.
[0074] Figure 4 is a flowchart 400 showing a method for intelligent deployment of a hybrid operation system according to an embodiment of the present disclosure, wherein the method may be performed as follows Figure 2 The method is mainly performed by the electronic device running the hybrid operating system 200. Figure 3 Completed before starting the second operating system. The main task is for the user to deploy a distribution version suitable for the electronic device to be used, and to infer the window management mechanism corresponding to the distribution version of the electronic device through an inference model.
[0075] Before the process begins, the electronic device can first train an inference model. How to train this inference model will be described in Figure 5 In the description.
[0076] In step S405 , the electronic device receives a desktop menu input by the user, such as Ubuntu, Debian, Deepin, RedHat, etc.
[0077] In step S410 , the electronic device updates the package repository to determine that the installed package software is the latest version and is applicable to the electronic device.
[0078] In step S415, the electronic device inputs the CPU model, memory capacity and desktop menu of the electronic device into an inference model to obtain a desktop environment and a window synthesis manager corresponding to the above desktop environment, wherein the above inference model is based on a Convolutional Neural Network (CNN) model or a Deep Neural Network (DNN) model.
[0079] In step S420, the electronic device installs the desktop environment and a window composition manager corresponding to the desktop environment. In one embodiment, when the operating system is a Linux operating system, the window composition manager includes a window manager, a display server, and a synthesizer. The electronic device will install the desktop environment and the window manager, display server, and synthesizer corresponding to the desktop environment.
[0080] In step S425 , the electronic device performs mapping of graphical application program interfaces in different operating systems to achieve the effect of GPU acceleration.
[0081] Figure 5 is a schematic diagram showing a training inference model 500 according to an embodiment of the present disclosure. This method can be performed as follows Figure 2 The electronic device running the hybrid operating system 200 is mainly used to deploy a desktop environment suitable for the user and to maintain the same user experience as the native operating system.
[0082] The electronic device may first obtain training data before training the inference model 500. It should be noted that, as used herein, the term "training" is used to identify an object used for the inference model 500. Therefore, the training data refers to data used for training the inference model.
[0083] The training data may include at least: a desktop menu 510, a desktop environment 520, a window composition manager 530, a CPU model, and a memory capacity 540. Therefore, the electronic device may train the inference model 500 for the correlation between the desktop menu and the desktop environment, the correlation between the desktop environment and the window composition manager, and the correlation between the CPU model and the memory capacity.
[0084] The relationship between desktop menus and desktop environments: In order to maintain the same user experience as the native operating system, each operating system desktop menu has a corresponding desktop environment. For example, the most commonly used or popular desktop environment for an operating system desktop menu can be obtained through relevant literature and market research. Another example is to illustrate the mapping relationship between which desktop environment is most commonly used for desktop menus in electronic devices with low computing power, or which desktop environment is most commonly used for desktop menus in electronic devices dedicated to e-sports.
[0085] The relationship between the desktop environment and the window compositing manager: Each desktop environment has a supported window compositing manager, and the most commonly used combinations can be obtained from the supported window compositing managers. For example: Steam OS uses KDE Plasma as its desktop environment, and its most commonly used window compositing manager is Kwin-X11.
[0086] Correlation between desktop environment and CPU model and memory capacity: For example, in the Linux operating system, there are many types of desktop environments, some are more modern and fancy like GNOME, and some are simpler like LXDE. The former GNOME requires more CPU and memory space, while the latter LXDE is more suitable for low-spec electronic devices.
[0087] Therefore, the training data set used to train the inference model 500 at least includes training data such as desktop menu, desktop environment, window composition manager, CPU model and memory capacity. The electronic device is trained based on the correlation between the above training data to train the inference model 500.
[0088] In one embodiment, the inference model 500 is based on a Convolutional Neural Network (CNN) model or a Deep Neural Network (DNN) model.
[0089] In summary, the present disclosure proposes a method and device for displaying image objects, which can enable users to achieve consistency with their existing impressions when running two different operating systems, reduce the gap between cognition and impression, and improve the user's experience of using a hybrid operating system.
[0090] For the embodiments of the present invention that have been described, the following describes an exemplary operating environment in which the embodiments of the present invention can be implemented. Figure 6 , Figure 6600. The electronic device 600 is merely an example of a suitable computing environment and is not intended to suggest any limitation on the scope of use or functionality of the invention. The electronic device 600 should not be interpreted as having any dependency or requirement relating to any one or combination of the illustrated elements.
[0091] The present invention may be implemented in computer program code or machine-usable instructions, which may be computer executable instructions of a program module, which is executed by a computer or other machine, such as a personal digital assistant or other portable device. Generally speaking, a program module includes routines, programs, objects, components, data structures, etc. A program module refers to a program code that performs a specific task or implements a specific abstract data type. The present invention may be implemented in a variety of system configurations, including portable devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The present invention may also be implemented in a distributed computing environment, processing devices connected by a communication network.
[0092] refer to Figure 6 Electronic device 600 includes bus 610 that directly or indirectly couples the following devices, memory 612, one or more processors 614, one or more display elements 616, input / output (I / O) ports 618, input / output (I / O) elements 620, and an illustrative power supply 622. Bus 610 represents an element that can be one or more buses (e.g., an address bus, a data bus, or a combination thereof). Although Figure 6 For simplicity, the various blocks are shown with lines. Actually, the boundaries of the various elements are not specific. For example, the presentation elements of the display device may be regarded as I / O elements; and the processor may have a memory.
[0093] The electronic device 600 generally includes various computer-readable media. Computer-readable media can be any available media that can be accessed by the electronic device 600, including both volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media. Computer-readable media also includes volatile and non-volatile media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage devices, magnetic disks, magnetic disks, magnetic disk storage devices or other magnetic storage devices, or any other media that can be used to store the desired information and can be accessed by the electronic device 600. Computer storage media itself does not include signals.
[0094] Communication media generally contain computer readable instructions, data structures, program modules or other data in the form of modular data signals such as carrier waves or other transmission mechanisms, and include any information delivery media. The term "modular data signal" refers to a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example and not limitation, communication media include wired media such as a wired network or direct wired connection and wireless media such as audio, radio frequency, infrared and other wireless media. Combinations of the above media are included within the scope of computer readable media.
[0095] Memory 612 includes computer storage media in the form of volatile and non-volatile memory. Memory can be removable, non-removable, or a combination of the two. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Electronic device 600 includes one or more processors that read data from entities such as memory 612 or I / O components 620. Display component 616 displays data indications to a user or other device. Exemplary display components include display devices, speakers, printing components, vibration components, etc.
[0096] I / O port 618 allows electronic device 600 to be logically connected to other devices including I / O element 620, some of which are built-in devices. Exemplary elements include microphones, joysticks, game consoles, satellite dish signal receivers, scanners, printers, wireless devices, etc. I / O element 620 can provide a natural user interface for processing gestures, sounds or other physiological inputs generated by the user. In some examples, these inputs can be transmitted to a suitable network element for further processing. Electronic device 600 can be equipped with a depth camera, such as a stereo camera system, an infrared camera system, an RGB camera system, and a combination of these systems to detect and identify objects. In addition, electronic device 600 can be equipped with a sensor (e.g., radar, lidar) to periodically sense the surrounding environment within a sensing range, and generate sensor information indicating that it is associated with the surrounding environment. Furthermore, electronic device 600 can be equipped with an accelerometer or gyroscope that detects motion. The output of the accelerometer or gyroscope can be provided to the electronic device 600 for display.
[0097] In addition, the processor 614 in the electronic device 600 may also execute the programs and instructions in the memory 612 to present the actions and steps described in the above embodiments, or other descriptions of the contents in the specification.
[0098] Any specific order or hierarchy of steps in the procedures disclosed herein is purely by way of example. Based on design preferences, it should be understood that any specific order or hierarchy of steps in the procedures may be rearranged within the scope of this document. The accompanying method claims present elements of the various steps in an exemplary order and, therefore, should not be limited to the specific order or hierarchy presented.
[0099] The use of ordinal numbers such as "first", "second", "third", etc. in the claims to modify elements does not itself imply any priority, precedence, sequence between elements, or order of steps performed by the method, but is only used as an identifier to distinguish different elements with the same name (with different ordinal numbers).
[0100] Although the present disclosure has been disclosed as above by way of implementation examples, it is not intended to limit the present case. Anyone familiar with the art may make some changes and modifications without departing from the spirit and scope of the present disclosure. Therefore, the protection scope of the present case shall be determined by the definition of the appended claims.
Claims
1. A method for displaying an image object, used in a device, comprising: When executing a first operating system, starting a second operating system; Determining whether an image object in a register comes from a window composition manager of the second operating system; When the graphic object comes from the window composition manager of the second operating system, determining whether a graphic content of the graphic object is consistent with that of the window composition manager; and When the image content is inconsistent with the window composition manager, the image object is set to be hidden.
2. The method for displaying a graphic object as claimed in claim 1, wherein the method further comprises: When the image content is consistent with the window composition manager, the image object is displayed.
3. The method for displaying an image object as claimed in claim 1, wherein the image content at least comprises: Pixels, ARGB (Alpha, Red, Green, Blue) values and application interface.
4. The method for displaying a graphic object as claimed in claim 1, wherein before starting the second operating system, the method further comprises: training an inference model; and A CPU model, a memory capacity and a desktop menu of the device are input into the inference model to obtain the window synthesis manager.
5. The method for displaying an image object as claimed in claim 4, wherein the inference model is based on a convolutional neural network model or a deep neural network model.
6. A device for displaying an image object, comprising: one or more processors; as well as One or more computer storage media storing computer readable instructions, wherein the processor uses the computer storage media to perform: When executing a first operating system, starting a second operating system; Determining whether an image object in a register comes from a window composition manager of the second operating system; When the graphic object comes from the window composition manager of the second operating system, determining whether a graphic content of the graphic object is consistent with that of the window composition manager; and When the image content is inconsistent with the window composition manager, the image object is set to be hidden.
7. The apparatus for displaying a graphic object as claimed in claim 6, wherein the processor uses the computer storage medium to further execute: When the image content is consistent with the window composition manager, the image object is displayed.
8. The device for displaying an image object as claimed in claim 6, wherein the image content at least comprises: Pixels, ARGB (Alpha, Red, Green, Blue) values and application interface.
9. The apparatus for displaying a graphic object as claimed in claim 6, wherein before starting the second operating system, the processor uses the computer storage medium to further execute: training an inference model; and A CPU model, a memory capacity and a desktop menu of the device are input into the inference model to obtain the window synthesis manager.
10. The device for displaying an image object as claimed in claim 9, wherein the inference model is based on a convolutional neural network model or a deep neural network model.
