Graphics system, graphic system operation method and device, storage medium and equipment

Abstract

The invention discloses a graphic system, a graphic system operation method, a storage medium and equipment, and relates to the technical field of communication, the graphic system comprises a system service layer, a native service layer and an operation library; the system service layer comprises self-research system service; the native service layer reuses native system service in a native user interface framework of the system; the runtime library comprises a three-party engine layer and an engine adaptation layer, and the three-party engine layer adopts an engine of a third-party user interface framework; the self-research system service in the system service layer is in butt joint with the three-party engine layer through the engine adaptation layer; and the self-research system service is also used for connecting the native system service with a user interface development application constructed based on the graphic system. According to the method, a third-party user interface framework and a system native user interface framework can be effectively integrated, and a high-performance graphic system is realized.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a graphics system, a graphics system operation method, an apparatus, a storage medium and a device. Background Art

[0002] The user interface (UI) framework, such as the Flutter framework, can run on various operating systems such as Android or iOS, and can be used by developers to develop user interfaces. Currently, the user interface (UI) framework, such as the Flutter framework, in various systems is used as a third-party user interface (UI) framework in collaboration with the system's native user interface (UI) framework. The collaborative use method involves communication between the third-party user interface (UI) framework and the native operating system, resulting in performance loss. Summary of the invention

[0003] The embodiment of the present application provides a graphics system, which can realize a high-performance graphics system.

[0004] The embodiments of the present application provide the following technical solutions:

[0005] According to one embodiment of the present application, a graphics system includes a system service layer, a native service layer and a runtime library; the system service layer includes self-developed system services: the native service layer reuses the native system services in the system native user interface framework; the runtime library includes a third-party engine layer and an engine adaptation layer, and the third-party engine layer uses the engine of the third-party user interface framework; the self-developed system services in the system service layer are connected to the third-party engine layer through the engine adaptation layer; the self-developed system services are also used to connect the native system services with user interface development applications built based on the graphics system.

[0006] In some embodiments of the present application, the self-developed system service includes a development window management service and an application creation service; the development window management service is used to connect the native system service with the user interface development application; the application creation service is used to create the user interface development application.

[0007] In some embodiments of the present application, the self-developed system service also includes a development application management service; the development application management service is used to manage the startup and life cycle of the user interface development application.

[0008] In some embodiments of the present application, the graphics system further includes a system framework layer; the user interface development application includes the runtime library and screen components, and the screen components include services in the system framework layer.

[0009] In some embodiments of the present application, the screen component includes a system window management service and a system application management service in the system framework layer; the system application management service is used to perform lifecycle callbacks with the development application management service in the self-developed system service; the system window management service is used to apply to the development window management service in the self-developed system service for building display and input channels.

[0010] In some embodiments of the present application, the graphics system also includes an interface layer and an application layer, the user interface development application includes the interface layer and the application layer, and the sub-application in the application layer calls some specified system capabilities in the graphics system through the interface layer.

[0011] According to one embodiment of the present application, a graphics system operation method is applicable to the graphics system described in any one of the embodiments of the present application, and the graphics system operation method includes: during the system startup process, starting the native system service and the self-developed system service through an initialization process; in response to an application startup instruction, requesting the application creation service to create the user interface development application through the application development management service in the self-developed system service; applying to the development window management service in the self-developed system service to build a display and input channel through the system window management service in the user interface development application.

[0012] In some embodiments of the present application, the application for building a display and input channel from the development window management service in the self-developed system service through the system window management service in the user interface development application includes: delivering a channel construction application to the development window management service in the self-developed system service through the system window management service in the user interface development application; the development window management service responds to the channel construction application, instructs the display content synthesis service in the native system service to build a display channel, and instructs the key processing service in the native system service to build an input channel.

[0013] According to another embodiment of the present application, a storage medium stores a computer program thereon, and when the computer program is executed by a processor of a computer, the computer executes the method described in the embodiment of the present application.

[0014] According to another embodiment of the present application, a device may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the method described in the embodiment of the present application.

[0015] According to another embodiment of the present application, a computer program product or a computer program includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0016] In the embodiment of the present application, the graphics system includes a system service layer, a native service layer and a runtime library; the system service layer includes self-developed system services: the native service layer reuses the native system services in the system native user interface framework; the runtime library includes a third-party engine layer and an engine adaptation layer, and the third-party engine layer uses the engine of the third-party user interface framework; the self-developed system service in the system service layer is connected to the third-party engine layer through the engine adaptation layer; the self-developed system service is also used to connect the native system service with the user interface development application built on the graphics system. In this way, the third-party user interface framework and the system native user interface framework are effectively integrated to achieve a high-performance graphics system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A schematic diagram of the structure of a graphics system according to an embodiment of the present application is shown.

[0019] Figure 2 A schematic structural diagram of a graphics system according to another embodiment of the present application is shown.

[0020] Figure 3 A flowchart of a graphics system operation according to an embodiment of the present application is shown.

[0021] Figure 4 A flowchart of a graphics system operation according to another embodiment of the present application is shown.

[0022] Figure 5 A block diagram of a device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are only used to explain the present disclosure and are not intended to limit the present disclosure. In addition, the embodiments provided below are partial embodiments for implementing the present disclosure, rather than providing all embodiments for implementing the present disclosure. In the absence of conflict, the technical solutions recorded in the embodiments of the present disclosure can be implemented in any combination.

[0024] It should be noted that, in the embodiments of the present disclosure, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or apparatus including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or apparatus. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other related elements in the method or apparatus including the element (such as a step in the method or a unit in the apparatus, for example, a unit may be a part of a circuit, a part of a processor, a part of a program or software, etc.).

[0025] For example, the graphics system method provided in the embodiment of the present disclosure includes a series of steps, but the graphics system method provided in the embodiment of the present disclosure is not limited to the recorded steps. Similarly, the graphics system device provided in the embodiment of the present disclosure includes a series of units, but the device provided in the embodiment of the present disclosure is not limited to including the units explicitly recorded, and may also include units required to obtain relevant information or perform processing based on the information.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0027] It is understandable that in the specific implementation of this application, relevant data is involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0028] Figure 1 The flowchart of a graphics system method according to an embodiment of the present application is schematically shown. The graphics system method can be executed by any device with processing capabilities, such as a television, a computer, a mobile phone, a smart watch, and a home appliance.

[0029] like Figure 1As shown, the graphics system may include a runtime 110, a system service layer 120 and a native service layer 130; the system service layer 120 includes self-developed system services: the native service layer 130 reuses the native system services in the system native user interface framework; the runtime 110 includes a third-party engine layer and an engine adaptation layer, and the third-party engine layer adopts the engine of the third-party user interface framework; the self-developed system services in the system service layer are connected to the third-party engine layer through the engine adaptation layer; the self-developed system services are also used to connect the native system services with the user interface development application built based on the graphics system.

[0030] The system service layer 120 includes self-developed system services, which can be some system services that a third-party user interface framework configured according to needs needs to rely on. The native service layer 130 reuses the native system services in the system native user interface framework, which is the native user interface (UI) framework in the operating system (such as the Android system). Some native system services in the system native user interface framework are directly reused to form the native service layer 130.

[0031] Furthermore, the runtime 110 includes a third-party engine layer and an engine adaptation layer. The third-party engine layer uses the engine of a third-party user interface framework. For example, the third-party engine layer directly uses the native engine of the Flutter framework without modification. The engine adaptation layer is configured in the runtime 110 to connect to the self-developed system service in the system service layer through the engine adaptation layer.

[0032] Based on the graphics system, a user interface development application for user interface development can be built (for example, when the third-party user interface framework is the Flutter framework, the user interface development application is a Flutter application (Flutter APP) built based on the graphics system). The self-developed system service in the system service layer 120 can connect the native system service with the user interface development application built based on the graphics system, so that the user can perform high-performance user interface development based on the user interface development application built based on the graphics system.

[0033] In short, in this way of the embodiment of the present application, the third-party user interface framework and the system native user interface framework are effectively integrated to realize a high-performance graphics system. For example, when the third-party user interface framework is the Flutter framework and the operating system is the Android system (Android), the Flutter framework and the system native user interface framework in the Android Open Source Project (AOSP, "Android Open Source Project") can be conveniently and effectively integrated to realize a high-performance graphics system.

[0034] Described below Figure 1 When the graphics system is implemented under the embodiment, further optional specific embodiments are provided under each step performed.

[0035] In one embodiment, the self-developed system service includes a development window management service and an application creation service; the development window management service is used to connect the native system service with the user interface development application; the application creation service is used to create the user interface development application.

[0036] See also Figure 2 , a graphics system in a scene such as Figure 2 As shown in the architecture of , the self-developed system services in the system service layer 120 may include the development of window management services (WindowMS) and application creation services (FlutterZygote). The application creation service can be based on the graphics system to create at least Figure 2 The user interface development application of the runtime library 110 shown, such as Figure 2 When the third-party engine layer used in the runtime library 110 shown is the engine (Engine) of the Flutter framework, the user interface development application is a Flutter application. At this time, the application creation service can reuse the Dart virtual machine.

[0037] Furthermore, the development window management service is used to connect the native system services in the native service layer with the user interface development application, so that the user interface development application can request related services in the native system service to perform related operations through the development window management service. For example, the user interface development application can request related services in the native system service to build a display and input channel through the development window management service.

[0038] Furthermore, in one embodiment, the self-developed system service also includes a development application management service; the development application management service is used to manage the startup and life cycle of the user interface development application.

[0039] The development application management service (AppMS) can manage the startup of the user interface development application. For example, when it is necessary to start a user interface development application, the development application management service can pass instructions to the application creation service, so that the application creation service creates the user interface development application and starts it.

[0040] The development application management service can also manage the life cycle of the user interface development application. For example, the development application management service can monitor the performance of the user interface development application, maintain its functions, and so on.

[0041] In one embodiment, the graphics system further includes a system framework layer; the user interface development application includes the runtime library and screen components, and the screen components include services in the system framework layer.

[0042] For example, Figure 2 As shown, the graphics system may also include a system framework layer (Framework), which may include related services in the framework layer in the Android system. The user interface development application built based on the graphics system may include a runtime library and a screen component (Activity). The screen component may specifically include screen-related services in the system framework layer (Framework), such as a system window management service (WindowManager) and a system application management service (AppManager), and thus, the user interface development application may implement a more complete graphics system function based on the screen component and the runtime library.

[0043] In one embodiment, the screen component includes a system window management service and a system application management service in the system framework layer; the system application management service is used to perform lifecycle callbacks with the development application management service in the self-developed system service; the system window management service is used to apply to the development window management service in the self-developed system service for building display and input channels.

[0044] The screen components of the user interface development application include the system window management service (WindowManager) and the system application management service (AppManager). The system application management service can perform lifecycle callbacks with the development application management service in the self-developed system service, so that lifecycle management can be performed reliably.

[0045] Furthermore, the system window management service is used to apply to the window management service developed in the self-developed system service for building display and input channels. Therefore, when the user interface development application is started, it can send a request to the window management service developed in the self-developed system service based on the system window management service to apply for building display and input channels for the user interface development application, thereby effectively realizing the graphics function of the user interface development application.

[0046] In one embodiment, the graphics system further includes an interface layer and an application layer, the user interface development application includes the interface layer and the application layer, and the sub-application in the application layer calls part of the specified system capabilities in the graphics system through the interface layer.

[0047] For example, Figure 2As shown, the graphics system also includes an interface layer 220 and an application layer 210. The interface layer 220 includes multiple interfaces (APIs), which can connect to some specified system capabilities in the graphics system. Therefore, some specified system capabilities can be provided to sub-applications in the application layer in the form of SDK, thereby further improving the performance of the graphics system.

[0048] The user interface development application may include an interface layer and an application layer. The application layer may include multiple sub-applications in the user interface development application, for example, Figure 2 Multiple Dart APPs (sub-applications developed based on the Dart language) are shown.

[0049] Furthermore, the graphics system may also include Figure 2 The HAL layer 240 and the kernel layer 250 in the operating system are shown. The HAL layer 240 may include a graphics allocation (Gralloc), a hardware compositor (HWC), etc., and the kernel layer 250 may include a frame buffer device (framework) and a display component (Display).

[0050] It should be noted that, in one scenario, the graphics system of the embodiment of the present application includes: Figure 2 All layers shown, where the services, components, or modules in each layer come from Figure 2 As shown in the source representation box, the services, components or modules in the boxes with the same background belong to the same source. For example, the box with a slash background corresponds to the source of the chip (Soc), the box with a gray background corresponds to the source of the third-party user interface framework, the box with a dotted background corresponds to the system native user interface framework in the Android Open Source Project (AOSP), and the box with a white background corresponds to the source of the developer's self-developed.

[0051] In addition, the present application provides a graphics system operation method, which is applicable to any of the graphics systems described above, wherein the meanings of the nouns are the same as in the above-mentioned graphics systems, and specific implementation details can refer to the description in the embodiments. Figure 3 The flowchart of the graphics system operation method according to an embodiment of the present application is shown. Figure 4 The flowchart of the graphics system operation method according to another embodiment of the present application is shown.

[0052] See also Figure 3 and Figure 4 , the graphics system operation method may include:

[0053] Step S310, during the system startup process, start the native system service and the self-developed system service through the initialization process; Step S320, in response to the application startup instruction, request the application creation service to create the user interface development application through the application development management service in the self-developed system service; Step S330, apply to the window management service developed in the self-developed system service to build a display and input channel through the system window management service in the user interface development application.

[0054] In step S310, during the system startup process, the system first starts the initialization process (init), and further starts the native system services and the self-developed system services through the initialization process. Figure 4 As shown, during the system startup process, the initialization process (init) can start the self-developed system service in the system service layer (including development application management service, development window management service and application creation service) in step S410, start the display content synthesis service (SurfaceFlinger) in the native system service in step S420, and start the key processing service (InputFlinger) in the native system service in step S430.

[0055] In step S320, in response to the application start instruction, the development application management service in the self-developed system service may request the application creation service to create a user interface development application. Figure 4 As shown, in step S440, the development application management service sends a request to the application creation service; in step S450, the application creation service creates Figure 4 The user interface development application shown includes an application library and screen components. The screen components include screen-related services in the system framework layer (Framework), such as the system application management service (AppManager) and the system window management service (WindowManager). The system window management service (WindowManager) can be located in the application window service (FlutterWindow), and the application window service (FlutterWindow) can also include services such as key event receiving service (InputReciever).

[0056] In step S330, the system window management service in the user interface development application applies to the development window management service in the self-developed system service to build a display and input channel. Specifically, the application to build a display and input channel from the system window management service in the user interface development application to the development window management service in the self-developed system service may include: passing a channel construction application to the development window management service in the self-developed system service from the system window management service in the user interface development application; the development window management service responds to the channel construction application, instructs the display content synthesis service in the native system service to build a display channel, and instructs the key processing service in the native system service to build an input channel.

[0057] like Figure 4 As shown, in step S460, the system window management service in the application window service (FlutterWindow) transmits a channel construction application to the development window management service, and then, in step S470, the development window management service instructs the display content synthesis service in the native system service to build a display channel, and in step S480, the development window management service instructs the key processing service in the native system service to build an input channel. After the display channel and the input channel are built, the user interface development can be carried out based on the display channel and the input channel.

[0058] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0059] In addition, the present application also provides a device, such as Figure 5 As shown, Figure 5 A block diagram of a device according to an embodiment of the present application is shown, specifically:

[0060] The device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will appreciate that Figure 5 The device structure shown in the figure does not constitute a limitation of the device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0061] The processor 501 is the control center of the device, and uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, it executes various functions of the computer device and processes data, thereby monitoring the device as a whole. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501.

[0062] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0063] The device also includes a power supply 503 for supplying power to each component. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 503 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0064] The device may further include an input unit 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0065] Although not shown, the device may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 501 in the device will load the executable files corresponding to the processes of one or more computer programs into the memory 502 according to the following instructions, and the processor 501 will run the computer programs stored in the memory 502, thereby realizing various functions in the aforementioned embodiments of the present application.

[0066] For example, processor 501 may execute the following steps: during the system startup process, the native system service and the self-developed system service are started through an initialization process; in response to an application startup instruction, the application creation service is requested to create the user interface development application through the application management service developed in the self-developed system service; and the system window management service in the user interface development application requests the window management service developed in the self-developed system service to build a display and input channel.

[0067] In one embodiment, the application for building a display and input channel from the development window management service in the self-developed system service through the system window management service in the user interface development application may include: delivering a channel construction application to the development window management service in the self-developed system service through the system window management service in the user interface development application; the development window management service responds to the channel construction application, instructs the display content synthesis service in the native system service to build a display channel, and instructs the key processing service in the native system service to build an input channel.

[0068] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0069] To this end, an embodiment of the present application further provides a storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any method provided in the embodiment of the present application.

[0070] The storage medium may be a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0071] Since the computer program stored in the storage medium can execute the steps in any method provided in the embodiments of the present application, the beneficial effects that can be achieved by the method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0072] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0073] It should be understood that the present application is not limited to the embodiments that have been described above and shown in the accompanying drawings, but various modifications and changes may be made without departing from the scope thereof.

Claims

1. A graphics system, characterized in that: The graphics system includes a system service layer, a native service layer and a runtime library; The system service layer includes self-developed system services: The native service layer reuses native system services in the system native user interface framework; The runtime library includes a third-party engine layer and an engine adaptation layer, wherein the third-party engine layer adopts an engine of a third-party user interface framework; The self-developed system service in the system service layer is connected to the third-party engine layer through the engine adaptation layer; the self-developed system service is also used to connect the native system service with the user interface development application built based on the graphics system.

2. The graphics system according to claim 1, characterized in that The self-developed system services include developing window management services and application creation services; The development window management service is used to connect the native system service with the user interface development application; the application creation service is used to create the user interface development application.

3. The graphics system according to claim 2, characterized in that The self-developed system service also includes a development application management service; the development application management service is used to manage the startup and life cycle of the user interface development application.

4. The graphics system according to any one of claims 1 to 3, characterized in that: The graphic system also includes a system framework layer; the user interface development application includes the runtime library and screen components, and the screen components include services in the system framework layer.

5. The graphics system according to claim 4, characterized in that The screen component includes the system window management service and the system application management service in the system framework layer; the system application management service is used to perform life cycle callback with the development application management service in the self-developed system service; The system window management service is used to apply to the window management service developed in the self-developed system service to build a display and input channel.

6. The graphics system according to claim 1, characterized in that The graphic system also includes an interface layer and an application layer. The user interface development application includes the interface layer and the application layer. The sub-application in the application layer calls part of the specified system capabilities in the graphic system through the interface layer.

7. A method for operating a graphics system, characterized in that: Applicable to the graphics system according to any one of claims 1 to 6, the graphics system operation method comprising: During the system startup process, the native system service and the self-developed system service are started through the initialization process; In response to the application start instruction, requesting the application creation service to create the user interface development application through the application development management service in the self-developed system service; The system window management service in the user interface development application applies to the development window management service in the self-developed system service to build a display and input channel.

8. The method according to claim 7, characterized in that The step of applying to the development window management service in the self-developed system service to build a display and input channel through the system window management service in the user interface development application includes: Delivering a channel construction application to the development window management service in the self-developed system service through the system window management service in the user interface development application; In response to the channel construction request, the development window management service instructs the display content synthesis service in the native system service to construct a display channel, and instructs the key processing service in the native system service to construct an input channel.

9. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 7 to 8.

10. A device, characterized in that: include: a memory storing a computer program; A processor reads a computer program stored in a memory to execute the method according to any one of claims 7 to 8.

Images