Address mapping method and apparatus

Abstract

Embodiments of the present application provide an address mapping method and apparatus. In the method, a memory allocation operation can be divided into a production phase and a consumption phase; and in each phase, virtual addresses are only dynamically mapped to related chips as required, so that the chips do not affect each other. In this way, a shared physical memory between a plurality of chips can be increased, thereby improving the overall performance of an electronic device.

Description

Address mapping method and device

[0001] The present application claims priority to the Chinese patent application No. 202411319910.6, filed on September 20, 2024, and entitled "Address mapping method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of electronic devices, and in particular to an address mapping method and device. BACKGROUND

[0003] With the rapid development of electronic devices, the performance of some hardware of the electronic devices is also getting stronger and stronger, for example, the bit width of the hardware such as the graphics processing unit (GPU) and the central processing unit (CPU) of the electronic devices is also getting larger and larger.

[0004] However, although the performance of the hardware of the electronic devices is getting stronger and stronger, the overall performance of the electronic devices does not improve much. SUMMARY

[0005] The present application provides an address mapping method and device. It is beneficial to improve the shared physical memory between multiple chips, and thus improve the overall performance of the electronic device.

[0006] In a first aspect, an address mapping method is provided, which is applied to an electronic device including a first application, a first device, a second device, a third device and a physical memory. The bit width of the first device, the bit width of the second device and the bit width of the third device are not completely equal. The method can include the following steps.

[0007] The first application applies for a memory. Then, the electronic device applies for a first virtual address of the first device and a second virtual address of the second device, and allocates a first memory in the physical memory, and maps the first virtual address and the second virtual address to a first physical address of the first memory, respectively. The first memory is used at least for storing first data written by the first device. Then, the electronic device applies for a third virtual address of the third device, and maps the third virtual address to the first physical address. Then, the third device obtains the first data from the first memory corresponding to the first physical address for processing.

[0008] In the embodiment of the present application, by applying the first virtual address of the first device in the production data stage, and allocating the first memory in the physical memory and mapping the first virtual address to the first physical address of the first memory, the first device can write the first data to the first memory, then, the second virtual address of the second device is applied, and the second virtual address is mapped to the first physical address of the first memory, so that if the second device is the producer device, the second device can write the second data to the first memory, and if the second device is the consumer device, the second device can also obtain the first data from the first memory for processing. Then, when the third device is the consumer device, the third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address, so that the third device can obtain the first data from the first memory corresponding to the first physical address for processing. Since the mapping of the virtual address is divided into multiple stages, for example, divided into the production data stage and the consumption data stage, the shared physical memory can be dynamically mapped to the virtual address of each chip, thereby improving the overall performance of the electronic device.

[0009] In a possible implementation, after the first application applies for the memory, the method can further include:

[0010] The first identifier of the first device and the second device is set, and the first identifier indicates that the device has been mapped to the first physical address.

[0011] In the embodiment of the present application, the first identifier of the first device is set, which indicates that the first device has been mapped to the first physical address, in other words, the virtual address of the applied first device has been mapped to the first physical address. The second identifier of the second device is set, which indicates that the second device has been mapped to the first physical address, in other words, the virtual address of the applied second device has been mapped to the first physical address. Optionally, the identifier of the embodiment of the present application can also be called a flag bit.

[0012] In the embodiment of the present application, by setting the identifier after the virtual address of the device is mapped to the physical address, it can be known through the identifier whether the physical address is mapped to the virtual address of the device, so that the data in the memory corresponding to the physical address can be read and written in time, thereby improving the efficiency of data processing. Moreover, the number of virtual addresses of the applied device can be reduced.

[0013] In a possible implementation, the third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address, including:

[0014] In a case where it is determined that the data is processed by the third device and the first identifier of the third device is not set, the third virtual address of the third device is applied, the third virtual address is mapped to the first physical address, and the first identifier of the third device is set, and the first identifier indicates that the device has been mapped to the first physical address.

[0015] In the embodiment of the present application, in the case that it is determined that the third device processes data and the first identifier of the third device is not set, it is indicated that the virtual address of the third device has not been mapped to the first physical address, and therefore the third virtual address of the third device can be applied and the third virtual address is mapped to the first physical address, that is, in the case that the third device is a consumer device, whether the virtual address of the third device needs to be applied and mapped to the first physical address is determined by the identifier, and in this way, the accuracy and efficiency of applying and mapping the virtual address of the third device can be improved.

[0016] In a possible implementation, before the third virtual address of the third device is applied and the third virtual address is mapped to the first physical address, the method further includes:

[0017] The first set and the second set are obtained, the first set includes devices on which the first identifier is set, and the second set includes devices for processing data, the devices on which the first identifier is set include the first device and the second device, and the devices for processing data include the third device;

[0018] In the case that it is determined based on the first set and the second set that there is a first difference set, a virtual device of a device in the first difference set is applied, and the virtual device of the device in the first difference set is mapped to the first physical address, the device in the first difference set is a device in the second set and is different from any device in the first set;

[0019] The third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address, including:

[0020] In the case that the first difference set includes the third device, the third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address.

[0021] In the embodiment of the present application, the device in the first difference set is a determined consumer device and is not marked with the first identifier, that is, the device in the first difference set needs to apply a virtual address and map the virtual address to the first physical address. If the first difference set includes the third device, it is indicated that the third device needs to apply a virtual address and map the virtual address to the first physical address, and therefore the third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address.

[0022] In a possible implementation, the electronic device further includes a first service, and the first application applies for memory, including:

[0023] The first application applies for memory from the first service;

[0024] The first virtual address of the first device and the second virtual address of the second device are applied, and the first memory in the physical memory is allocated, and the first virtual address and the second virtual address are respectively mapped to the first physical address of the first memory, comprising:

[0025] The first service applies the first virtual address of the first device and the second virtual address of the second device, and allocates the first memory in the physical memory, and maps the first virtual address and the second virtual address to the first physical address of the first memory.

[0026] In the embodiment of the application, the first application can apply the memory to the first service, and the first service can apply the first virtual address of the first device and the second virtual address of the second device, and allocate the first memory in the physical memory, and map the first virtual address and the second virtual address to the first physical address of the first memory. In this way, the problem of low efficiency and high resource occupation caused by the interaction of multiple services to realize address mapping can be reduced, thereby the efficiency of address mapping can be improved and the resource occupation can be reduced.

[0027] In a possible implementation, the third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address, comprising:

[0028] The first service applies the third virtual address of the third device, and maps the third virtual address to the first physical address.

[0029] In the embodiment of the application, the first service can also apply the virtual address of the third device and do mapping, which can reduce the problem of low efficiency and high resource occupation caused by the interaction of multiple services to realize address mapping, thereby the efficiency of address mapping can be improved and the resource occupation can be reduced.

[0030] In a possible implementation, the first application applies the memory, comprising:

[0031] The first application applies the graphics memory in response to the opening of the window of the electronic device, and the graphics memory is used to store the graphics data required by the window.

[0032] In the embodiment of the application, the first application applies the graphics memory in response to the opening of the window of the electronic device, which can improve the number of windows opened by the electronic device.

[0033] In a possible implementation, the first memory is also used to store the second data written by the second device, that is, the second device as a producer device writes data to the first memory.

[0034] In another possible implementation, the second virtual address of the second device is applied, comprising:

[0035] In a case where it is determined that the data is processed by the second device and the first identification of the second device is not set, a second virtual address of the second device is applied, and the first identification of the second device is set, the first identification indicating that the device is mapped to the first physical address.

[0036] In the embodiment of the application, the second device is a consumer device. If it is determined that the data needs to be processed by the second device and the first identification of the second device is not set, it is indicated that the virtual address of the second device is not mapped to the first physical address, and the second device cannot normally access the first memory. Therefore, the second virtual address of the second device is applied, so that the second virtual address is mapped to the first physical address, and the second device can normally access the first memory to obtain the first data for processing.

[0037] In a second aspect, the embodiment of the application provides an address mapping device, including a processor coupled with a memory, and used for executing instructions in the memory to implement the method in any possible implementation manner in the first aspect. Optionally, the device further includes the memory. Optionally, the device further includes a communication interface, and the processor is coupled with the communication interface.

[0038] In a third aspect, the embodiment of the application provides a processor including an input circuit, an output circuit and a processing circuit. The processing circuit is used for receiving a signal through the input circuit and transmitting a signal through the output circuit, so that the processor executes the method in any possible implementation manner in the first aspect.

[0039] In the specific implementation process, the processor can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a flip-flop and various logic circuits and the like. The input signal received by the input circuit can be received and input by, for example but not limited to, a receiver, the signal output by the output circuit can be output to and transmitted by, for example but not limited to, a transmitter, and the input circuit and the output circuit can be the same circuit, which is used as the input circuit and the output circuit at different times. The embodiment of the application does not limit the specific implementation manner of the processor and various circuits.

[0040] In a fourth aspect, an address mapping device is provided, including a processor and a memory. The processor is used for reading instructions stored in the memory, and can receive a signal through a receiver and transmit a signal through a transmitter to execute the method in any possible implementation manner in the first aspect.

[0041] Optionally, the processor is one or more, and the memory is one or more.

[0042] Optionally, the memory can be integrated with the processor, or the memory and the processor are separately arranged.

[0043] In the implementation process, the memory can be a non-transitory memory, for example, a read only memory (ROM), which can be integrated on the same chip as the processor, or can be separately arranged on different chips. The embodiments of the present application do not limit the type of memory and the arrangement mode of the memory and the processor.

[0044] It should be understood that the related data interaction process, such as sending indication information, can be a process of outputting indication information from the processor, and receiving capability information can be a process of receiving input capability information by the processor. Specifically, the processed output data can be output to the transmitter, and the input data received by the processor can come from the receiver. Wherein, the transmitter and the receiver can be collectively referred to as a transceiver.

[0045] The address mapping device in the fourth aspect described above can be a chip, and the processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading software code stored in a memory. The memory can be integrated in the processor or exist independently outside the processor.

[0046] In a fifth aspect, a computer program product is provided, which includes a computer program (also referred to as code or instructions), which, when executed, causes a computer to perform the method in any possible implementation manner of the first aspect.

[0047] In a sixth aspect, a computer readable storage medium is provided, which stores a computer program (also referred to as code or instructions), which, when executed on a computer, causes the computer to perform the method in any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0048] FIG. 1 is a schematic diagram of a scenario of data processing by multiple chips according to an embodiment of the present application;

[0049] FIG. 2 is a schematic diagram of a limited memory size of an electronic device according to the related art;

[0050] FIG. 3 is a schematic diagram of a hardware architecture of an electronic device according to an embodiment of the present application;

[0051] FIG. 4 is a schematic diagram of a software architecture of an electronic device according to an embodiment of the present application;

[0052] FIG. 5 is a schematic diagram of a system architecture of data processing according to an embodiment of the present application;

[0053] Figure 6 is a flow diagram of a method for dynamically mapping shared physical memory to multiple device virtual addresses according to an embodiment of the present application;

[0054] Figure 7 is a flow diagram of another method for dynamically mapping shared physical memory to multiple device virtual addresses according to an embodiment of the present application;

[0055] Figure 8 is a diagram of a physical memory usage according to an embodiment of the present application;

[0056] Figure 9 is a flow diagram of a method for address mapping according to an embodiment of the present application;

[0057] Figure 10 is a diagram of a framework of an address mapping device according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions of the present application more clear and intuitive, the following will combine the drawings and embodiments to make a detailed description of the embodiments of the present application through address mapping methods and devices. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0059] Before introducing the method and device provided by the embodiments of the present application, the following points will be explained.

[0060] First, in the embodiments shown below, each term and English abbreviation, such as CPU, etc., are exemplary examples given for convenience of description, and should not constitute any limitation on the present application. The present application does not exclude the possibility of defining other terms capable of achieving the same or similar functions in existing or future protocols.

[0061] Second, in the embodiments shown below, the first, second, and various numerical numbers are only for differentiation for convenience of description, and do not limit the scope of the embodiments of the present application.

[0062] Third, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean: A alone, A and B together, B alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, and c, can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0063] The technical solutions in the present application will be described below with reference to the drawings.

[0064] The technical solutions of the embodiments of the present application can be applied to various communication systems, such as a long term evolution (LTE) system, an LTE frequency division duplex (FDD) system, an LTE time division duplex (TDD), a universal mobile telecommunication system (UMTS), a 5th generation (5G) system or a new radio (NR), or other evolved communication systems, etc.

[0065] The terminal device in the embodiments of the present application can also be referred to as a user equipment (UE), a mobile station (MS), a mobile terminal (MT), an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile terminal, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a device, a wireless communication device, an electronic device, a user agent, or a user equipment, etc.

[0066] The terminal device can be a device that provides voice / data connectivity to a user, for example, a handheld device with wireless connection function, a vehicle-mounted device, etc. At present, some examples of terminals are: mobile phones, car radios, tablet computers, notebook computers, palm computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication functions, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc. The embodiments of the present application are not limited thereto.

[0067] By way of example and not limitation, in the embodiments of the present application, the terminal device can also be a wearable device. The wearable device can also be referred to as a wearable smart device, which is a general term for devices that are designed and developed by applying wearable technology to daily wear, such as glasses, gloves, watches, clothing, and shoes. The wearable device is a portable device that is directly worn on the body or integrated into the user's clothes or accessories. The wearable device is not only a hardware device, but also a powerful function realized through software support and data interaction, cloud interaction. The general wearable smart device includes a full function, large size, and can realize complete or partial functions without relying on a smart phone, such as smart watches or smart glasses, etc., and focuses on a certain application function and needs to cooperate with other devices such as smart phones, such as various smart wristbands, smart jewelry, etc. for monitoring vital signs.

[0068] In addition, in the embodiments of the present application, the terminal device can also be a terminal device in an internet of things (IoT) system. The IoT is an important part of future information technology development, and its main technical feature is to connect objects through communication technology and network, so as to realize the intelligent network of man-machine interconnection and object-object interconnection. The terminal device of the present application can also be a vehicle-mounted unit, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit built in as one or more components or units in a vehicle. The vehicle can implement the method of the present application through the built-in vehicle-mounted unit, vehicle-mounted module, vehicle-mounted component, vehicle-mounted chip or vehicle-mounted unit. Therefore, the embodiments of the present application can be applied to the Internet of Vehicles, such as vehicle to everything (V2X), long term evolution-vehicle (LTE-V), vehicle-to-vehicle (V2V), etc.

[0069] At present, although the performance of part of the hardware of the electronic device is getting stronger, for example, the bit width of the hardware such as the GPU and the central processing unit (CPU) of the electronic device is also getting larger, but the overall performance of the electronic device is not greatly improved. In a possible case, the reason why the overall performance of the electronic device is not greatly improved includes that the lowest bit width in the plurality of hardware will become a bottleneck, causing a barrel effect, affecting the overall available memory size, and leading to the overall performance of the electronic device not being greatly improved. For example, the number of web pages that can be opened by the browser is limited, and if the number of web pages opened by the browser is too much, the browser will crash or be prohibited from opening new web pages.

[0070] Next, taking the scene of using CPU, GPU and display subsystem (DSS) chips to process data as an example, the reason why the lowest bit width will become a bottleneck and affect the overall available memory size is explained.

[0071] Please refer to FIG. 1, which is a scene diagram of a plurality of chips processing data provided by an embodiment of the present application. As shown in FIG. 1, the scene can include CPU, GPU, DSS and physical memory. The bit width of CPU and GPU can be 64, and the bit width of DSS can be 32 bits. The physical memory can include a buffer. The buffer can exist in a buffer area on a random access memory (RAM) or a hard disk.

[0072] In the embodiments of the present application, the buffer needs to be accessed by the CPU, GPU and DSS, and the memory of the buffer is physically shared, so that the CPU, GPU and DSS can transmit data through the buffer. Taking data processing including image processing as an example, the original layer drawing can be completed by the CPU or GPU, which can also be understood as producing data by the CPU or GPU; and the image to be displayed can be synthesized by the GPU or DSS, which can also be understood as consuming data by the GPU or DSS. In the embodiments of the present application, the CPU can draw layers, the GPU can flush and compose images. Flush generally refers to the operation of flushing buffer data from cache or temporary memory to permanent memory or external device. Compose is generally related to the creation, combination or synthesis of images, and it can refer to combining multiple image elements, layers or image segments into a complete image.

[0073] Optionally, the scenarios of image processing by the CPU, GPU and DSS include but are not limited to the following scenarios:

[0074] In the data production phase, the CPU and / or GPU complete the original layer drawing and write the drawn data into the buffer, and then in the data consumption phase, the GPU and / or DSS perform subsequent data processing, such as writing the drawn data into a read-only memory (ROM) or an external device, or synthesizing an image to be displayed using the drawn data.

[0075] In the embodiments of the present application, the CPU completing the original layer drawing can be understood as producing data, and the GPU performing rendering and other image enhancement processing on the drawn data can be understood as consuming data. In addition, the GPU performing rendering and other image enhancement processing on the drawn data can also be understood as producing data, and the DSS synthesizing an image to be displayed using the data processed by the image enhancement processing can be understood as consuming data.

[0076] It should be understood that the scenario shown in FIG. 1 can also be applied to other data processing, such as training or inference of an artificial intelligence model, and the specific data processing is not limited herein. The training or inference of the artificial intelligence model can refer to the process of image processing by the CPU, GPU and DSS, and the difference between the training or inference of the artificial intelligence model and the process of image processing includes different chips and different processes.

[0077] In the related art, when data processing is needed, a virtual address is first applied for in the process of an application program for drawing a layer, and the application program can also be referred to as an application or a program. When the program needs to read or write the content of the address, a memory page is allocated from the physical memory by the system, and the virtual address is mapped to the physical address.

[0078] Taking the above scenario as an example, a virtual address needs to be first applied for in the CPU, GPU and DSS respectively, and then the virtual addresses applied for by the CPU, GPU and DSS are mapped to the same physical address, so as to realize the shared processing among the CPU, GPU and DSS, which can also be understood as realizing the processing of the production data stage and the consumption data stage among the CPU, GPU and DSS. The virtual address range is affected by the bit width n of the chip. For example, the maximum range of the virtual address is 2 n However, the physical address range is related to the capacity of the physical memory.

[0079] Therefore, the virtual address is first applied for in the CPU, GPU and DSS respectively, and then the virtual addresses applied for by the CPU, GPU and DSS are mapped to the same physical address, so as to realize the shared processing among the CPU, GPU and DSS. In this way, the range of the physical address that can be used is affected by the lowest bit width among the CPU, GPU and DSS, that is, the lowest bit width among the CPU, GPU and DSS will become a bottleneck, causing the bucket effect and affecting the overall available memory size. Taking the number of opened web pages of a browser as an example, since the image elements need to be displayed when the web pages are opened, the physical memory needs to be applied for to store the image elements. However, when the overall available memory size of the physical address is limited, the number of opened web pages of the browser will be affected.

[0080] Please refer to FIG. 2, which is a schematic diagram of the limited overall available memory size of an electronic device in the related art. As shown in FIG. 2, since the bit width of the CPU and the GPU is 64 bits, the maximum memory size that can be theoretically used by the CPU and the GPU is 2 64 = 8 gigabytes (G), and the maximum memory size that can be theoretically used by the DSS is 2 32= 4G. Even if the physical memory RAM of the electronic device is 16G or even 32G, and the bit width of the CPU and GPU is 64 bits, due to the limitation of the bit width of the DSS, the CPU, GPU and DSS can only use 4G memory, and the excess memory cannot be used as graphics memory. For example, for 16G of RAM, only 4G of memory can be shared by the CPU, GPU and DSS as graphics memory, and the remaining 12G of memory cannot be used in graphics. In this way, the number of applications that can be opened simultaneously by the electronic device and the performance of the electronic device are greatly limited. Therefore, even if the performance of part of the hardware of the electronic device is getting stronger, the overall performance of the electronic device is not greatly improved.

[0081] In general, multiple chips share physical memory, and the total amount of available memory is limited by the chip with the smallest bit width. For example, a 32-bit chip limits 4G of physical memory, which results in more than 4G of physical memory being unable to be used, which is a waste. For 16G of physical memory, the utilization rate is only 4G / 16G = 25%, and for 32G of physical memory, the utilization rate is only 4G / 32G = 12.5%.

[0082] Therefore, the embodiments of the present application provide an address mapping method and device, which can improve the overall performance of the electronic device by dynamically mapping the shared physical memory to the virtual address of each chip. Taking the number of web pages that can be opened by the browser as an example, the shared memory of the chip that needs to consume data with the smallest bit width is limited by the smallest bit width, but the shared memory of the chip that does not need to consume data is not limited by the smallest bit width. Therefore, the number of new applications that can be opened by the entire machine is multiplied, and more new web pages or applications can be opened as long as there is physical memory.

[0083] The address mapping method will be described below in combination with the hardware architecture and software architecture of the electronic device.

[0084] Please refer to FIG. 3, which is a hardware architecture diagram of an electronic device provided by an embodiment of the present application.

[0085] As shown in FIG. 3, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyro sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0086] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a CPU, a GPU, a DSS, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0088] The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0089] The processor 110 can also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has recently used or has used frequently. If the processor 110 needs to use the instructions or data again, it can be retrieved directly from the memory. This avoids repeated access and reduces the latency of the processor 110, thereby improving the efficiency of the system.

[0090] The electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0091] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. The data storage area can store data created during use of the electronic device 100 (such as audio data, image data, a phonebook, etc.), and the like. In addition, the internal memory 121 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like. The processor 110 executes various function applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory disposed in the processor.

[0092] In the embodiments of the present application, taking the data processing including image data (which can also be referred to as graphics data) processing as an example, the drawing of the layer can be completed through the CPU or the GPU, and the drawn image data is stored in the data storage area. Then, the GPU or the DSS obtains the drawn image data from the data storage area for image synthesis, and then the synthesized image data is displayed through the display screen 194. Taking the training or inference of an artificial intelligence model as an example, the CPU can obtain operation data of a user, and store the operation data in the data storage area, and then the NPU obtains the operation data from the data storage area for training or inference. The operation data can be data generated by detecting the operation of the user on the electronic device, for example, data of an application program clicked by the user detected by the electronic device.

[0093] Please refer to FIG. 4, which is a software architecture diagram of an electronic device provided in an embodiment of the present application.

[0094] The software system of the electronic device 100 can employ a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Embodiments of the present application exemplarily illustrate the software structure of the electronic device 100 in a layered architecture of the Harmony system.

[0095] The Harmony system employs a multi-kernel design, which can optionally include a Linux kernel, a Harmony micro-kernel, and LiteOS. With this design, devices with different device capabilities can select appropriate system kernels. The kernel layer further includes a kernel abstract layer that provides basic kernel capabilities to other Harmony layers, such as process management, thread management, memory management, file system management, network management, and peripheral management. Optionally, the kernel layer further includes a rendering service, which is generally responsible for converting user interface (UI) elements or graphical data into images that can be displayed on a screen. This includes handling visual effects such as colors, textures, lighting, and shadows, as well as ensuring that graphical elements are drawn to the screen in the correct manner and order to achieve smooth animations and interactive experiences. It should be understood that the rendering service can also be configured at other levels of the software framework, and is not limited to the kernel layer, as long as the rendering service can achieve its functions in the embodiments of the present application.

[0096] The system foundation service layer is a core capability set of the Harmony system, which supports the Harmony system to provide services to application services through the framework layer in the scenario of multi-device deployment. This layer can optionally include the following parts:

[0097] System basic capability subsystem set: provides basic capabilities for distributed applications to run, schedule, migrate, and other operations on multiple devices of the Harmony system, which consists of a distributed software bus, distributed data management and file management, distributed task scheduling, Ark runtime, distributed security and privacy protection, and the like. Among them, the Ark runtime provides a C / C++ / JavaScript multi-language runtime and basic system class library, and also provides a runtime for Java programs (i.e., application programs or parts developed using Java language in the framework layer) that are statically compiled using the Ark compiler.

[0098] Basic software service subsystem set: provides common and general software services for the Harmony system, which consists of graphical images, distributed media, distributed AI, multi-modal input, MSDP&DV, event notification, telephone service, distributed DFX, and the like. The basic software service subsystem set can be trimmed according to the deployment environment of different device forms, and each subsystem can be trimmed according to the functional granularity.

[0099] Enhanced software service subsystem set: provides differentiated capability-enhanced software services for different devices for the Harmony system, composed of tablet business software, smart screen business software, car machine business software, IoT business software, and the like. The enhanced software service subsystem set can be tailored according to the deployment environment of different device forms, at the granularity of a subsystem, and each subsystem can be tailored at the granularity of a function.

[0100] Harmony Driver Framework (HDF) and Hardware Abstraction Adaptation Layer (HAL): is the basis for the hardware ecological openness of the Harmony system, providing hardware capability abstraction to hardware upwards and providing a development framework and a running environment for various peripheral drivers downwards.

[0101] Hardware service subsystem set: provides common and adapted hardware services for the Harmony system, composed of generic Sensor, location, power, USB, biometric identification, and the like. The hardware service subsystem set can be tailored according to the deployment environment of different device forms, and each subsystem can be tailored at the granularity of a function.

[0102] Special hardware service subsystem: provides differentiated hardware services for different devices for the Harmony system, and can optionally include tablet-specific hardware services, car machine-specific hardware services, wearable-specific hardware services, IoT-specific hardware services, and the like. The special hardware service subsystem can be tailored at the granularity of a subsystem, and each subsystem can be tailored at the granularity of a function.

[0103] The framework layer provides a user program framework and a meta-capability framework in multiple languages such as Java / C / C++ / JavaScript for applications of the Harmony system, and a multi-language framework API exposed by various software and hardware services. The application layer includes system applications and third-party applications, which can include a browser, a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, and the like. Applications in the Harmony system are constructed based on AA and FA.

[0104] In the embodiments of the present application, a physical memory is first allocated to a producer device (such as a CPU or a GPU), and the producer device can write produced data (such as rendered image data) into the physical memory. Then, when the produced data needs to be consumed, a consumer device (such as a GPU or a DSS) for consuming the data can acquire the produced data from a physical address where the produced data is written by using a mapping between a virtual address of the consumer device and the physical address, and consume the produced data. The consumption of the data may, for example, be the synthesis of an image to be displayed by using image data.

[0105] The producer device can be a device that produces data, and the consumer device can be a device that consumes data. The definitions of the producer data and the consumer data can refer to the descriptions of the above embodiments, which are not repeated here. For example, the producer device can be a CPU or a GPU, and the consumer device can be a GPU or a DSS.

[0106] To better understand the data processing scheme of the embodiments of the present application, the following embodiments take the producer device, the consumer device, and the graphics memory management decision device as examples of several types of functional units in an electronic device, and respectively illustrate the cases of occupying physical addresses in the producer data stage and the consumer data stage. The graphics memory management decision device can be used to manage graphics memory, for example, to manage each independent graphics memory.

[0107] Referring to FIG. 5, FIG. 5 is a schematic diagram of a system architecture for data processing provided by an embodiment of the present application. As shown in the system architecture of FIG. 5, the system architecture can include a producer device, a consumer device, a graphics memory management decision device, and a physical memory device. The physical memory device is also referred to as a physical memory. Optionally, the producer device can be one or more independent devices. When the producer device is multiple, it can be 1-M independent producer devices, where M is an integer greater than or equal to 2. Optionally, the consumer device can be one or more independent devices. When the consumer device is multiple, it can be 1-N independent consumer devices. In the embodiments of the present application, the producer device and the consumer device can be hardware devices, and the graphics memory management decision device can be a software device. Taking graphics processing as an example, the producer device can be used to draw and write graphics data, and the consumer device can be used to synthesize an image or read graphics data for display.

[0108] In the producer data stage, the producer device applies for a virtual address of the producer device. Then, the graphics memory management decision device allocates shared physical memory in the physical memory device for the producer device, and marks the producer device with an identifier (also referred to as a flag bit), which is used to indicate the physical address of the shared physical memory allocated for the producer device. That is, the identifier is used to indicate that the virtual address of the producer device has been mapped to a physical address, or in other words, the identifier is used to indicate that the producer device has been mapped to a physical address. Moreover, the graphics memory management decision device maps the virtual address of the producer device to the physical address, so that the producer device can use the mapping between the virtual address of the producer device and the physical address to write the data produced by the producer device into the memory corresponding to the physical address.

[0109] Then, in the data consumption stage, in the embodiments of the present application, the graphics memory management decision device can perform, but is not limited to, the following steps:

[0110] 1. Find all consumer devices of a certain piece of memory according to business logic.

[0111] 2. Find the difference set between the set of device identifiers that have been stamped on the piece of memory and the consumer devices, and apply for a virtual address to the consumer devices in the difference set.

[0112] 3. Bind the newly applied virtual address to the original physical memory.

[0113] In the embodiment of the present application, the difference set includes consumer devices that have not been stamped with an identifier. For example, the graphics memory management decision device decides a set of consumer devices for consuming production data according to business logic, and the set of consumer devices includes one or more consumer devices, for example, the set of consumer devices includes n consumer devices, 0 < n < N, and n is an integer. Then, the set of consumer devices is compared with the set of producer devices that have been stamped with an identifier, and it is determined whether there is a difference set. If there is a difference set, it means that at least one consumer device in the set of consumer devices has not been stamped with an identifier, that is, at least one consumer device in the set of consumer devices has not applied for a virtual address mapped to the physical address where the production data is stored. In other words, at most part of the consumer devices in the set of consumer devices have been stamped with an identifier before the data consumption stage, that is, at most part of the consumer devices in the set of consumer devices have applied for a virtual address mapped to the physical address where the production data is stored.

[0114] Therefore, the graphics memory management decision device can apply for a virtual address for each of the at least one consumer device, and then the graphics memory management decision device maps the virtual address applied for by each of the at least one consumer device to the physical address, so that each of the at least one consumer device can obtain the production data from the memory corresponding to the physical address through the mapping between the applied virtual address and the physical address, and consume the production data. For the consumer devices that have been stamped with an identifier, these consumer devices can obtain the production data from the memory corresponding to the physical address through the mapping between the applied virtual address and the physical address, and consume the production data.

[0115] It should be noted that if there is no difference set, it means that each consumer device in the set of consumer devices has applied for a virtual address mapped to the physical address before the data consumption, and each consumer device in the set of consumer devices can obtain the production data from the memory corresponding to the physical address through the mapping between the applied virtual address and the physical address, and consume the production data.

[0116] In general, the embodiments of the present application divide the memory allocation operation into a production phase and a consumption phase, and each phase dynamically maps the virtual address for the related chip on demand, so that the chips do not affect each other. In this way, the whole machine can use the memory and is no longer limited by the bit width of a single chip. The theoretical maximum value is the sum of the virtual spaces of all chips, which is greater than the capacity of the physical address, that is, the utilization rate of the physical memory is improved as much as possible.

[0117] For example, the data processing includes image data processing. It is assumed that the system framework can include a CPU, a GPU, and a DSS, wherein the GPU can be both a producer device and a consumer device, the CPU is a producer device, and the DSS is a consumer device. In the production data phase, the CPU and the GPU apply for virtual addresses respectively, it is assumed that the virtual address applied for by the CPU is virtual address 1, the virtual address applied for by the GPU is virtual address 2, then the graphic memory management decision module allocates memory 1 for the CPU and the GPU, maps the virtual address 1 and the virtual address 2 to the physical address 1 of the memory 1 respectively, and marks the CPU and the GPU with an identifier, which is used to indicate the physical address 1. At this time, after the CPU produces data, the CPU can write the produced data into the memory 1 corresponding to the physical address 1 by using the mapping between the virtual address 1 and the physical address 1. Then, in the data consumption phase, the graphic memory management decision device decides a consumer device set for consuming the produced data according to the business logic, and the consumer device set includes the GPU and the DSS. Thus, the consumer device set is compared with the marked producer device set, and a difference set is obtained, which includes the DSS. Then, for the DSS, the graphic memory management decision device can apply for a virtual address 3, and then the graphic memory management decision device maps the virtual address 3 to the physical address 1, so that the DSS can obtain the produced data from the memory 1 corresponding to the physical address 1 by using the mapping between the virtual address 3 and the physical address 1, and consume the produced data. For the GPU, since the GPU is marked in the production data phase, that is, the virtual address 2 applied for by the GPU is mapped to the physical address 1 in the production data phase, in the data consumption phase, the GPU can obtain the produced data from the memory 1 corresponding to the physical address 1 by using the mapping between the virtual address 2 and the physical address 1, and consume the produced data.

[0118] It should be noted that if the decided consumer device set includes the GPU but does not include the DSS, then there is no difference set when the consumer device set is compared with the marked producer device set. If the decided consumer device set includes the DSS but does not include the GPU, then there is a difference set when the consumer device set is compared with the marked producer device set.

[0119] In the embodiment of the present application, in the data production stage, the CPU and the GPU respectively apply for virtual addresses, and respectively map the virtual addresses of the CPU and the GPU to the physical addresses of the allocated memory. In the data consumption stage, if the data needs to be consumed by the GPU, since the GPU has applied for the virtual addresses and mapped to the physical addresses, the GPU can no longer apply for and map the virtual addresses in the data consumption stage, thereby improving the efficiency of data consumption. That is, in the embodiment of the present application, by applying for the virtual addresses of the plurality of devices respectively in the data production stage, and mapping the virtual addresses of the plurality of devices respectively to the physical addresses, if the device which has applied for the virtual addresses and mapped to the physical addresses needs to consume data, in the data consumption stage, the device which has applied for the virtual addresses and mapped to the physical addresses can directly obtain the data for consumption by using the mapping between the virtual addresses and the physical addresses, thereby improving the efficiency of data consumption.

[0120] It should be noted that in the embodiment of the present application, the devices involved in data processing include at least three, and the bit width of at least two of the at least three devices is greater than the bit width of the other devices except the at least two devices. For example, the devices involved in image processing include a CPU, a GPU and a DSS, wherein the bit width of the CPU and the GPU is greater than the bit width of the DSS, but the bit width of the CPU and the GPU can be the same or different, which is determined according to the actual situation, for example, the bit width of the CPU is greater than the bit width of the GPU, or the bit width of the CPU is equal to the bit width of the GPU, or for example, the bit width of the CPU is less than the bit width of the GPU.

[0121] In the embodiment of the present application, in the data production stage, in addition to the producer device needing to apply for the virtual addresses and map to the physical addresses, for at least one device with the maximum bit width among the at least three devices, the virtual addresses can also be applied for, and then the virtual addresses of the at least one device with the maximum bit width are respectively mapped to the allocated physical addresses, and the range of the physical addresses supported by the device with the maximum bit width is greater than or equal to the range of the physical addresses supported by the devices with other bit widths, that is, the range of the physical addresses supported by the device with the maximum bit width can cover the range of the physical addresses supported by the devices with other bit widths, so that not only the efficiency of data consumption can be improved, but also the situation that the physical addresses storing the production data of the device with a larger bit width cannot be accessed by the device with a smaller bit width due to the smaller bit width of the device applying for the virtual addresses in the data production stage can be reduced, thereby reducing the situation that the data cannot be consumed due to the smaller bit width device cannot access the production data.

[0122] For example, assuming that the bit width of the CPU and the GPU is 64 bits, and the bit width of the DSS is 32 bits, since the bit width of the CPU and the GPU is the maximum, in the production data stage, the CPU and / or the GPU can apply for a virtual address, and the applied virtual address is mapped to a physical address. For another example, assuming that the bit width of the CPU is 128 bits, the bit width of the GPU is 64 bits, and the bit width of the DSS is 32 bits, since the bit width of the CPU is the maximum, in the production data stage, the CPU can apply for a virtual address, and the applied virtual address is mapped to a physical address; at this time, if the GPU is a producer device, the GPU can also apply for a virtual address and map it to a physical address; if the GPU is not a producer device, the virtual address of the GPU can not be applied first. For another example, assuming that the bit width of the CPU is 64 bits, the bit width of the GPU is 128 bits, and the bit width of the DSS is 32 bits, since the bit width of the GPU is the maximum, in the production data stage, the GPU can apply for a virtual address, and the applied virtual address is mapped to a physical address; at this time, if the CPU is a producer device, the CPU can also apply for a virtual address and map it to a physical address; if the CPU is not a producer device, the virtual address of the CPU can not be applied first. For another example, assuming that the bit width of the CPU is 32 bits, and the bit width of the GPU and the DSS is 64 bits, since the bit width of the GPU and the DSS is the maximum, in the production data stage, the GPU and / or the DSS can apply for a virtual address, and the applied virtual address is mapped to a physical address. For another example, assuming that the bit width of the CPU is 64 bits, the bit width of the CPU is 32 bits, and the bit width of the DSS is 128 bits, in the production data stage, the DSS can apply for a virtual address, and the applied virtual address is mapped to a physical address.

[0123] It should be understood that, if in the production data stage, the producer device is not the same device as the device with the maximum bit width, since the producer device needs to store the produced data, the producer device needs to apply for a virtual address and map it to a physical address; and for the device with the maximum bit width, the virtual address can also be applied in the production data stage and mapped to the physical address where the produced data is stored, so that when the device with the maximum bit width is used as a consumer device, the produced data can be acquired in time and consumed. Alternatively, the device with the maximum bit width can also apply for a virtual address and map it to the physical address where the produced data is stored only when it is determined to be a producer device or a consumer device.

[0124] In another possible implementation, the virtual address can also be applied for by the devices other than the device with the minimum bit width among the at least three devices, and then the virtual addresses of the devices other than the device with the minimum bit width are respectively mapped to the allocated physical addresses, so that when the devices other than the device with the minimum bit width need to consume data, the data can be consumed in time, thereby improving the efficiency of consuming data.

[0125] For example, assuming that the bit width of the CPU is 128 bits, the bit width of the GPU is 64 bits, and the bit width of the DSS is 32 bits, the CPU and / or the GPU can apply for a virtual address, and the applied virtual address is mapped to a physical address. For another example, assuming that the bit width of the CPU is 64 bits, the bit width of the GPU is 128 bits, and the bit width of the DSS is 32 bits, the CPU and / or the GPU can apply for a virtual address, and the applied virtual address is mapped to a physical address. For yet another example, assuming that the bit width of the CPU is 64 bits, the bit width of the GPU is 32 bits, and the bit width of the DSS is 128 bits, the CPU and / or the DSS can apply for a virtual address, and the applied virtual address is mapped to a physical address.

[0126] In another possible implementation, the virtual address can be applied for and mapped to a physical address only when the device needs to process data, so that the situation that a virtual address is applied for and mapped to a physical address but is not used can be reduced, thereby improving the utilization of device resources.

[0127] It should be noted that, in the embodiment of the present application, the producer devices can be marked with an identifier in the production data stage. For example, the M producer devices can be marked with an identifier, or part of the M producer devices can be marked with an identifier. The part of the M producer devices can be, for example, the producer devices that actually participate in data processing in the current round of data processing, such as the producer devices that actually participate in drawing layer data.

[0128] Next, the method for dynamically mapping a shared physical memory to a virtual address of a plurality of devices will be described. The following embodiment takes the plurality of devices as an example, which includes a CPU, a GPU, and a DSS. In the embodiment of the present application, the CPU and the GPU can be producer devices, and the GPU and the DSS can be consumer devices, that is, the GPU can be both a producer device and a consumer device. Optionally, the bit width of the CPU is greater than the bit width of the DSS, and the bit width of the GPU is greater than the bit width of the DSS. The bit width of the CPU and the bit width of the GPU can be the same or different. For example, the bit width of the CPU and the bit width of the GPU are both 64 bits, and the bit width of the DSS is 32 bits. For another example, the bit width of the CPU is 256 bits, the bit width of the GPU is 128 bits, and the bit width of the DSS is 64 bits.

[0129] FIG. 6 is a flowchart of a method for dynamically mapping a shared physical memory to a virtual address of a plurality of devices according to an embodiment of the present application. The method of the embodiment can include the following steps.

[0130] S611, the application program applies to the rendering service for a graphics memory.

[0131] The graphics memory can be a space for storing graphics data, and the graphics memory corresponds to a physical address. In the embodiments of the present application, the application program can be an application program capable of generating graphics data, such as a camera application, a browser application, and the like, which is not limited herein.

[0132] In S612, the rendering service applies for a virtual address to the CPU and the GPU respectively, and allocates the memory 1, and maps the virtual address 1 applied for by the CPU and the virtual address 2 applied for by the GPU to the physical address 1 of the memory 1 respectively, and sets the flag bits of the CPU and the GPU.

[0133] In the embodiments of the present application, the rendering service plays a role of a graphics memory management decision device. It should be understood that the function of the graphics memory management decision device can also be implemented by other services or modules, and is not limited to the rendering service. Alternatively, the rendering service can transmit a virtual address application request to the CPU and the GPU respectively, and then the CPU applies for the virtual address 1 after receiving the virtual address application request, and the CPU transmits the applied virtual address 1 to the rendering service, and then the rendering service can map the virtual address 1 to the physical address 1 to obtain a mapping relationship 1, which maps the relationship between the virtual address 1 and the physical address 1, and then the rendering service transmits the mapping relationship 1 to the CPU. In addition, the GPU applies for the virtual address 2 after receiving the virtual address application request, and the GPU transmits the applied virtual address 2 to the rendering service, and then the rendering service can map the virtual address 2 to the physical address 1 to obtain a mapping relationship 2, which maps the relationship between the virtual address 2 and the physical address 1, and then the rendering service transmits the mapping relationship 2 to the GPU. In addition, the rendering service also sets the flag bits of the CPU and the GPU, which record that the CPU and the GPU have both allocated the physical address 1 and have made the mapping.

[0134] It should be noted that in the embodiments of the present application, the application program and the rendering service can be software processes running on the CPU. The CPU running the application program and the service rendering, and the CPU writing the layer data to the memory 1 corresponding to the physical address 1 can be different CPUs, that is, the CPU running the application program and the service rendering is CPU1, and the CPU writing the layer data to the memory 1 corresponding to the physical address 1 is CPU2.

[0135] In S613, the CPU writes the layer data generated by the application program to the memory 1 based on the mapping between the virtual address 1 and the physical address 1.

[0136] In this embodiment, the layer data generated by the application can be transmitted to the CPU. The CPU can then write the layer data generated by the application into memory 1 corresponding to physical address 1, or the CPU can process the layer data generated by the application and then write it into memory 1 corresponding to physical address 1. It should be noted that the layer data can be drawn by the CPU running the application, or it can be drawn jointly by the CPU and GPU running the application; no limitation is imposed here. If the layer data is drawn jointly by the CPU and GPU running the application, the method in this embodiment may further include the GPU writing the layer data generated by the GPU into memory 1 corresponding to physical address 1 based on the mapping between virtual address 2 and physical address 1.

[0137] S614, The rendering service determines that layer data needs to be composited by the GPU.

[0138] In this embodiment, since the GPU flag has been set in S612, the rendering service can know that the virtual address requested by the GPU has been mapped to the physical address 1 through the GPU flag. In other words, the physical address 1 already has the GPU flag, so the rendering service does not need to request a virtual address from the GPU again.

[0139] It should be noted that if the GPU does not have a flag, then you need to request a virtual address from the GPU, and then map the virtual address 2 requested by the GPU to physical address 1.

[0140] S615, Rendering service instructs the GPU to composite images.

[0141] In this embodiment of the application, the rendering service may transmit an instruction message 1 to the GPU, which is used to instruct the GPU to synthesize an image.

[0142] The S616 GPU obtains layer data from memory 1 corresponding to physical address 1 based on the mapping between virtual address 2 and physical address 1, and synthesizes images based on the layer data.

[0143] S617. The rendering service determines that layer data needs to be composited via DSS.

[0144] S618, The rendering service requests a virtual address from the DSS, maps the virtual address 3 requested by the DSS to the physical address 1, and sets the flag bit of the DSS.

[0145] In the embodiment of the present application, since in S612, the flag bit of the DSS is not set, in other words, the DSS has not applied for the virtual address and mapped into the physical address 1, the rendering service applies for the virtual address to the DSS. Optionally, the embodiment of the present application sets the flag bit of the DSS, and then if the DSS needs to access the memory 1 corresponding to the physical address 1 again, it can be known from the flag bit of the DSS that the physical address 1 has the flag bit of the DSS, so that the DSS can be instructed to access the physical address 1 without applying for the virtual address to the DSS again and mapping the physical address 1.

[0146] It should be noted that if the DSS has the flag bit, it can not need to apply for the virtual address to the DSS again.

[0147] S619, the rendering service instructs the DSS to synthesize the image.

[0148] In the embodiment of the present application, the rendering service can transmit an instruction message 2 to the GPU, where the instruction message 2 is used to instruct the DSS to synthesize the image.

[0149] S620, the DSS obtains the layer data from the memory 1 corresponding to the physical address 1 based on the mapping between the virtual address 3 and the physical address 1, and synthesizes the image based on the layer data.

[0150] In the embodiment of the present application, S614-S616 and S617-S620 are two parallel branches, in other words, either the steps of S614-S616 or the steps of S617-S620 can be executed.

[0151] Optionally, the rendering service can select whether to synthesize the image by the GPU or by the DSS according to certain business logic. In a possible implementation, whether to synthesize the image by the GPU or by the DSS can be selected according to the balance logic of power consumption and performance. Optionally, it is assumed that the performance of the GPU is better than that of the DSS and the power consumption of the GPU is greater than that of the DSS, at this time, if the rendering service detects that the electronic device starts the power saving mode or the current power of the electronic device is lower than the power threshold, the rendering service can synthesize the image by the DSS; if the rendering service detects that the electronic device starts the performance mode or does not start the power saving mode, or the current power of the electronic device is higher than or equal to the power threshold, the rendering service can select to synthesize the image by the GPU. Optionally, the synthesis image priority of the GPU and the DSS can be set, and the image synthesis is first performed by the one with higher priority, and when the memory with higher priority is used up, the image synthesis is performed by the memory with lower priority.

[0152] It should be understood that the business logic of selecting the GPU or the CPU for image composition provided above is only an example, which can be set as needed, and is not limited herein.

[0153] It should be noted that S611-S613 can be understood as a production data stage, and S614-S620 can be understood as a consumption data stage.

[0154] In another possible implementation, if the GPU does not participate in the drawing of the layer data, in S612, the rendering service can apply for a virtual address from the CPU, allocate memory 1, and map the virtual address 1 applied for by the CPU to the physical address 1, and set the flag bit of the CPU. Then, if the image needs to be composed by the GPU, before S615, the rendering service applies for a virtual address from the GPU, and maps the virtual address 2 applied for by the GPU to the physical address 1.

[0155] In another possible implementation, the GPU or the DSS does not necessarily need to compose the image, but can also perform image enhancement processing on the layer data, which is not limited herein.

[0156] In the embodiment shown in FIG. 6, the CPU running the application and the service rendering, and the CPU writing the layer data to the physical address 1 can be different CPUs.

[0157] In the following embodiment, the CPU running the application and the service rendering, and the CPU writing the layer data to the memory 1 corresponding to the physical address 1 are the same CPU. In other words, the application and the service rendering are software processes running in the CPU writing the layer data to the memory 1 corresponding to the physical address 1.

[0158] Referring to FIG. 7, FIG. 7 is a flowchart of another method for sharing physical memory dynamically mapped to multiple device virtual addresses according to an embodiment of the present application. The method of the embodiment can include:

[0159] S711, the application applies for graphics memory from the rendering service.

[0160] S711 can refer to the description of S611, which is not repeated here.

[0161] S712, the rendering service applies for a virtual address 1 of the CPU and a virtual address 2 of the GPU, allocates memory 1, and maps the virtual address 1 and the virtual address 2 applied for by the GPU to the physical address 1 of the memory 1, respectively, and sets the flag bits of the CPU and the GPU.

[0162] In the embodiment of the present application, since the rendering service runs in the CPU writing the data, the rendering service can directly apply for the virtual address 1.

[0163] It should be understood that it is also possible to request the mapping between virtual and physical addresses through other services running on the CPU and then forward it to the rendering service.

[0164] S713, the rendering service writes the layer data generated by the application to memory 1 based on the mapping between virtual address 1 and physical address 1.

[0165] In this embodiment, since the rendering service runs on the CPU that writes data, it can directly write the layer data generated by the application to memory 1 corresponding to physical address 1. Optionally, the layer data generated by the application can be transmitted to the rendering service, and then written to memory 1 corresponding to physical address 1 by the rendering service.

[0166] It should be understood that the layer data generated by the application can also be written to memory 1 corresponding to physical address 1 through other services running on the CPU. In this case, the rendering service can transfer the layer data to other services, and then the other services can write the layer data to memory 1 corresponding to physical address 1.

[0167] S714, The rendering service determines that layer data needs to be composited via the GPU.

[0168] S714 can be referred to in the explanation of S614, and will not be repeated here.

[0169] S715, Rendering service instructs the GPU to composite images.

[0170] S715 can be referred to in the description of S615, and will not be repeated here.

[0171] The S716 GPU obtains layer data from memory 1 corresponding to physical address 1 based on the mapping between virtual address 2 and physical address 1, and synthesizes images based on the layer data.

[0172] S716 can be referred to in the description of S616, and will not be repeated here.

[0173] S717, The rendering service determines that layer data needs to be composited via DSS.

[0174] S717 can be referred to in the explanation of S617, and will not be repeated here.

[0175] S718 The rendering service requests a virtual address from the DSS, maps the virtual address 3 requested by the DSS to the physical address 1, and sets the flag bit of the DSS.

[0176] S718 can be referred to in the explanation of S618, and will not be repeated here.

[0177] S719, the rendering service instructs the DSS to compose an image.

[0178] S719 can refer to the description of S619, and details are not described herein.

[0179] S720, the DSS obtains layer data from the memory 1 corresponding to the physical address 1 based on the mapping between the virtual address 3 and the physical address 1, and composes an image based on the layer data.

[0180] S720 can refer to the description of S620, and details are not described herein.

[0181] In the embodiments of the present application, in general, when an application in the application layer is started, the application can apply for a graphics memory to a graphics memory management decision device (for example, a rendering service), and then the graphics memory management decision device can first allocate a physical address of a producer device (for example, a CPU) and a virtual address of the producer device, and set a flag bit of the producer device, which can be used to mark the physical address allocated for the producer device, that is, the flag bit of the producer device can be used to mark that the physical address allocated for the producer device is ready to be accessed by the producer device. Then, the graphics memory management decision device maps the virtual address of the producer device to the physical address of the memory allocated for the producer device, and then the producer device can use the mapping between the virtual address of the producer device and the physical address of the memory allocated for the producer device to write the produced data (for example, image data) into the physical address of the memory allocated for the producer device. Then, the graphics memory management decision device selects a consumer device that needs to consume data. If the consumer device is not set with a flag bit, for example, the DSS is not set with a flag bit, then the flag bit of the consumer device is set and a virtual address of the consumer device is applied, and the flag bit of the consumer device indicates the physical address of the memory where the produced data is stored, that is, the flag bit of the consumer device can be used to mark that the physical address where the produced data is stored is ready to be accessed by the consumer device. Then, the graphics memory management decision device maps the virtual address of the consumer device to the physical address where the produced data is stored, and then the consumer device can use the mapping between the virtual address of the consumer device and the physical address where the produced data is stored to obtain the produced data from the memory corresponding to the physical address where the produced data is stored, and perform consumption processing on the produced data.

[0182] In the embodiments of the present application, the part of the image processing using the CPU, the GPU and the DSS can include but is not limited to:

[0183] Case 1: The CPU completes the original layer drawing and writes the drawn data into the buffer. Then the GPU or the DSS reads the drawn data from the buffer and synthesizes the image to be displayed using the drawn data.

[0184] Case 2: The CPU completes the original layer drawing and writes the drawn data into the buffer. Then the GPU reads the drawn data from the buffer and writes the drawn data into the read-only memory (ROM) or an external device.

[0185] Case 3: The CPU completes the original layer drawing and writes the drawn data into the buffer. Then the GPU reads the drawn data from the buffer and performs image enhancement processing such as rendering on the drawn data. Then the DSS reads the image enhancement processed data from the buffer and synthesizes the image to be displayed using the image enhancement processed data. The image enhancement processing can include, but is not limited to, one or more of contrast adjustment, sharpening, color enhancement or filtering processing.

[0186] Case 4: The CPU and the GPU complete the original layer drawing, that is, the CPU completes the drawing of a part of the original layer and the GPU completes the drawing of another part of the original layer, and then the CPU and the GPU each write the drawn original layer into the buffer. Then the DSS reads the drawn data from the buffer and synthesizes the image to be displayed using the drawn data.

[0187] Case 5: The CPU completes the original layer drawing and writes the drawn data into the buffer. Then the GPU and the DSS read the drawn data from the buffer and synthesize the image to be displayed using the drawn data by the GPU and the DSS. In this case, optionally, the GPU can synthesize a part of the image using a part of the drawn data and the DSS can synthesize another part of the image using another part of the drawn data, and then the part of the image synthesized by the GPU and the other part of the image synthesized by the DSS are spliced to obtain the complete image. In another possible implementation, the GPU can synthesize the image using the drawn data and the DSS can synthesize the image using the drawn data, and then the image synthesized by the GPU and the image synthesized by the DSS are fused to obtain the fused image.

[0188] The following embodiments respectively explain the dynamic mapping of the shared physical memory to the virtual addresses of the multiple devices for the above cases.

[0189] If the image processing corresponds to case one, the rendering service can allocate memory for the CPU, apply a virtual address of the CPU, and set a flag of the CPU, the flag of the CPU being used to indicate a physical address of the memory allocated for the CPU, that is, the flag of the CPU can be used to mark that the physical address allocated for the CPU is ready to be accessed by the CPU. Then, the rendering service maps the virtual address of the CPU to the physical address, and the CPU can write image data (which can also be referred to as rendered data or layer data) into the memory corresponding to the physical address by using the mapping between the virtual address of the CPU and the physical address. Then, if the image needs to be composed by the GPU, the rendering service sets a flag of the GPU, the flag of the GPU being used to indicate the physical address where the image data is stored, that is, the flag of the GPU can be used to mark that the physical address where the image data is stored is ready to be accessed by the GPU. And the rendering service can apply a virtual address of the GPU, and map the virtual address of the GPU to the physical address where the image data is stored, and the GPU can acquire the image data from the memory corresponding to the physical address where the image data is stored by using the mapping between the virtual address of the GPU and the physical address where the image data is stored for image composition. If the image needs to be composed by the DSS, the rendering service sets a flag of the DSS, the flag of the DSS being used to indicate the physical address where the image data is stored, that is, the flag of the DSS can be used to mark that the physical address where the image data is stored is ready to be accessed by the DSS. And the rendering service can apply a virtual address of the DSS, and map the virtual address of the DSS to the physical address where the image data is stored, and the DSS can acquire the image data from the memory corresponding to the physical address where the image data is stored by using the mapping between the virtual address of the DSS and the physical address where the image data is stored for image composition.

[0190] If the image processing corresponds to case two, the rendering service can allocate memory for the CPU, apply a virtual address of the CPU, and set a flag of the CPU. Then, the rendering service maps the virtual address of the CPU to a physical address of the memory, and the CPU can write image data into the physical address by using the mapping between the virtual address of the CPU and the physical address. Then, if the image data needs to be written into the read-only memory by the GPU, the rendering service sets a flag of the GPU, and the rendering service can apply a virtual address of the GPU, and map the virtual address of the GPU to the physical address where the image data is stored, and the GPU can acquire the image data from the memory corresponding to the physical address where the image data is stored by using the mapping between the virtual address of the GPU and the physical address where the image data is stored and write the image data into the read-only memory.

[0191] If the image processing corresponds to case three, the rendering service can allocate a memory for the CPU, apply a virtual address of the CPU, and set a flag of the CPU, and then the rendering service maps the virtual address of the CPU to a physical address of the memory, and then the CPU can write the image data into the physical address by using the mapping between the virtual address of the CPU and the physical address. Then, if the image enhancement processing needs to be performed by the GPU, the rendering service sets a flag of the GPU, and the rendering service can apply a virtual address of the GPU and map the virtual address of the GPU to the physical address where the image data is stored, and then the GPU can perform the image enhancement processing on the image data from the memory corresponding to the physical address by using the mapping between the virtual address of the GPU and the physical address of the image data. At this time, the GPU can write the image data after the image enhancement processing into the physical address where the image data is stored, or into another physical address. If the image data after the image enhancement processing is written into another physical address, the rendering service can allocate another memory for the GPU and set another flag of the GPU, the another flag indicating another physical address of the another memory, that is, the another flag can be used to mark that the another physical address allocated for the GPU is ready to be accessed by the GPU. And the rendering service can apply another virtual address of the GPU and map the another virtual address of the GPU to the another physical address, and then the GPU can write the image data after the image enhancement processing into the another physical address by using the mapping between the another virtual address of the GPU and the another physical address. Then, if the image synthesis needs to be performed by the DSS, the rendering service sets a flag of the DSS, the flag of the DSS can be used to indicate the another physical address, that is, the flag of the DSS can be used to mark that the another physical address is ready to be accessed by the DSS. And the rendering service can apply a virtual address of the DSS and map the virtual address of the DSS to the another physical address, and then the DSS can perform the image synthesis on the image data after the image enhancement processing from the another memory corresponding to the another physical address by using the mapping between the virtual address of the DSS and the another physical address.

[0192] It should be noted that in the image processing of case three, if the original image data needs to be preserved, the image data after the image enhancement processing can be written into another memory corresponding to another physical address; if the original image data does not need to be preserved, the image data after the image enhancement processing can be written into the memory corresponding to the physical address to replace the original image data previously written into the physical address.

[0193] If the image processing corresponds to case four, the rendering service can allocate memory for the CPU and the GPU, apply a virtual address for the CPU, apply a virtual address for the GPU, and set a flag bit for the CPU and the GPU, where the flag bit is used to indicate the physical address of the memory allocated for the CPU and the GPU, that is, the flag bit can be used to mark that the memory is ready to be accessed by the CPU or the GPU. Then, the rendering service maps the virtual address of the CPU and the GPU to the physical address respectively, so that the CPU can write image data into the memory corresponding to the physical address by using the mapping between the virtual address of the CPU and the physical address, and the GPU can also write image data into the memory corresponding to the physical address by using the mapping between the virtual address of the GPU and the physical address. Then, the rendering service judges that the image data needs to be sent to the DSS for synthesis, sets a flag bit of the DSS to indicate that the memory is ready to be accessed by the DSS, and then the rendering service can apply a virtual address of the DSS and map the virtual address of the DSS to the physical address, so that the DSS can obtain image data from the memory corresponding to the physical address by using the mapping between the virtual address of the DSS and the physical address for synthesis.

[0194] If the image processing corresponds to case five, the rendering service can allocate memory for the CPU, apply a virtual address for the CPU, and set a flag bit for the CPU. Then, the rendering service maps the virtual address of the CPU to the physical address, so that the CPU can write image data into the memory corresponding to the physical address by using the mapping between the virtual address of the CPU and the physical address. Then, the rendering service applies a virtual address for the CPU and a virtual address for the GPU, sets flag bits for the CPU and the GPU, and maps the virtual address of the CPU and the virtual address of the GPU to the physical address respectively, so that the CPU can obtain image data from the memory corresponding to the physical address by using the mapping between the virtual address applied by the CPU and the physical address for synthesis, and the GPU can obtain image data from the memory corresponding to the physical address by using the mapping between the virtual address applied by the GPU and the physical address for synthesis.

[0195] In order to more intuitively use the physical memory usage of the embodiments of the present application, the following embodiments provide diagrams to exemplarily illustrate the physical memory usage of the embodiments of the present application.

[0196] Referring to FIG. 8, FIG. 8 is a diagram of a physical memory usage according to an embodiment of the present application. The embodiment of the present application takes the bit width of CPU and GPU as 64 bits and the bit width of DSS as 32 bits as an example for illustration. As shown in FIG. 8, the shared physical memory shared by CPU, GPU and DSS is 2G, and the shared physical memory shared by CPU and GPU is 6G, that is, the shared physical memory available for CPU and GPU is 8G. Thus, compared with the physical memory usage of FIG. 2, the shared physical memory used by CPU and GPU is improved, thereby improving the physical memory usage of CPU and GPU, and thus improving the overall performance of the electronic device.

[0197] It should be noted that the physical memory shared by CPU, GPU and DSS is not limited to 2G, but can be the maximum physical memory supported by DSS. For example, when the bit width of DSS is 32 bits, the physical memory shared by CPU, GPU and DSS can be at most 4G. However, since the maximum physical memory supported by CPU and GPU is 8G, in this case, the shared physical memory shared by CPU and GPU can be 4G, that is, the shared physical memory available for CPU and GPU can still be 8G.

[0198] In this example, the physical memory usage can exceed 4G (the maximum bit width available for the chip with the minimum bit width), and the virtual address of the chip with the minimum bit width does not exceed 4G.

[0199] In the embodiment of the present application, for example, for 16G physical memory, the shared memory utilization rate of CPU and GPU is 8G / 16G=50%, and for 32G physical memory, the shared memory utilization rate of CPU and GPU is 8G / 32G=25%. By comparing the utilization rate of the present application with that of the related art, it can be seen that the utilization rate of the shared memory of the embodiment of the present application is improved.

[0200] It should be understood that, in the embodiment of the present application, for the shared physical memory shared by multiple devices, it is limited by the maximum physical memory supported by the device with the minimum bit width among the multiple devices. As for the shared physical memory between the devices with larger bit widths, it is limited by the maximum physical memory supported by the device with the minimum bit width among the devices with larger bit widths.

[0201] For example, assuming that the bit width of the CPU is 128, the bit width of the GPU is 64, and the bit width of the DSS is 32, the shared physical memory among the CPU, the GPU, and the DSS is limited by the maximum physical memory supported by the DSS (4G). As for the shared physical memory between the CPU and the GPU, it is limited by the maximum physical memory supported by the GPU (8G), that is, the shared memory between the CPU and the GPU can be at most 8G and at least 4G (the difference between the maximum physical memory supported by the GPU 8G and the maximum physical memory supported by the DSS 4G).

[0202] For example, assuming that the bit width of the CPU is 128, the bit width of the GPU is 64, and the bit width of the DSS is 32, the shared physical memory among the CPU, the GPU, and the DSS is limited by the maximum physical memory supported by the DSS (4G). As for the shared physical memory between the CPU and the GPU, it is limited by the maximum physical memory supported by the GPU (8G), that is, the shared memory between the CPU and the GPU can be at most 8G and at least 4G (the difference between the maximum physical memory supported by the GPU 8G and the maximum physical memory supported by the DSS 4G).

[0203] In the above embodiments, the scenario in which the CPU, the GPU, and the DSS are one is described. In the following embodiments, the scenario in which the CPU, the GPU, and the DSS are two or more is described.

[0204] In a possible implementation, assuming that the CPU is multiple, the multiple CPUs include CPU3 and CPU4, if the CPU is required to produce data, the production data can be first written to the physical address by CPU3, when the address used by CPU3 reaches the maximum physical memory supported by CPU3, the production data is subsequently written to the physical address by CPU4, in this way, another CPU can be used after one CPU reaches the usage limit, thereby the service life of the part of the CPUs can be improved. Alternatively, when the CPU is required to produce data, CPU3 and CPU4 can alternately write the production data to the physical address, for example, CPU3 writes the first batch of production data to the physical address 3, CPU4 writes the second batch of production data to the physical address 4, CPU3 writes the third batch of production data to the physical address 5, CPU4 writes the fourth batch of production data to the physical address 6, and so on, until the physical address used by at least one of CPU3 and CPU4 reaches the maximum physical memory supported by the CPU, in this way, the service life of the multiple CPUs as a whole can be improved.

[0205] In another possible implementation, assuming that the GPUs are multiple, the multiple GPUs include GPU1 and GPU2. Then if the GPU needs to produce data, the GPU1 can be used to write the production data into the physical address first, and when the address used by the GPU1 reaches the maximum physical memory supported by the GPU1, the GPU4 can be used to write the production data into the physical address subsequently. In this way, another GPU can be used after one GPU reaches the usage limit, and thus the service life of part of the GPUs can be improved. Alternatively, the GPU1 can be used to write the production data into the physical address exclusively, and the GPU2 can be used to read the data from the physical address and process the data exclusively. Alternatively, the GPU1 and the GPU2 can be used to process the data alternately, for example, the GPU1 writes the fifth batch of production data into the physical address 7, then the GPU2 obtains the sixth batch of production data from the physical address 8 for consumption, then the CPU1 obtains the seventh batch of production data from the physical address 9 for consumption, and then the GPU2 obtains the eighth batch of production data from the physical address 10 for consumption. In this way, the service life of the multiple GPUs as a whole can be improved.

[0206] It should be noted that the alternation of the GPUs can be the alternation of the production data, the alternation of the consumption data, or the alternation of the production data and the consumption data, which is not limited herein.

[0207] In another possible implementation, assuming that the DSSs are multiple, the multiple DSSs include DSS1 and DSS2. Then if the DSS needs to consume data, the DSS1 can be used to obtain the data from the physical address for consumption first, and when the address used by the DSS1 reaches the maximum physical memory supported by the DSS1, the DSS can be used to continue the data consumption subsequently. In this way, another DSS can be used after one DSS reaches the usage limit, and thus the service life of part of the DSSs can be improved. Alternatively, the DSS1 and the DSS2 can be used to process the data alternately, for example, the DSS1 obtains the ninth batch of production data from the physical address 11 for consumption, and then the DSS2 obtains the tenth batch of production data from the physical address 12 for consumption. In this way, the service life of the multiple DSSs as a whole can be improved.

[0208] It should be noted that in the embodiments of the present application, how the CPU writes the data into the physical address, how the GPU writes the data into the physical address, how the GPU reads the data from the physical address for processing, and how the DSS reads the data from the physical address for processing can be referred to the related descriptions of the embodiments of FIG. 6 and FIG. 7, which will not be repeated herein.

[0209] It should be understood that in the embodiments of the present application, the relationship between the devices performing data processing is mainly considered, and the size of the physical memory can meet the physical memory required by the multiple devices performing data processing by default.

[0210] In the above embodiments, the processing of the graphics data is taken as an example for illustration. The graphics data can comprise layer data. It should be understood that for other data processing, such as machine learning scenarios, including but not limited to training or inference of an artificial intelligence model, the training or inference of the artificial intelligence model can also be referred to the description of the embodiments of the present application, which is not described herein.

[0211] For example, the electronic device comprises a data preprocessing chip (such as a CPU), a first training chip (such as a GPU), and a second training chip (such as an NPU). The data preprocessing chip can be used to preprocess data, such as data cleaning, model selection, and the like. Then, the model can be trained by the first training chip or the second training chip. Alternatively, the model is selected by the first training chip and pre-trained, and then the pre-trained model is further trained by the second training chip. In this case, the data preprocessing chip is a producer device, the first training chip can be a producer device or a consumer device, and the second training chip can be a consumer device.

[0212] In this case, in the data production stage, the data preprocessing chip and the first training chip can apply for virtual addresses, respectively, and the memory 2 is allocated, and the virtual address 4 of the data preprocessing chip and the virtual address 5 of the first training chip are mapped to the physical address 2 of the memory 2. Then, the data preprocessing chip can write the selected model into the memory 2 corresponding to the physical address 2. Then, in the data consumption stage, if the first training chip needs to be trained, the first training chip does not need to apply for a virtual address, and the first training chip can use the mapping between the virtual address 5 and the physical address 2 to obtain the model from the memory 2 corresponding to the physical address 2 for training, and if the second training chip needs to be trained, a virtual address 6 is applied to the second training chip, and the virtual address 6 is mapped to the physical address 2, and then the second training chip can obtain the model from the memory 2 corresponding to the physical address 2 based on the mapping between the virtual address 6 and the physical address 2 for training.

[0213] In addition, in the data production stage, the first training chip can select the model and pre-train the model, and then the first training chip can write the pre-trained model into the memory 2 corresponding to the physical address 2 using the mapping between the virtual address 5 and the physical address 2. Then, in the data consumption stage, a virtual address 6 can be applied to the second training chip, and the virtual address 6 is mapped to the memory 2 corresponding to the physical address 2, and then the second training chip can obtain the pre-trained model from the memory 2 corresponding to the physical address 2 based on the mapping between the virtual address 6 and the physical address 2 for further training.

[0214] It should be understood that in addition to the above example scenarios, the present solution can also be applied to scenarios of implementing automotive head-up display by a GPU, such as displaying content at a windshield of a vehicle, and the like.

[0215] In general, the shared memory allocation of the embodiments of the present application is divided into a production phase and a consumption phase, and each phase dynamically maps virtual addresses for related chips on demand.

[0216] Referring to FIG. 9, FIG. 9 is a flowchart of an address mapping method provided by an embodiment of the present application. The method shown in FIG. 9 can be applied to an electronic device, which includes a first application, a first device, a second device, a third device, and a physical memory. The bit width of the first device, the bit width of the second device, and the bit width of the third device are not completely equal. For example, the bit width of the first device and the bit width of the second device can both be greater than the bit width of the third device, or the bit width of the first device can be less than the bit width of the second device and the bit width of the third device, or the bit width of the second device can be less than the bit width of the first device and the bit width of the third device, which is not limited herein. The first device, the second device, and the third device can be chips. For example, the first device can include a CPU, the second device can include a GPU, and the third device can include a DSS. In the embodiments of the present application, the first device is a producer device, the second device can be a producer device or a consumer device, and the third device can be a consumer device. The method shown in FIG. 9 can include the following steps.

[0217] S901, the first application applies for memory.

[0218] The first application can be an application program installed on the electronic device. In the embodiments of the present application, the first application can be an application that needs to use the physical memory. For example, the first application can include, but is not limited to, a browser application and a photographing application, and the like. For example, when the browser application and the photographing application open an application window, memory is needed to store graphics data.

[0219] For example, S901 can refer to the description of S611, which is not repeated herein.

[0220] S902, the first virtual address of the first device and the second virtual address of the second device are applied, and a first memory in the physical memory is allocated, and the first virtual address and the second virtual address are respectively mapped to a first physical address of the first memory, and the first memory is at least used to store first data written by the first device.

[0221] The virtual address is a concept relative to the physical address. It is an address in the address space used when accessing the memory. The main purpose of the virtual address is to provide an abstraction layer for memory management, so that the operating system can effectively manage the physical memory while protecting the memory isolation between processes. The physical address is a unique identifier for each byte in the physical memory, which is an address directly corresponding to the actual location in the computer hardware (such as the memory chip).

[0222] In the embodiment of the present application, the first virtual address and the second virtual address are mapped to the first physical address of the first memory respectively, so that the first device and the second device can access the first physical address, and then data can be read and written in the first memory. Optionally, the first device of the embodiment can write the first data to the first memory as a producer device. If the second device is a producer device, the second device can also write the second data of the second device to the first memory, and if the second device is a consumer device, the second device can read the first data from the first memory for processing.

[0223] For example, if the first device includes a CPU, the first virtual address may, for example, be virtual address 1, and if the second device includes a GPU, the second virtual address may, for example, be virtual address 2. The first memory may, for example, be memory 1, and the first physical address may, for example, be physical address 1. The third virtual address may, for example, be virtual address 3. S902 can refer to the description of S712, and will not be repeated here.

[0224] S903, apply for a third virtual address of a third device, and map the third virtual address to the first physical address.

[0225] In the embodiment of the present application, the virtual address of the third device can be applied for and mapped to the first physical address in the case where the third device needs to process data, that is, in the case where the third device is a consumer device, so that the third device can also access the first physical address, and thus data can be obtained from the first memory for processing.

[0226] For example, S903 can refer to the description of S718, and will not be repeated here.

[0227] S904, the third device obtains first data from the first memory corresponding to the first physical address for processing.

[0228] In the embodiment of the present application, the processing responsible for by the third device may, for example, be image synthesis, and may, for example, also be model training or inference, and the like, which is not limited here.

[0229] For example, S904 can refer to the description of S720, and will not be repeated here.

[0230] It should be noted that, since the first memory stores the first data, the third device can at least acquire the first data from the first memory for processing. If the second memory also stores the second data written by the second device, the third device can also acquire the second data from the first memory for processing.

[0231] In the embodiment of the present application, by applying for the first virtual address of the first device in the production data stage, allocating the first memory in the physical memory, and mapping the first virtual address to the first physical address of the first memory, the first device can write the first data to the first memory. Then, the second virtual address of the second device is applied for, and the second virtual address is mapped to the first physical address of the first memory respectively. In this way, if the second device is a producer device, it can write the second data to the first memory, and if the second device is a consumer device, it can also acquire the first data from the first memory for processing. Then, when the third device is a consumer device, the third virtual address of the third device is applied for, and the third virtual address is mapped to the first physical address. In this way, the third device can acquire the first data from the first memory corresponding to the first physical address for processing. Since the mapping of the virtual address is divided into multiple stages, for example, divided into a production data stage and a consumption data stage, the shared physical memory can be dynamically mapped to the virtual addresses of the respective chips, thereby improving the overall performance of the electronic device.

[0232] For example, the first data and the second data can be, for example, layer data drawn, and the data processing of the corresponding third device can be, for example, image synthesis processing.

[0233] In a possible implementation, after the first application applies for the memory, the method can further include:

[0234] The first identifier of the first device and the second device is set, and the first identifier indicates that the device has been mapped to the first physical address.

[0235] In the embodiment of the present application, the first identifier of the first device is set, which indicates that the first device has been mapped to the first physical address, in other words, the virtual address of the applied first device has been mapped to the first physical address. The second identifier of the second device is set, which indicates that the second device has been mapped to the first physical address, in other words, the virtual address of the applied second device has been mapped to the first physical address. Optionally, the identifier of the embodiment of the present application can also be referred to as a flag bit.

[0236] It should be noted that, if the device already has the first identifier, when the device needs to access the first physical address of the first memory, mapping is no longer needed.

[0237] For example, the embodiment of the present application can refer to the description of S612, which is not repeated here.

[0238] In the embodiment of the present application, the identifier is set after the virtual address of the device is mapped to the physical address, so that it can be known through the identifier whether the physical address is mapped to the virtual address of the device, and then the data in the memory corresponding to the physical address can be read and written in time, thereby improving the efficiency of data processing. In addition, the number of virtual addresses of the device applied for can be reduced.

[0239] In another possible implementation, the virtual address of the device can also be applied for and mapped to the physical address of the memory every time the device accesses the memory, so that the resources required for setting the identifier can be reduced.

[0240] In a possible implementation, the third virtual address of the third device is applied for and mapped to the first physical address, comprising:

[0241] In a case where it is determined that the data is processed by the third device and the first identifier of the third device is not set, the third virtual address of the third device is applied for and mapped to the first physical address, and the first identifier of the third device is set, the first identifier indicating that the device has been mapped to the first physical address.

[0242] For example, the embodiment can refer to the description of S614-S616, which is not repeated here.

[0243] In the embodiment of the present application, in a case where it is determined that the data is processed by the third device and the first identifier of the third device is not set, it indicates that the virtual address of the third device has not been mapped to the first physical address, so the third virtual address of the third device can be applied for and mapped to the first physical address. That is, when the third device is a consumer device, whether the virtual address of the third device needs to be applied for and mapped to the first physical address is determined through the identifier, so that the accuracy and efficiency of applying for and mapping the virtual address of the third device can be improved.

[0244] In another possible implementation, the virtual address of the third device can also be applied for and mapped to the first virtual address in a case where it is determined that the data is processed by the third device, so that the resources required for setting the identifier of the third device can be reduced.

[0245] In a possible implementation, before the third virtual address of the third device is applied for and mapped to the first physical address, the method further comprises:

[0246] The first set and the second set are obtained, the first set comprising devices with the first identifier set, and the second set comprising devices for processing data, the devices with the first identifier set comprising the first device and the second device, and the devices for processing data comprising the third device;

[0247] In a case where it is determined that there is the first difference set based on the first set and the second set, a virtual device of the device in the first difference set is applied, and the virtual device of the device in the first difference set is mapped to the first physical address, the device in the first difference set is the device in the second set and the device in the first difference set is different from any device in the first set;

[0248] The third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address, comprising:

[0249] In a case where the first difference set includes the third device, the third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address.

[0250] The first set can also be referred to as a set of device identifiers that have been marked, and the second set can also be referred to as a set of consumer devices.

[0251] In the embodiment of the present application, the device in the first difference set is the determined consumer device and has not been marked with the first identifier, that is, the device in the first difference set needs to apply for a virtual address and be mapped to the first physical address. If the first difference set includes the third device, it means that the third device needs to apply for a virtual address and be mapped to the first physical address, and then the third virtual address of the third device is applied, and the third virtual address is mapped to the first physical address.

[0252] For example, the embodiment of the present application can refer to the description of the embodiment of FIG. 5, which is not repeated here.

[0253] In a possible implementation, the electronic device further includes a first service, and the first application applies for memory, comprising:

[0254] The first application applies for memory from the first service;

[0255] The first virtual address of the first device and the second virtual address of the second device are applied, and the first memory in the physical memory is allocated, and the first virtual address and the second virtual address are respectively mapped to the first physical address of the first memory, comprising:

[0256] The first service applies for the first virtual address of the first device and the second virtual address of the second device, and allocates the first memory in the physical memory, and maps the first virtual address and the second virtual address to the first physical address of the first memory respectively.

[0257] For example, the first service can be a rendering service, or other services running in the system, which is not limited here.

[0258] In the embodiments of the present application, the first application can apply for memory from the first service, and the first service can apply for a first virtual address of the first device and a second virtual address of the second device, and allocate a first memory in the physical memory, and map the first virtual address and the second virtual address to a first physical address of the first memory respectively. In this way, the problem of low efficiency and high resource occupation caused by interaction of multiple services to implement address mapping can be reduced, and thus the efficiency of address mapping can be improved and the resource occupation can be reduced.

[0259] In another possible implementation, the application of the virtual address, the allocation of the memory, and the mapping between the virtual address and the physical address can be implemented by different services or modules. For example, the first service applies for the virtual address, the second service allocates the memory, and the third service maps between the virtual address and the physical address, which is not limited herein. In this way, the problem of excessive service computing power caused by multiple processes performed by one service can be reduced.

[0260] In a possible implementation, the application of the third virtual address of the third device and the mapping of the third virtual address to the first physical address comprises:

[0261] The first service applies for the third virtual address of the third device and maps the third virtual address to the first physical address.

[0262] In the embodiments of the present application, the first service can also apply for the virtual address of the third device and perform mapping, which can reduce the problem of low efficiency and high resource occupation caused by interaction of multiple services to implement address mapping, and thus the efficiency of address mapping can be improved and the resource occupation can be reduced.

[0263] In another possible implementation, the third virtual address of the third device can also be applied for by other services, and the third virtual address is mapped to the first physical address, which can reduce the problem of excessive service computing power caused by multiple processes performed by one service.

[0264] In a possible implementation, the first application applies for memory, comprising:

[0265] The first application applies for graphics memory in response to opening a window on the electronic device, and the graphics memory is used to store graphics data required by the window.

[0266] In the embodiments of the present application, the window can be a window of the first application, or a newly opened window in the first application, which is not limited herein.

[0267] In the embodiments of the present application, the first application applies for graphics memory in response to opening a window on the electronic device, which can improve the number of windows opened by the electronic device.

[0268] In a possible implementation, the first memory is also used to store second data written by the second device, that is, the second device writes data to the first memory as a producer device.

[0269] In another possible implementation, the second virtual address of the second device is applied, including:

[0270] In a case where it is determined that the data is processed by the second device and the first identifier of the second device is not set, the second virtual address of the second device is applied, and the first identifier of the second device is set, the first identifier indicating that the device is mapped to the first physical address.

[0271] In the embodiment of the application, the second device is a consumer device. If it is determined that the data needs to be processed by the second device and the first identifier of the second device is not set, it indicates that the virtual address of the second device is not mapped to the first physical address, and the second device cannot normally access the first memory. Therefore, the second virtual address of the second device is applied, so that the second virtual address can be mapped to the first physical address, so that the second device can normally access the first memory, and then the first data is obtained from the first memory for processing.

[0272] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to select authorization or refusal.

[0273] It should also be understood that the steps of each of the above embodiments can also be coupled with each other, and the application does not limit this. The size of the serial number of the above processes does not mean the order of execution, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0274] The address mapping method of the embodiments of the application is described in detail above, and the address mapping device of the embodiments of the application will be described in detail below with reference to FIG. 10.

[0275] FIG. 10 shows a schematic block diagram of an address mapping apparatus 1000 provided by an embodiment of the present application. The apparatus 1000 includes a processor 1001, a transceiver 1002, and a memory 1003. The processor 1001, the transceiver 1002, and the memory 1003 communicate with each other through an internal connection path. The memory 1003 is configured to store instructions, and the processor 1001 is configured to execute the instructions stored in the memory 1003 to control the transceiver 1002 to transmit and / or receive signals.

[0276] It should be understood that the steps performed by the apparatus 1000 of the embodiments of the present application can be described with reference to the above method embodiments, which will not be repeated here.

[0277] In the embodiments of the present application, the apparatus 1000 in FIG. 10 can also be a chip or a chip system, for example, a system on chip (SoC).

[0278] It should be understood that the apparatus 1000 can be specifically an electronic device in the above embodiments, and can be used to perform each step and / or process corresponding to the electronic device in the above method embodiments. Alternatively, the memory 1003 can include a read-only memory and a random access memory, and provide instructions and data for the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store device type information. The processor 1001 can be used to execute the instructions stored in the memory, and when the processor 1001 executes the instructions stored in the memory, the processor 1001 is used to perform each step and / or process of the above method embodiments. The transceiver 1002 can include a transmitter and a receiver. The transmitter can be used to implement each step and / or process corresponding to the transmitter for performing a transmitting action, and the receiver can be used to implement each step and / or process corresponding to the receiver for performing a receiving action.

[0279] It should be understood that in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0280] In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution completion, or executed by hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor executes the instructions in the memory to complete the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0281] It should be understood that the apparatus 1000 herein can also be embodied in the form of functional modules. For example, the apparatus 1000 can refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, and a combination of logical circuits and / or other suitable components supporting the described functions.

[0282] The embodiments of the present application also provide a computer readable storage medium for storing a computer program for implementing the method shown in the above method embodiments.

[0283] The embodiments of the present application also provide a computer program product, which includes a computer program (also referred to as code or instructions), and when the computer program runs on a computer, the computer can execute the method shown in the above method embodiments.

[0284] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0285] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, apparatus and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0286] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other other forms.

[0287] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0288] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0289] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

Claims

1. An address mapping method, characterized by, The application is applied to an electronic device, the electronic device comprises a first application, a first device, a second device, a third device and a physical memory, a bit width of the first device, a bit width of the second device and a bit width of the third device are not completely equal, and the method comprises the following steps of: The first application applies for memory; A first virtual address of the first device and a second virtual address of the second device are applied for, a first memory in the physical memory is allocated, and the first virtual address and the second virtual address are respectively mapped to a first physical address of the first memory, and the first memory is used at least for storing first data written by the first device; A third virtual address of the third device is applied for, and the third virtual address is mapped to the first physical address; The third device obtains the first data from the first memory corresponding to the first physical address and processes the first data.

2. The method of claim 1, wherein, After the first application applies for memory, the method further comprises the following steps of: A first identifier of the first device and a first identifier of the second device are set, and the first identifier indicates that a device has been mapped to the first physical address.

3. The method according to claim 1 or 2, characterized in that, The step of applying for the third virtual address of the third device and mapping the third virtual address to the first physical address comprises the following steps of: In a case where it is determined that the data is processed by the third device and the first identifier of the third device is not set, the third virtual address of the third device is applied for, the third virtual address is mapped to the first physical address, and the first identifier of the third device is set, and the first identifier indicates that a device has been mapped to the first physical address.

4. The method according to any one of claims 1-3, characterized in that, Before the step of applying for the third virtual address of the third device and mapping the third virtual address to the first physical address, the method further comprises the following steps of: A first set and a second set are obtained, the first set comprises devices whose first identifiers are set, the second set comprises devices for processing data, the devices whose first identifiers are set comprise the first device and the second device, and the devices for processing data comprise the third device; In a case where it is determined that there is a first difference set based on the first set and the second set, a virtual device of a device in the first difference set is applied for, and the virtual device of the device in the first difference set is mapped to the first physical address, the device in the first difference set is a device in the second set, and the device in the first difference set is different from any device in the first set; The step of applying for the third virtual address of the third device and mapping the third virtual address to the first physical address comprises the following steps of: In a case where the first difference set comprises the third device, the third virtual address of the third device is applied for, and the third virtual address is mapped to the first physical address.

5. The method according to any one of claims 1-4, characterized in that, The first device comprises a central processing unit, the second device comprises a graphic processing unit, and the third device comprises a display subsystem.

6. The method according to any one of claims 1-5, characterized in that, The electronic device further comprises a first service, and the first application applies for memory, comprising the following steps of: The first application applies for memory from the first service; The first application applies for memory from the first service; The first virtual address of the first device and the second virtual address of the second device are applied, and a first memory in the physical memory is allocated, and the first virtual address and the second virtual address are respectively mapped to a first physical address of the first memory, comprising: The first service applies the first virtual address of the first device and the second virtual address of the second device, and allocates a first memory in the physical memory, and maps the first virtual address and the second virtual address to a first physical address of the first memory.

7. The method according to any one of claims 1 to 6, characterized in that, The first application applies memory, comprising: The first application applies graphics memory in response to opening a window of the electronic device, and the graphics memory is used to store graphics data required by the window.

8. The method according to any one of claims 1 to 7, characterized in that, The first memory is also used to store second data written by the second device; or The second virtual address of the second device is applied, comprising: In a case where it is determined that the data is processed by the second device and the first identifier of the second device is not set, the second virtual address of the second device is applied, and the first identifier of the second device is set, the first identifier indicating that the device has been mapped to the first physical address.

9. An address mapping apparatus, characterized by comprising: Comprising: A processor coupled with the memory, the memory storing computer-executable instructions, and the processor executing the computer-executable instructions stored in the memory, so that the processor executes the method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, A computer program for storing, the computer program comprising instructions for implementing the method of any one of claims 1 to 8.

11. A computer program product comprising computer program code in said computer program product, characterised in that, When the computer program code runs on the computer, the computer implements the method of any one of claims 1 to 8.

Images