Image storage, memory allocation, image synthesis method, device, equipment and medium

Abstract

Embodiments of the present application disclose an image storage, memory allocation, image synthesis method, device, equipment and medium. The method comprises: acquiring at least one shared memory address; writing at least one image into a shared memory pointed by the corresponding shared memory address, so that a video stream module extracts each image from each shared memory and adds the image to a video stream. Embodiments of the present application can improve the synthesis efficiency of images and video streams and reduce the synthesis consumption of images and video streams.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of data processing, and particularly relate to an image storage, memory allocation, image synthesis method and device, equipment and medium. BACKGROUND

[0002] On Screen Display (OSD) technology is to generate text or graphics on the display, so that the user obtains more information, and provides a friendly man-machine interface for the user.

[0003] The application program draws the OSD image in the user space and stores it in the user space, and the OSD controller of the video stream usually uses one or several fixed video memories for storing the OSD image. When the OSD image and the video stream are synthesized, the OSD image needs to be moved from the user space to the fixed video memory, and then the OSD image in the video memory is synthesized with the video stream.

[0004] At this time, the coupling between the drawing and moving operations of the OSD image and the OSD image synthesis operation is high, and the drawing operation and the moving operation will compete for the CPU resources, while the moving operation and the synthesis operation will compete for the OSD hardware resources (such as the OSD controller), and the moving operation will additionally consume resources. SUMMARY

[0005] Embodiments of the present application provide an image storage, memory allocation, image synthesis method, device, equipment and medium, which can improve the synthesis efficiency of the image and the video stream and reduce the synthesis consumption of the image and the video stream.

[0006] In a first aspect, embodiments of the present application provide an image storage method applied to a central processing unit, comprising:

[0007] Obtaining at least one shared memory address;

[0008] Writing at least one image into a shared memory pointed by the corresponding shared memory address, so that a video stream module extracts each image from each shared memory and adds it to the video stream.

[0009] In a second aspect, embodiments of the present application further provide a memory allocation method applied to a graphics processing unit, comprising:

[0010] Allocating a shared memory for at least one image to be written;

[0011] Obtaining a shared memory address of each shared memory;

[0012] The shared memory address sending module is configured to send the shared memory addresses to the central processor, so that the central processor writes the images into the corresponding shared memories, and the video stream module extracts the images from the shared memories and adds the images into a video stream.

[0013] In a third aspect, an embodiment of the present application further provides an image storage device configured in a central processor, comprising:

[0014] The shared memory address obtaining module is configured to obtain at least one shared memory address.

[0015] The image drawing module is configured to write at least one image into a shared memory pointed by the shared memory address, so that the video stream module extracts the images from the shared memories and adds the images into a video stream.

[0016] In a fourth aspect, an embodiment of the present application further provides a memory allocation device configured in a graphics processor, comprising:

[0017] The shared memory allocation module is configured to allocate a shared memory for at least one image to be written.

[0018] The shared memory address obtaining module is configured to obtain a shared memory address of each shared memory.

[0019] The shared memory address sending module is configured to send the shared memory addresses to the central processor, so that the central processor writes the images into the corresponding shared memories, and the video stream module extracts the images from the shared memories and adds the images into a video stream.

[0020] In a fifth aspect, an embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image storage method or the memory allocation method according to any of the embodiments of the present application.

[0021] In a sixth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program executable by a processor, and the program implements the image storage method or the memory allocation method according to any of the embodiments of the present application.

[0022] In a seventh aspect, an embodiment of the present application further provides an image synthesis method, comprising:

[0023] The shared memory address obtaining module is configured to obtain at least one shared memory address, and write at least one image into a shared memory pointed by the shared memory address.

[0024] The storage data including the image is extracted from the shared memory pointed by each of the shared memory addresses by the video stream module and added to the video stream.

[0025] The embodiment of the present application configures the shared memory and writes the image into the shared memory by the central processor, so that the video stream module can read the image from the shared memory as needed and synthesize the image and the video stream, omits the image carrying operation, decouples the image writing operation and the image synthesis operation, and solves the problem of high coupling between the image carrying operation and the image synthesis operation in the prior art, low image synthesis efficiency and high cost. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a flowchart of an image storage method in the embodiment one of the present application;

[0027] Figure 2 is a flowchart of a memory allocation method in the embodiment two of the present application;

[0028] Figure 3 is a structural schematic diagram of an image storage device in the embodiment three of the present application;

[0029] Figure 4 is a structural schematic diagram of a memory allocation device in the embodiment four of the present application;

[0030] Figure 5 is a structural schematic diagram of a computer device in the embodiment five of the present application;

[0031] Figure 6a is a flowchart of an image synthesis method in the embodiment seven of the present application;

[0032] Figure 6b is a structural schematic diagram of an image synthesis system in the embodiment seven of the present application;

[0033] Figure 6c is a flowchart of an application scenario in the embodiment seven of the present application. DETAILED DESCRIPTION

[0034] The application will be described in further detail below with reference to the drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not to be used to limit the scope of the application. In addition, it should also be pointed out that, for the sake of description, only the parts related to the application are shown in the drawings and not all the structures.

[0035] Embodiment one

[0036] Figure 1 A flow chart of an image storage method according to an embodiment of the application. The embodiment can be applied to a case where a central processing unit writes an image to be added to a video stream into shared memory. The method can be executed by an image storage device according to an embodiment of the application. The device can be implemented in software and / or hardware and can be integrated into a computer device, specifically a central processing unit. As shown in Figure 1 the method according to the embodiment specifically includes:

[0037] S110, obtaining at least one shared memory address.

[0038] The shared memory address points to shared memory, and the shared memory is used to store images. Shared memory can refer to a large-capacity memory that can be accessed by multiple different programs or processors. Through shared memory, processors and processors, or programs and programs can communicate with each other.

[0039] Optionally, the shared memory address is obtained by generating a shared memory application request according to image attribute information of each image, sending the shared memory application request to a graphics processor, and receiving at least one shared memory address fed back by the graphics processor, the shared memory address pointing to shared memory with a capacity matching the image attribute information of the corresponding image.

[0040] Image attribute information is used to determine the occupied space of an image. Image attribute information can refer to information describing an image. Image attribute information can include at least one of the following: image size, image color mode, and image depth, etc. The image size can include the width of a rectangular image and the height of a rectangular image, or the radius of a circular image, etc. The image color mode can include Red Green Blue (RGB) mode, grayscale mode, Cyan Magenta Yellow Black (CMYK) mode, Luminance or Luma and Chrominance or Chroma (YUV) mode, or Lightness AB (LAB) mode, etc. Image depth information is used to describe the depth information of an image. Image attribute information determines the storage space capacity that an image will occupy.

[0041] The shared memory application request is used to apply for shared memory for writing images. The shared memory application request can include image attribute information of each image to be written. In fact, before applying for shared memory, the shared memory for storing images is not configured, the video memory for configuring fixed storage space for images can be avoided, the storage resources can be reasonably configured, the consumption of storage resources can be reduced, and the utilization rate of storage resources can be improved. The shared memory application request is sent to the graphics processor by the central processor. The graphics processor (Graphics Processing Unit, GPU) is used to allocate shared memory for the central processor. The graphics processor can allocate shared memory for the image according to the image attribute information of the image. The allocated shared memory is sufficient to carry the image. For example, the capacity of the allocated shared memory is equal to the occupied space capacity of the image, or the capacity of the allocated shared memory is slightly larger than the occupied space capacity of the image, for example, the occupied space capacity of the image is 10MB, and the capacity of the shared memory is 10.1MB. According to the occupied space capacity of the image, the shared memory is allocated dynamically according to the size of the image. The shared memory can be allocated specifically, the capacity of the shared memory is fixed, the complete image cannot be written, the image data is lost, or the shared memory is written after the image, and the remaining capacity is too large, which causes waste of storage resources.

[0042] By generating a shared memory application request according to the image attribute information of the image and sending it to the graphics processor, the graphics processor allocates shared memory matching the occupied space capacity of each image respectively, and receives the address of the allocated shared memory. The shared memory can be allocated according to the actual occupied space capacity of the image, the resources of the shared memory can be effectively utilized, the storage resources can be reasonably configured, and the waste of storage resources can be reduced.

[0043] S120, write at least one image into the shared memory pointed by the corresponding shared memory address, so that the video stream module extracts each image from each shared memory and adds it to the video stream.

[0044] Write the image into the shared memory corresponding to the shared memory address. The video stream module can read data from the shared memory, save the image writing buffer, and save the carrying operation of the video memory. The resource consumption of the carrying operation can be reduced, and the operation time of the carrying operation can be saved.

[0045] When the number of images is multiple, one image is written into one shared memory, and the storage location of each image is isolated, that is, one image corresponds to one shared memory, and one image corresponds to one shared memory address. In addition, multiple images can also be written into the same shared memory, for example, the images of the same layer are written into the same shared memory. The specific setting can be made according to the actual needs.

[0046] The video stream module is configured to acquire a video stream and to synthesize the acquired video stream and the image to output a synthesized video stream. The video stream acquisition includes generating a video stream or receiving (or downloading) a video stream. The video stream module can be an integrated circuit configured to acquire a video stream. For example, the video stream module includes at least one of an image acquisition module (e.g., a camera), a digital signal processor (DSP), an encoder, a decoder, an output module (e.g., a display), and the like. The video stream module can capture a real-time video stream of a scene or download a video stream from the Internet.

[0047] The video stream module can add the at least one image to the video stream. When there are multiple images, the video stream module can sequentially acquire the images and superimpose and synthesize the images to the video stream. The order of addition can be preconfigured.

[0048] Optionally, the image storage method further includes sending image synthesis parameters to the graphic processor, so that the graphic processor forwards the image synthesis parameters to the video stream module, and the video stream module sequentially adds the images to the video stream according to the image synthesis parameters.

[0049] The image synthesis parameters are used for video stream and image synthesis. The image synthesis parameters can be parameters associated with the addition of the image to the video stream. Optionally, the image synthesis parameters can include at least one of a shared memory address, an image addition position, an image size, and an addition order of multiple images. The shared memory address is used to indicate where the video stream module acquires the image to be added to the video stream. The image addition position of the image is used to determine the position of the image in the video stream. Each image is configured with a corresponding image addition position. For example, the image addition position can be the position of the image added in each frame of the video stream. The addition order can be the addition order of the images in the video stream when the number of images is multiple. The addition order is used to determine the superimposition order of the images and the display area of the images. It can be understood that multiple layers are configured on the video stream, each layer adds an image, and the superimposition order from the bottom layer to the top layer between the layers matches the addition order of the images, for example, the image with the first addition order is added to the bottom layer, and the image with the last addition order is added to the top layer. If the image addition positions of the images overlap, the upper layer will block the lower layer.

[0050] The video stream module sequentially acquires the images according to the shared memory address, adds the images according to the image addition positions of the images in the video stream according to the addition order, and the size of the added image is the same as the image size.

[0051] In one specific example, the image and the frame of the video stream are all rectangular images. The image is acquired, scaled according to the image size, or can be unchanged, to form an image of the image size. An image is selected from each image in the order of the adding sequence. For each image, a two-dimensional coordinate system is established in each frame of the video stream, and the image adding position is the coordinate (a, b). The key point of the image is placed at the coordinate (a, b), which can be the left top vertex of the image (or the intersection of the diagonal lines or other vertices, etc.). After the current image is added to the video stream, the next image in the adding sequence is returned, and the foregoing operations are repeated until all images are added to the video stream.

[0052] The graphics processor transmits the image synthesis parameters to the central processor through the inter-core communication. The graphics processor is further configured to receive the image synthesis parameters sent by the central processor, and forward the image synthesis parameters to the video stream module, so that the video stream module adds the image in the shared memory to the video stream according to the image synthesis parameters.

[0053] By configuring the image synthesis parameters and sending them to the graphics processor, and forwarding them to the video stream module by the graphics processor, the video stream module can synthesize the image and the video stream according to the image synthesis parameters. The synthesis mode of the image can be flexibly configured, the synthesis mode of the image can be enriched, the synthesis of multiple images and the video stream can be accurately controlled, and the number of images that can be added to the video stream can be increased.

[0054] Optionally, the image storage method further comprises: sending a video stream control instruction to the graphics processor, so that the graphics processor controls the operation of the video stream module.

[0055] The graphics processor is further configured to receive the video stream control instruction sent by the central processor, and control the video stream module, so that the video stream module performs the operation associated with the video stream. The video stream control instruction can include at least one of the following: a video stream start instruction, a video stream synthesis instruction, a video stream close instruction, etc. The video stream start instruction is used to control the video stream module to start generating and outputting the video stream. The video stream synthesis instruction is used to control the video stream module to add a synthesis link of adding the image to the video stream in the process of acquiring and outputting the video stream, so as to control the video stream module to start adding the data acquired from the shared memory to the video stream. The video stream close instruction is used to control the video stream module to stop generating the video stream.

[0056] By sending the video stream control instruction to the graphics processor, the graphics processor can control the video stream module to generate, synthesize and close the video stream, and accurately realize the synthesis of the video stream and the image.

[0057] The embodiment of the present application configures shared memory, and writes images into the shared memory by a central processor, so that a video stream module can read images from the shared memory as needed, and synthesizes the images and the video stream, omits the image carrying operation, decouples the image writing operation and the image synthesis operation, and the image writing operation does not need to be placed after the image carrying operation is completed, solves the problem that the image carrying operation and the image synthesis operation are highly coupled in the prior art, and the image synthesis efficiency is low and the cost is high, the image synthesis operation and the image writing operation are independent of each other, the image synthesis efficiency is improved, the image carrying operation is omitted, the image synthesis cost can be reduced, and the fixed display memory used in the traditional OSD technology is removed, and the shared memory of a size that can be configured is dynamically applied and released in time, and the memory resource use efficiency of the whole system is improved.

[0058] Embodiment two

[0059] Figure 2 For a flowchart of an image storage method in the embodiment two of the present application, the embodiment can be applicable to a case that a shared memory for storing images is allocated by a graphics processor, and images are read from the shared memory and added to a video stream, the method can be executed by the image storage device provided by the embodiment of the present application, the device can be realized in a software and / or hardware manner, and can be generally integrated into a computer device, and specifically a graphics processor.

[0060] The method of the embodiment specifically includes:

[0061] S210, allocating shared memory for at least one image to be written.

[0062] Corresponding shared memory can be dynamically allocated for each image in the memory, and the image is bound to the shared memory.

[0063] Descriptions not specifically introduced in the embodiment of the present application can refer to the descriptions of the above embodiments.

[0064] S220, obtaining shared memory addresses of the shared memories.

[0065] The shared memory address of the shared memory points to the shared memory.

[0066] S230, sending the shared memory addresses to a central processor, so that the central processor writes the images into corresponding shared memories, and the shared memories are used for a video stream module to extract the images and add the images to a video stream.

[0067] The shared memory address is sent to the central processor, and the central processor can draw an image in the shared memory pointed by the shared memory address. Meanwhile, the shared memory address is also sent to the video stream module, so that the video stream module obtains data from the shared memory and adds the data to the video stream.

[0068] Optionally, the method for allocating memory further comprises: receiving a shared memory application request sent by the central processor; extracting image attribute information of each image from the shared memory application request; determining the occupied space corresponding to each image according to the image attribute information of each image; and allocating shared memory for each image in the memory according to the occupied space corresponding to each image.

[0069] The occupied space capacity of an image can be calculated according to image attribute information. For example, the occupied space capacity of an image can be calculated according to image attribute information and a pre-configured occupied space capacity calculation formula. The image attribute information can include at least one of the following: size, color mode, depth information, etc. For example, the number of pixel points included in an image can be determined according to the size and the color mode, and the number of pixel points is directly proportional to the occupied space capacity. The richness of depth information determines the amount of information of a pixel point, and the amount of information is directly proportional to the occupied space capacity. The weight of each item can be configured, the product of the weight and each item can be calculated, and a weighted sum can be calculated to obtain the occupied space capacity.

[0070] In addition, the image attribute information can also include resolution, and the weight of the resolution can also be configured, the product of the weight and the resolution can be calculated, and a sum can be calculated to obtain the occupied space capacity.

[0071] By determining the occupied space corresponding to an image according to image attribute information and allocating shared memory correspondingly, the shared memory can be allocated according to the occupied space of an image, the storage resources can be reasonably configured, excessive allocation of shared memory can be avoided, storage resource waste can be avoided, and the utilization rate of storage resources can be improved.

[0072] Optionally, the method for allocating memory further comprises: receiving image synthesis parameters sent by the central processor; and forwarding the image synthesis parameters to the video stream module, so that the video stream module adds each image to the video stream according to the image synthesis parameters.

[0073] Optionally, the image synthesis parameters include at least one of the following: the shared memory address, an image adding position, an image size, and an adding order of multiple images.

[0074] Optionally, the method for allocating memory further comprises: receiving a video stream control instruction sent by the central processor, and controlling the video stream module to work according to the video stream control instruction.

[0075] The received video stream control instruction can be directly forwarded to the video stream module, so that the video stream module performs corresponding operation according to the video stream control instruction. The video stream control instruction can include at least one of the following: a video stream starting instruction, a video stream synthesizing instruction, a video stream closing instruction, and the like.

[0076] The embodiment of the present application allocates shared memory for an image to be written into the shared memory by a graphic processor, and sends the shared memory address to a central processor, so that the central processor writes the image into the allocated shared memory, and the video stream module directly reads data from the shared memory, which reduces the coupling of the image writing operation and the image synthesizing operation, makes the image synthesizing operation and the image writing operation independent of each other, improves the image synthesizing efficiency, omits the image carrying operation, reduces the image synthesizing cost, and speeds up the image synthesizing speed.

[0077] Embodiment three

[0078] Figure 3 Fig. 1 is a schematic diagram of an image storage device according to the embodiment three of the present application. The embodiment three is a corresponding device for implementing the image storage method provided in the above embodiments of the present application, which can be implemented in the form of software and / or hardware, and can be integrated into a computer device.

[0079] Correspondingly, the device of the embodiment can include:

[0080] The shared memory address obtaining module 310 is configured to obtain at least one shared memory address.

[0081] The image drawing module 320 is configured to write at least one image into the shared memory pointed by the corresponding shared memory address, so that the video stream module extracts each image from each shared memory and adds the image to the video stream.

[0082] The embodiment of the present application allocates shared memory, and writes the image into the shared memory by the central processor, so that the video stream module can read the image from the shared memory as needed, and synthesize the image and the video stream, which omits the image carrying operation, decouples the image writing operation and the image synthesizing operation, and makes the image writing operation not need to be placed after the image carrying operation is completed, thereby solving the problem of high coupling between the image carrying operation and the image synthesizing operation in the prior art, low image synthesizing efficiency, and high cost. The image synthesizing operation and the image writing operation are independent of each other, which improves the image synthesizing efficiency, omits the image carrying operation, reduces the image synthesizing cost, and removes the fixed display memory used in the traditional OSD technology, and updates to the dynamically applied and timely released shared memory with a configurable size, thereby improving the memory resource use efficiency of the entire system.

[0083] Further, the shared memory address obtaining module 310 comprises a shared memory application unit, configured to generate a shared memory application request according to image attribute information of each image, send the shared memory application request to a graphic processor, and receive at least one shared memory address fed back by the graphic processor, the shared memory address pointing to shared memory with a capacity matching the image attribute information of the corresponding image.

[0084] Further, the image storage device further comprises an image synthesis parameter configuration module, configured to send image synthesis parameters to the graphic processor, so that the graphic processor forwards the image synthesis parameters to the video stream module, and the video stream module adds each image into the video stream according to the image synthesis parameters.

[0085] Further, the image synthesis parameters comprise at least one of the shared memory address, an image adding position, an image size, and an adding order of multiple images.

[0086] Further, the image storage device further comprises a control instruction sending module, configured to send a video stream control instruction to the graphic processor, so that the graphic processor controls the video stream module to work.

[0087] The device described above can execute the method provided in the embodiments of the present application, and has the corresponding function components and beneficial effects of executing the method.

[0088] Embodiment Four

[0089] Figure 4 FIG. 1 is a schematic diagram of a memory allocation device in Embodiment Four of the present application. Embodiment Four is a device corresponding to the memory allocation method provided in the above embodiments of the present application, which can be realized in the form of software and / or hardware, and can be integrated into a computer device.

[0090] Correspondingly, the device of the present embodiment can comprise:

[0091] The memory allocation module 410 is configured to allocate shared memory for at least one image to be written;

[0092] The shared memory address obtaining module 420 is configured to obtain shared memory addresses of each shared memory;

[0093] The shared memory address sending module 430 is configured to send each shared memory address to a central processing unit, so that the central processing unit writes each image into the corresponding shared memory, and the shared memory is used for a video stream module to extract each image therefrom and add the image into a video stream.

[0094] The embodiment of the present application allocates shared memory for images to be written into the shared memory by the graphic processor, and sends the shared memory address to the central processor, so that the central processor writes the images into the allocated shared memory, and the video stream module directly reads data from the shared memory, reduces the coupling of the image writing operation and the image synthesis operation, makes the image synthesis operation and the image writing operation independent of each other, improves the image synthesis efficiency, omits the image carrying operation, reduces the image synthesis cost, and speeds up the image synthesis speed.

[0095] Further, the memory allocation module 410 comprises a shared memory dynamic allocation unit, configured to receive the shared memory application request sent by the central processor, extract the image attribute information of each image from the shared memory application request, determine the occupied space corresponding to each image according to the image attribute information of each image, and allocate shared memory for each image in the memory according to the occupied space corresponding to each image.

[0096] Further, the memory allocation device further comprises an image synthesis parameter forwarding module, configured to receive the image synthesis parameter sent by the central processor, and forward the image synthesis parameter to the video stream module, so that the video stream module sequentially adds each image into the video stream according to the image synthesis parameter.

[0097] Further, the image synthesis parameter comprises at least one of the shared memory address, an image adding position, an image size, and an adding order of multiple images.

[0098] Further, the memory allocation device further comprises a video stream control module, configured to receive the video stream control instruction sent by the central processor, and control the video stream module to work according to the video stream control instruction.

[0099] The above device can execute the method provided by the embodiment of the present application, has the corresponding function components and beneficial effects of executing the method.

[0100] Embodiment five

[0101] Figure 5 A structural schematic diagram of a computer device provided for the fifth embodiment of the present application. Figure 5 A block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present application is shown. Figure 5 The displayed computer device 12 is only an example, and should not bring any limitation to the function and use range of the embodiments of the present application.

[0102] As Figure 5As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 can include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 to the processing unit 16. The computer device 12 can be a device that is bus-connected.

[0103] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus (e.g., an Accelerated Graphics Port, or AGP bus) and a local bus using any of a variety of bus architectures (e.g., Industrial Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus).

[0104] The computer device 12 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by the computer device 12 and includes both volatile and non- volatile media, removable and non-removable media.

[0105] The system memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 can be used for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Figure 5 (not shown) that is typically used for reading from and writing to a non-removable, non-volatile magnetic media (e.g., a "hard drive"). Although not specifically shown, such computer system typically can further include other removable / non-removable, volatile / non-volatile computer system storage media including, but not limited to, a magnetic floppy disk, a Figure 5A disk drive, a floppy disk drive, and / or other mass storage device can be provided for in the example environment, which can be used to read from or write to a removable, nonvolatile magnetic or optical disk, and to move data among computer devices 12 and some computing devices 12 and the removable disk. In these instances, each drive can be connected to bus 18 by one or more data media interfaces. System memory 28 can include at least one program product having a set (e.g., at least one) of program components 42 configured to carry out the functions of embodiments of the application.

[0106] Program / utility 40, having a set (at least one) of program components 42, can be stored in system memory 28 and / or floppy disk drive as exemplary computer readable media for purposes of loading in the general-purpose registers and / or caches of the computer device 12. Program components 42 include, but are not limited to, an operating system, one or more applications, other program components, and program data, each or some combination thereof, which can include implementation of a networking environment. Program components 42 are generally executed by processing unit 16 to implement the functions and / or method steps of embodiments of the application.

[0107] Computer device 12 can also communicate with one or more external devices 14 such as a keyboard or a pointing device, displays 24, etc.; one or more devices that enable a user to interact with computer device 12; and / or any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 22. Still yet, computer device 12 can communicate with one or more networks such as a local area network (LAN), a wide area network (WAN), and / or the Internet through network adapter 20. As an example, communications can be achieved via Input / Output (I / O) adapters 22, network interface card(s) 20, and communications Figure 5 Other hardware and / or software components can also be used with computer device 12. Such components, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc., can be

[0108] Processing unit 16 can execute instructions, for example, stored in system memory 28, to implement various functionality and / or methods of embodiments of the application, such as the image storage method or memory allocation method.​

[0109] Embodiment six

[0110] Embodiment six of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the image storage method provided by all embodiments of the present application.

[0111] That is, the program is executed by the processor to implement: obtaining at least one shared memory address; writing at least one image into the shared memory pointed by the corresponding shared memory address, so that the video stream module extracts each of the images from each of the shared memories and adds them to the video stream.

[0112] Or implement the memory allocation method provided by all embodiments of the present application, the program is executed by the processor to implement: allocating shared memory for at least one image to be written; obtaining the shared memory address of each of the shared memories; sending each of the shared memory addresses to the central processor, so that the central processor writes each of the images into the corresponding shared memory, and the shared memory is used by the video stream module to extract each of the images and add them to the video stream.

[0113] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus.

[0114] These computer readable program instructions can be provided to a computer of an apparatus, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0115] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0116] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0117] Embodiment Seven

[0118] Figure 6a For a flowchart of an image synthesis method in the seventh embodiment of the present application, the embodiment can be applicable to the case that a central processing unit writes an image in a shared memory, a video stream module reads the image from the shared memory, and the image is synthesized with a video stream. The method can be executed by the image synthesis system provided by the embodiment of the present application. The apparatus can be realized by software and / or hardware, and can be integrated in a computer device, as shown in FIG. 7. The method of the embodiment specifically includes: Figure 6a

[0119] S701, obtaining at least one shared memory address by a central processing unit, and writing at least one image in a shared memory pointed by the at least one shared memory address.

[0120] The central processing unit is configured to draw the image in the shared memory.

[0121] ​The central processor is further configured to generate a shared memory application request according to image attribute information of each of the images, send the shared memory application request to the graphics processor, and receive at least one shared memory address fed back by the graphics processor, the capacity of the shared memory pointed to by the shared memory address matching the image attribute information of the corresponding image.

[0122] The central processor is further configured to send image synthesis parameters to the graphics processor, so that the graphics processor forwards the image synthesis parameters to the video stream module, and the video stream module sequentially adds each of the images to the video stream according to the image synthesis parameters, the image synthesis parameters including at least one of the shared memory address, an image addition position, an image size, and an addition order of multiple images.

[0123] The central processor is further configured to send a video stream control instruction to the graphics processor, so that the graphics processor controls the operation of the video stream module.

[0124] S702, the video stream module extracts stored data from the shared memory pointed to by each of the shared memory addresses and adds the stored data to the video stream, the stored data including the images.

[0125] The video stream module is configured to read stored data from the shared memory. The stored data is data stored in the shared memory. The stored data includes a complete image, a partial image, or nothing. When the central processor completes drawing, the stored data includes a complete image. When the central processor is drawing, the stored data includes a partial image. When the central processor is not drawing, the stored data is nothing.

[0126] The reading operation of the video stream module and the drawing operation of the central processor are independent of each other. The reading operation of the video stream module does not need to wait for the central processor to complete drawing, so that the drawing operation of the central processor does not affect the acquisition and output operation of the video stream module on the video stream, so that the output quality of the video stream is not affected, and the output stability of the video stream is improved.

[0127] Optionally, the image synthesis method further includes: allocating shared memory for at least one image to be written by the graphics processor, and sending shared memory addresses of each of the shared memory to the central processor and the video stream module.

[0128] The graphics processor is configured to allocate shared memory for each image, and provide the shared memory addresses to the central processor and the video stream module, so that the central processor writes each image into the shared memory pointed to by the corresponding shared memory address, and so that the video stream module extracts stored data from the shared memory pointed to by each of the shared memory addresses and adds the stored data to the video stream.

[0129] The graphics processor is further configured to receive a shared memory application request sent by the central processor, extract image attribute information of each image from the shared memory application request, determine the occupied space corresponding to each image according to the image attribute information of each image, and allocate shared memory for each image in the memory according to the occupied space corresponding to each image.

[0130] The graphics processor is further configured to receive a video stream control instruction sent by the central processor, and control the video stream module.

[0131] In one application scenario, as shown in Figure 6b The image synthesis system can include a central processor 710, a graphics processor 720, and a video stream module 730. The central processor 710, the graphics processor 720, the video stream module 730, and the shared memory are hardware modules. The image rendering main program and the video stream processing program are software modules.

[0132] For example, the central processor 710 can be an Advanced Reduced Instruction Set Computing Machine (ARM) processor. The central processor 710 runs an image rendering main program. For example, the image is an OSD image. The image rendering main program includes a video stream control module and an image rendering module. The video stream control module is configured to output a control instruction of the video stream module to control the video stream module to perform corresponding operations, which can include starting, pausing, and playing operations. The image rendering module is configured to render the image and write it into the pre-configured shared memory. In addition, the image rendering main program can further include the aforementioned shared memory address acquisition module.

[0133] The graphics processor 720 runs a video stream processing program, which includes a shared memory management module and a video stream processing module. The shared memory management module is configured to allocate shared memory for the central processor 710. The video stream processing module is configured to control the video stream module to perform corresponding operations according to the control instruction sent by the central processor 710, which includes the synthesis operation of the image and the video frame of the video stream. In addition, the shared memory management module can further include the aforementioned memory allocation module, shared memory address acquisition module, and shared memory address sending module.

[0134] The video stream processing program running on the graphics processor 720 has the ability to directly access the registers of the video stream module, and can control the working state and set parameters of the video stream module; turn on or off the image synthesis function switch; bind the memory address to the video stream module for synthesis. For example, the video stream module can be a pipeline. When the video stream module is working, if the image synthesis function switch is in the on state, the OSD data in the bound shared memory address will be synthesized into each video frame of the video stream according to the pixels before being output to the video stream output device.

[0135] The central processor 710 and the graphics processor 720 perform data transmission through inter-core communication. The central processor 710 applies (①) shared memory to the graphics processor 720, and the graphics processor 720 allocates (①) shared memory to the central processor 710. The central processor 710 configures (②) the allocated shared memory as shared memory for storing images. The graphics processor 720 binds (②) the allocated shared memory with images and notifies the video stream module 730. The video stream module 730 binds (②) the allocated shared memory with images. The central processor 710 writes (③) images in the shared memory. The video stream module 730 reads images from the pre-bound shared memory and synthesizes (④) them with the currently acquired video stream.

[0136] As shown in Figure 6c The video stream image synthesis method implemented by the synthesis system can specifically include:

[0137] S740, the image rendering main program starts the video stream.

[0138] The image rendering main program starts the video stream and generates a control instruction for starting the video stream.

[0139] The central processor sends a video stream control instruction to the graphics processor, and the current video stream control instruction is a video stream start instruction.

[0140] S741, the image rendering main program sends a video stream start instruction to the video stream processing program to start the video stream.

[0141] S742, the video stream processing program creates a video stream.

[0142] The video stream processing program creates a process and runs a program of the video stream module. When the program is executed, the video stream is acquired and output to the display for display.

[0143] The graphics processor controls the video stream module to acquire the video stream, which includes generating the video stream or downloading the video stream.

[0144] S743, the video stream processing program returns a result to the image rendering main program.

[0145] The result includes a video stream processing result and a state event. For example, the video stream processing program creates a video stream, and the video stream processing program returns a result of successful creation of the video stream; the video stream processing program fails to create the video stream, and the video stream processing program returns a result of failed creation of the video stream.

[0146] S744, the image drawing main program applies for a shared memory block to the video stream processing program.

[0147] The image drawing main program generates a shared memory application request according to image attribute information of the at least one image to be drawn, and sends the shared memory application request to the video stream processing program.

[0148] S745, the video stream processing program allocates the shared memory block.

[0149] The video stream processing program determines the occupied space corresponding to each image according to the image attribute information of each image, allocates shared memory for each image in the memory according to the occupied space corresponding to each image, and obtains a shared memory address of each shared memory.

[0150] S746, the video stream processing program returns the shared memory block address to the image drawing main program.

[0151] The graphics processor sends each shared memory address to the central processor. The central processor obtains at least one shared memory address.

[0152] S747, the image drawing main program sends image synthesis parameters to the video stream processing program.

[0153] The central processor sends the image synthesis parameters to the graphics processor, so that the graphics processor forwards the image synthesis parameters to the video stream module. The image synthesis parameters include at least one of the following: a shared memory address, an image addition position, an image size, and an addition order of multiple images, etc. The image size includes a length and a width.

[0154] S748, the video stream processing program sends the image synthesis parameters to the video stream module.

[0155] S749, the video stream processing program returns a result to the image drawing main program.

[0156] The result is an image synthesis parameter configuration result. For example, the video stream processing program successfully sends the image synthesis parameters, and the video stream processing program returns a result of successful configuration of the image synthesis parameters; the video stream processing program fails to successfully send the image synthesis parameters, and the video stream processing program returns a result of failed configuration of the image synthesis parameters.

[0157] S750, the image drawing main program sends a video stream synthesis instruction to the video stream processing program to start the image synthesis function.

[0158] The central processing unit sends a video stream control instruction to the graphic processing unit, and the current video stream control instruction is a video stream synthesis instruction.

[0159] In S751, the video stream processing program controls the video stream module to add an image synthesis link.

[0160] The adding of the image synthesis link is actually a step of synthesizing the video stream and the image between the acquisition of the video stream by the video stream module and the output of the video stream.

[0161] In S752, the video stream processing program returns a result to the image drawing main program.

[0162] The result is the result of the adding of the image synthesis link. For example, the video stream processing program returns a result of the successful adding of the image synthesis link when the video stream module successfully adds the image synthesis link, and returns a result of the unsuccessful adding of the image synthesis link when the video stream processing program fails to add the image synthesis link.

[0163] In S753, the image drawing main program draws an image on the shared memory block.

[0164] The central processing unit draws at least one image on the shared memory block, i.e., the central processing unit writes each image into the shared memory. The image is an OSD image.

[0165] It should be noted that S750-S752 and S753 are independent of each other, and the order of S750-S752 and S753 is not important. The flow of S750-S752 and S753 is only shown by way of example. S753 can be executed before S750-S752. When the image drawing main program does not draw or does not completely draw an image, the video stream module can still synthesize the video frame according to the read storage data in the bound shared memory address, and the storage data can include a complete image, a partial image or nothing.

[0166] In S754, the image drawing main program sends a video stream closing instruction to the video stream processing program to close the video stream.

[0167] The central processing unit sends a video stream closing instruction to the graphic processing unit, and the current video stream control instruction is the video stream closing instruction.

[0168] In S755, the video stream processing program controls the video stream module to destroy the video stream.

[0169] The video stream module stops generating or downloading the video stream, destroys the video stream, releases the resources associated with the video stream, such as the process and the cache occupied by the video stream, and also releases the shared memory.

[0170] In S756, the video stream processing program returns a result to the image drawing main program.

[0171] The result is a video stream closing result. For example, the video stream module successfully closes the video stream, and the video stream handler returns a result of successful closing of the video stream; the video stream handler fails to successfully close the video stream, and the video stream handler returns a result of failure to close the video stream. Closing the video stream can refer to destroying the video stream and releasing shared memory, processes and caches associated with the video stream, and the like.

[0172] The embodiment of the present application configures image drawing and video stream processing as two independent programs for execution and executes them asynchronously, so as not to affect each other. The video processing program only needs to configure the shared memory address, image width and height, and position information to the video stream module, so that the video stream module adds the image stored in the shared memory to the specified position of the video frame according to the bound shared memory address before the video stream is output. Since drawing and synthesis are independent of each other, the speed of image drawing will not affect the fluency of the video stream, and the processing of the video stream will not affect the efficiency of image drawing. In addition, the shared memory for storing images is dynamically allocated, and can be used by other modules when images are not needed. The space capacity is set according to the actual use, which is flexible and saves resources. At the same time, in the case that the idle physical memory is sufficient to carry multiple images, the number of images can be configured to be multiple, and the number of image layers added to the video frame is unlimited.

[0173] Note that the above are only the preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. An image storage method, characterized in that, Applied to central processing units, including: Obtain at least one shared memory address; At least one image is written to the shared memory pointed to by the corresponding shared memory address, so that the video streaming module extracts each image from each of the shared memories and adds it to the video stream for output to the display; the shared memory is allocated by the graphics processor; the image is an on-screen image; The step of writing at least one image into the shared memory pointed to by the corresponding shared memory address includes: The corresponding image will be drawn in the shared memory pointed to by the corresponding shared memory address; wherein, when the drawing is completed, the corresponding shared memory stores the complete image; during the drawing process, the corresponding shared memory stores a partial image; when the drawing is not completed, the corresponding shared memory does not store an image; when the image is not drawn or is not completely drawn, the video stream module can be used to read the corresponding stored data from each of the shared memories and synthesize it with the video stream; The central processing unit (CPU) and the graphics processing unit (GPU) transmit data via inter-core communication. The CPU requests shared memory from the GPU, and the GPU allocates shared memory to the CPU. The CPU configures the allocated shared memory as shared memory for storing images. The GPU binds the allocated shared memory to the images and notifies the video streaming module. The video streaming module binds the allocated shared memory to the images. The CPU writes images into the shared memory, and the video streaming module reads images from the pre-bound shared memory and combines them with the currently acquired video stream.

2. The method according to claim 1, characterized in that, The process of obtaining the shared memory address includes: Based on the image attribute information of each image, a shared memory request is generated; Send a shared memory request to the graphics processor; The system receives at least one shared memory address from the graphics processor, wherein the capacity of the shared memory pointed to by the shared memory address matches the image attribute information of the corresponding image.

3. The method according to claim 1, characterized in that, Also includes: Image compositing parameters are sent to the graphics processor, so that the graphics processor forwards the image compositing parameters to the video streaming module, and the video streaming module adds each of the images to the video stream sequentially according to the image compositing parameters.

4. The method according to claim 3, characterized in that, The image synthesis parameters include at least one of the following: the shared memory address, the image addition location, the image size, and the addition order of multiple images.

5. The method according to claim 1, characterized in that, Also includes: Send video stream control commands to the graphics processor so that the graphics processor controls the video stream module to operate.

6. A memory allocation method, characterized in that, Applied to graphics processors, including: Allocate shared memory for at least one image to be written; Obtain the shared memory address of each of the aforementioned shared memory locations; The shared memory addresses are sent to the central processing unit (CPU) so that the CPU writes each image into the corresponding shared memory. The shared memory is used by the video streaming module to extract each image and add it to the video stream for display on a monitor. The shared memory is allocated by the graphics processor (GPU). The image is an on-screen display image. The CPU is used to: draw the corresponding image in the shared memory pointed to by the corresponding shared memory address; wherein, when drawing is complete, the corresponding shared memory stores the complete image; during drawing, the corresponding shared memory stores a partial image; when not drawing, the corresponding shared memory does not store an image; when the image is not drawn or not fully drawn, the video streaming module can read the corresponding stored data from each of the shared memories and combine it with the video stream. The central processing unit (CPU) and the graphics processing unit (GPU) transmit data via inter-core communication. The CPU requests shared memory from the GPU, and the GPU allocates shared memory to the CPU. The CPU configures the allocated shared memory as shared memory for storing images. The GPU binds the allocated shared memory to the images and notifies the video streaming module. The video streaming module binds the allocated shared memory to the images. The CPU writes images into the shared memory, and the video streaming module reads images from the pre-bound shared memory and combines them with the currently acquired video stream.

7. The method according to claim 6, characterized in that, The allocation of shared memory for at least one image to be written includes: Receive the shared memory request sent by the central processing unit; Extract the image attribute information of each image from the shared memory request; Based on the image attribute information of each image, determine the space occupied by each image; Based on the space occupied by each image, shared memory is allocated in memory for each image.

8. The method according to claim 6, characterized in that, Also includes: Receive image synthesis parameters sent by the central processing unit; The image compositing parameters are forwarded to the video stream module, so that the video stream module adds each of the images to the video stream sequentially according to the image compositing parameters.

9. The method according to claim 8, characterized in that, The image synthesis parameters include at least one of the following: the shared memory address, the image addition location, the image size, and the addition order of multiple images.

10. The method according to claim 6, characterized in that, Also includes: The system receives video stream control instructions sent by the central processing unit and controls the video stream module to work according to the video stream control instructions.

11. An image storage device, characterized in that, Configured in the central processing unit, including: A shared memory address acquisition module is used to acquire at least one shared memory address; An image rendering module is used to write at least one image into shared memory pointed to by a corresponding shared memory address, so that the video streaming module can extract each image from each of the shared memories and add it to the video stream for output to the display; the shared memory is allocated by a graphics processor; the image is an on-screen image. The image rendering module is used for: The corresponding image will be drawn in the shared memory pointed to by the corresponding shared memory address; wherein, when the drawing is completed, the corresponding shared memory stores the complete image; during the drawing process, the corresponding shared memory stores a partial image; when the drawing is not completed, the corresponding shared memory does not store an image; when the image is not drawn or is not completely drawn, the video stream module can be used to read the corresponding stored data from each of the shared memories and synthesize it with the video stream; The central processing unit (CPU) and the graphics processing unit (GPU) transmit data via inter-core communication. The CPU requests shared memory from the GPU, and the GPU allocates shared memory to the CPU. The CPU configures the allocated shared memory as shared memory for storing images. The GPU binds the allocated shared memory to the images and notifies the video streaming module. The video streaming module binds the allocated shared memory to the images. The CPU writes images into the shared memory, and the video streaming module reads images from the pre-bound shared memory and combines them with the currently acquired video stream.

12. A memory allocation device, characterized in that, Configured on the graphics processor, including: A shared memory allocation module is used to allocate shared memory for at least one image to be written; A shared memory address acquisition module is used to acquire the shared memory address of each of the shared memory locations; A shared memory address sending module is used to send each of the shared memory addresses to the central processing unit (CPU), so that the CPU writes each of the images into the corresponding shared memory. The shared memory is used by the video streaming module to extract each of the images and add them to the video stream for output to the display. The shared memory is allocated by the graphics processor (GPU). The images are on-screen display images. The CPU is used to: draw the corresponding images in the shared memory pointed to by the corresponding shared memory addresses; wherein, when drawing is complete, the corresponding shared memory stores the complete image; during drawing, the corresponding shared memory stores a partial image; when not drawing, the corresponding shared memory does not store an image; when the image is not drawn or not fully drawn, the video streaming module can be used to read the corresponding stored data from each of the shared memories and synthesize it with the video stream. The central processing unit (CPU) and the graphics processing unit (GPU) transmit data via inter-core communication. The CPU requests shared memory from the GPU, and the GPU allocates shared memory to the CPU. The CPU configures the allocated shared memory as shared memory for storing images. The GPU binds the allocated shared memory to the images and notifies the video streaming module. The video streaming module binds the allocated shared memory to the images. The CPU writes images into the shared memory, and the video streaming module reads images from the pre-bound shared memory and combines them with the currently acquired video stream.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image storage method as described in any one of claims 1-5, or the memory allocation method as described in any one of claims 6-10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image storage method as described in any one of claims 1-5, or the memory allocation method as described in any one of claims 6-10.

15. An image synthesis method, characterized in that, include: Obtain at least one shared memory address through the central processing unit, and write at least one image into the shared memory pointed to by the corresponding shared memory address; The video streaming module extracts stored data from the shared memory pointed to by each of the shared memory addresses and adds it to the video stream for output to the display. The shared memory is allocated by the graphics processor. The image is an on-screen image, and the stored data includes the image. The central processing unit is used to: draw the corresponding image in the shared memory pointed to by the corresponding shared memory address; wherein, when drawing is complete, the corresponding shared memory stores the complete image; during drawing, the corresponding shared memory stores a partial image; when not drawing, the corresponding shared memory does not store an image; when the image is not drawn or not fully drawn, the video streaming module can be used to read the corresponding stored data from each of the shared memory addresses and combine it with the video stream. The central processing unit (CPU) and the graphics processing unit (GPU) transmit data via inter-core communication; the CPU requests shared memory from the GPU, the GPU allocates shared memory to the CPU, the CPU configures the allocated shared memory as shared memory for storing images, and the GPU binds the allocated shared memory to the images and notifies the video streaming module. The video streaming module is used to bind the allocated shared memory to the image; the central processing unit is used to write the image into the shared memory; and the video streaming module is used to read the image from the pre-bound shared memory and synthesize it with the currently acquired video stream.

16. The method according to claim 15, characterized in that, Also includes: The graphics processor allocates shared memory for at least one image to be written, and sends the shared memory address of each shared memory to the central processing unit and the video streaming module.

Images