A layer synthesis method, computer device and computer readable storage medium

Abstract

The present invention provides a layer synthesis method, a computer device and a computer-readable storage medium, the method comprising obtaining multiple layers of an image and calculating the area size of each layer; reordering the multiple layers: obtaining the pending center of gravity of the multiple layers; reordering the multiple layers according to each pending center of gravity, the layers with smaller areas are closer to the pending center of gravity; calculating the sum of the moments of the multiple layers corresponding to each pending center of gravity, taking the pending center of gravity with the best sum of moments as the target center of gravity, and reordering the multiple layers according to the target center of gravity; sending a preset number of layers with the largest areas to a layer carrier, and sending the remaining layers to an image processor, and the image processor synthesizing the remaining layers. The present invention also provides a computer device and a computer-readable storage medium for implementing the above method. The present invention can reduce the power consumption of an image processor.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a layer synthesis method, a computer device and a computer-readable storage medium for implementing the method. Background Art

[0002] Many smart electronic devices have the function of image capture. For example, smart phones, tablet computers, driving recorders, etc. are all equipped with camera devices, which are usually equipped with CMOS sensors to obtain images. After the image is captured, it needs to be output to a display device such as a display screen for display. When the image is displayed, an overlay layer is often added to the image, such as adding information such as the shooting date, shooting time, and photographer, or specific elements in the image need to be covered to avoid the leakage of personal privacy.

[0003] To this end, after the image sensor outputs the original image, the display module needs to add a large number of layers to the original image. Each layer may be used to add information such as the shooting date, or to add mosaics to cover personal privacy information in the image. Therefore, after the camera application submits an image containing multiple layers to the display module, the display module needs to synthesize the various layers and then send them to the display device for display. When synthesizing the layers, the display module generally uses an image carrier and an image processor. Usually, the display module first allocates the various layers submitted by the application to the layer carrier of the display controller as much as possible. The layer carrier is used to scan the layer data submitted by the application to the display controller, and the remaining layers that cannot be allocated to the layer carrier of the display controller are sent to the image processor for synthesis processing.

[0004] Since the number of image carriers of electronic devices is limited, but the number of layers submitted by the application is uncertain and may exceed the number of layer carriers, the remaining layers need to be sent to the image processor for synthesis to form an image, and then allocated to a hardware layer of the display controller for display.

[0005] Since the image processor needs to calculate the image when synthesizing the image, the power consumption is very large, especially when synthesizing a large layer, the power consumption is even greater. In order to reduce the power consumption of the image processor, the larger layer is often assigned to the image carrier for processing, while the smaller layer is assigned to the image processor for processing. However, even in this operation, there is still the problem of reasonable allocation between the multiple layers synthesized by the image processor to further reduce the power consumption of the image processor.

[0006] The image processor usually synthesizes multiple layers directly according to the upper and lower relationships of the multiple layers, but this method often requires the image processor to process a layer with a larger area, resulting in excessive power consumption of the image processor. An existing processing method is to reorder the layers using the "vertex method".

[0007] See also Figure 1 When the layers are reordered using the “vertex method”, a base point 10 is first set, for example, the base point 10 is located at the top or bottom of the multiple layers 11, and then the multiple layers 11 are reordered according to the size of the area, that is, the smaller the area of ​​the layer 11, the closer to the base point 10, and the larger the area of ​​the layer 11, the farther away from the base point 10.

[0008] See also Figure 2 and Figure 3 , it is assumed that the image processor needs to synthesize 6 layers, namely layers 21, 22, 23, 24, 25, and 26, where layer 21 covers layer 22, and the area of ​​layer 21 is larger than the area of ​​layer 22, layer 23 covers layers 24 and 25, layers 24 and 25 cover layer 26, and there is no overlapping relationship between layers 21 and 22 and layers 23, 24, 25, and 26.

[0009] In the Z-axis direction, layer 22 is at the bottom of all layers. After reordering the layers using the "vertex method", it is assumed that the base point is at the top of all layers, such as Figure 4 As shown, the covering relationship between the layers with covering relationship cannot be changed, so layer 21 still needs to be located above layer 22, but because layer 22 has a smaller area, layer 22 is moved upward to be located above layer 23. In addition, before reordering, layer 24 is located below layer 25, and there is no covering relationship between layer 24 and layer 25, so after reordering, layer 24 is located above layer 25.

[0010] However, this reordering method cannot move layers with smaller areas together. When the image processor synthesizes multiple layers, the amount of calculation is still large, resulting in high power consumption of the image processor. Summary of the invention

[0011] The main purpose of the present invention is to provide a layer synthesis method which can effectively reduce the power consumption of a layer processor in synthesizing layers.

[0012] Another object of the present invention is to provide a computer device for implementing the above-mentioned layer synthesis method.

[0013] Another object of the present invention is to provide a computer-readable storage medium for implementing the above-mentioned layer synthesis method.

[0014] To achieve the main purpose of the present invention, the layer synthesis method provided by the present invention includes obtaining multiple layers of an image and calculating the area size of each layer; and reordering the multiple layers: obtaining the pending center of gravity of the multiple layers, in the stacking direction of the multiple layers, the pending center of gravity is located above at least one layer and below at least one layer; reordering the multiple layers according to each pending center of gravity, and the layer with a smaller area is closer to the pending center of gravity; calculating the sum of the moments of the multiple layers corresponding to each pending center of gravity, taking the pending center of gravity with the optimal sum of the moments as the target center of gravity, and reordering the multiple layers with the target center of gravity; sending a preset number of layers with the largest areas to a layer carrier, and sending the remaining layers to image processing, and the image processor synthesizes the remaining layers.

[0015] It can be seen from the above scheme that multiple layers are reordered with the target center of gravity as the base point. The layers with smaller areas in the sorted multiple layers are concentrated near the target center of gravity, and the target center of gravity is located in the middle of the multiple layers, rather than at the top or bottom. In this way, the layers with smaller areas can be concentrated together, which is conducive to the synthesis of multiple layers.

[0016] A preferred solution is that reordering multiple layers based on the target center of gravity includes: obtaining the covering relationship between each layer, if there is an covering relationship between the layers, when reordering the layers, the covering relationship between the layers is not changed.

[0017] It can be seen from this that reordering the layers does not change the covering relationship between the layers, which can avoid changing the covering relationship between the layers after reordering, causing the final output image to be different from the user setting.

[0018] A further solution is that taking the undetermined center of gravity with the optimal total moment as the target center of gravity includes: taking the undetermined center of gravity with the minimum total moment as the target center of gravity.

[0019] Since the smaller the moment is, the closer the layer is to the center, the total moment corresponding to the undetermined center of gravity is the smallest, which means that in the sorting of multiple layers corresponding to the undetermined center of gravity, the layers with the smallest area are basically moved together, which is beneficial to reduce the amount of calculation of the image processor, thereby reducing the power consumption of the image processor.

[0020] A further solution is that calculating the moment between the layer and the to-be-determined center of gravity includes: determining a target distance between the layer and the to-be-determined center of gravity, the moment between the layer and the to-be-determined center of gravity being the product of the area of ​​the layer and the target distance.

[0021] It can be seen that the moment of the layer is calculated according to the area of ​​the layer and the target distance. The calculation method of the moment is simple and the calculation amount is small, which is conducive to improving the efficiency of layer reordering.

[0022] A further solution is that determining the target distance between the layer and the to-be-determined center of gravity includes: calculating the number of layers between the to-be-determined center of gravity and the layer, and increasing the target distance by one unit for each layer.

[0023] It can be seen that the target distance is a virtual distance, that is, the size of the target distance is determined according to the number of intervals between layers, which simplifies the difficulty of calculating the target distance.

[0024] A further solution is that the distance between the layer closest to the undetermined centroid and the undetermined centroid is one unit.

[0025] It can be seen that calculating the target distance corresponding to each layer based on the layer closest to the undetermined center of gravity and determining the target distance calculation method of the layer closest to the undetermined center of gravity can reduce the difficulty of calculating the target distance of each layer.

[0026] A further solution is that obtaining the pending centroids of multiple layers includes: setting any two adjacent layers as the pending centroid.

[0027] It can be seen that the determination of the pending center of gravity is very simple, thereby simplifying the amount of calculation for obtaining multiple pending centers of gravity and also avoiding the occurrence of missing the pending center of gravity.

[0028] To achieve the above-mentioned other purpose, the computer device provided by the present invention includes a processor and a memory, and the memory stores a computer program. When the computer program is executed by the processor, each step of the above-mentioned layer synthesis method is implemented.

[0029] To achieve another of the above-mentioned purposes, the present invention provides a computer program stored on a computer-readable storage medium, and when the computer program is executed by a processor, each step of the above-mentioned layer synthesis method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a diagram of the existing layer reordering.

[0031] Figure 2 This is a diagram of the existing layers before reordering.

[0032] Figure 3 A diagram showing another perspective of the layers before the existing layers are reordered.

[0033] Figure 4 This is a schematic diagram of the layers after the existing layers are reordered.

[0034] Figure 5 It is a flow chart of an embodiment of the layer synthesis method of the present invention.

[0035] Figure 6 It is a schematic diagram of reordering multiple layers in an embodiment of the layer synthesis method of the present invention.

[0036] Figure 7 It is a schematic diagram of multiple to-be-determined centers of gravity in an embodiment of the layer synthesis method of the present invention.

[0037] Figure 8 Schematic diagram of target distances of various layers in an embodiment of the layer synthesis method of the present invention.

[0038] Fig. 9 It is a schematic diagram of each layer after the layers are reordered in the layer synthesis method embodiment of the present invention.

[0039] The present invention is further described below in conjunction with the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0040] The layer synthesis method of the present invention is applied to an intelligent electronic device. Preferably, the intelligent electronic device is provided with a camera, such as a camera, etc. The camera has an image sensor such as a CMOS, etc. The intelligent electronic device uses the camera to obtain an original image, and adds multiple layers of layers on the basis of the original image. This method is a method of synthesizing multiple layers using an image processor. Preferably, the intelligent electronic device is provided with a processor and a memory, and a computer program is stored in the memory. The processor implements the above-mentioned layer synthesis method by executing the computer program.

[0041] Layer synthesis method embodiment:

[0042] After the image sensor acquires the original image, various information can be added to the original image according to user settings, that is, multiple layers need to be formed on the original image. Each layer can be used to add a certain information, such as shooting time, etc. The layer can also be used to block certain information in the image, such as information involving personal privacy.

[0043] See also Figure 1 , multiple layers of the image are synthesized, firstly step S1 is executed to obtain multiple layers of the image, and the area of ​​each layer is calculated. For example, the initial image output by the image sensor is obtained, and multiple layers generated according to the information set by the user are obtained at the same time, such as a masking layer for displaying the shooting time, shooting date or shielding personal privacy information. After obtaining multiple layers, the area of ​​the obtained multiple layers is calculated, that is, the pixel size of each layer is calculated, and the layer with more pixels has a larger area, and the layer with fewer pixels has a smaller area.

[0044] After acquiring multiple layers, the multiple layers are reordered so that layers with smaller areas are gathered together, thereby reducing power consumption when synthesizing the layers. It should be noted that if the number of layers that the image processor needs to process is small, for example, only two layers are needed, then the layers do not need to be reordered. Only when the number of layers to be processed is large, for example, the number of layers is more than three, then the layers need to be reordered.

[0045] Different from the prior art which uses the "vertex method", this embodiment uses the "center of gravity method" to reorder the layer processes. Figure 6 When using the "center of gravity method" to reorder the layer process, it is necessary to set the "center of gravity" of multiple layers. The "center of gravity" is located in the middle of the multiple layers, that is, in the direction of layer stacking, the "center of gravity" is neither located at the top of the multiple layers nor at the bottom of the multiple layers.

[0046] For example, the center of gravity 30 is set to be located in the middle of the multiple layers 31, a part of the layers 31 are located above the center of gravity 30, and another part of the layers 31 are located below the center of gravity 30. After determining the position of the center of gravity 30, the layers 31 are reordered, that is, the up-and-down relationship of the multiple layers 31 is changed. Specifically, the smaller the area of ​​the layer 31, the closer to the center of gravity 30, and the larger the area of ​​the layer 31, the farther away from the center of gravity 30. Of course, when reordering the multiple layers 31, the overlapping relationship between the multiple layers 31 cannot be changed, that is, if there is an overlapping relationship between two layers, the up-and-down relationship between the two layers cannot be changed.

[0047] In order to determine the appropriate position of the center of gravity 30, this embodiment obtains the possible positions of the center of gravity in advance, and determines the quality of each possible center of gravity by calculating the parameters corresponding to each possible center of gravity, thereby determining the final center of gravity. In this embodiment, the possible center of gravity is called the "undetermined center of gravity", and the final determined center of gravity is the "target center of gravity".

[0048] Therefore, it is necessary to execute step S2 to obtain multiple pending centroids, each of which can be a position between any two adjacent layers, such as Figure 7 As shown, the area between any two adjacent layers is used as the pending center of gravity. Therefore, each pending center of gravity is located below at least one layer and above at least one layer. Since the target center of gravity is one of the pending centers of gravity, the target center of gravity finally determined is also located below at least one layer and above at least one layer.

[0049] Figure 7In the example, a pending center of gravity is set between any two layers, for example, a pending center of gravity 41 is set between layers 51 and 52, a pending center of gravity 42 is set between layers 52 and 53, and so on. It should be noted that the pending center of gravity only needs to be located between two adjacent layers. On the plane where the layers are located, the pending center of gravity can be located at any position above the plane. It can be understood that the pending center of gravity is a virtual center of gravity, that is, it is assumed that there is a gap between multiple layers. In fact, there is no gap between multiple layers, but it is artificially set that there is a gap between two layers.

[0050] After determining multiple pending centers of gravity, step S3 is executed to reorder the layers according to the set pending centers of gravity. Specifically, for each pending center of gravity, a reordering operation is performed on all layers. During the reordering, the layers with smaller areas are closer to the pending center of gravity, and the layers with larger areas are farther away from the pending center of gravity. In addition, if there is an overlay relationship between two layers, that is, one layer overlaps with another layer, the upper and lower relationship between the two layers cannot be changed.

[0051] After reordering multiple layers based on a certain undetermined center of gravity, step S4 is executed to calculate the sum of the moments of each layer after reordering based on the undetermined center of gravity. When calculating the moment of a layer, it is necessary to calculate the target distance between the layer and the undetermined center of gravity, and then calculate the product of the area of ​​the layer and the target distance. Figure 8 , the target distance between the layer and the pending center of gravity is the number of layers between the pending center of gravity and the layer + 1. For example, if a layer is closest to the pending center of gravity, the target distance of the layer is 1 unit. If there is one more layer between a layer and the pending center of gravity, the target distance increases by 1 unit. For example, 1 unit is 1.

[0052] It can be understood that layers S(0) and Q(0) are the layers closest to the undetermined centroid. Therefore, the target distances L(0) of layers S(0) and Q(0) are both 1. There is layer S(0) between layer S(1) and the undetermined centroid, and there is layer Q(0) between layer Q(1) and the undetermined centroid. Therefore, the target distances L(1) of layers S(1) and Q(1) are both 2. Similarly, the target distance L(n) of layer S(n) is n+1, and the target distance L(m) of layer Q(m) is m+1.

[0053] The sum of the moments of each layer corresponding to the undetermined center of gravity can be calculated using the following formula:

[0054] Among them, S(i) and Q(i) represent the area of ​​the layer, and L(i) is the target distance of the layer.

[0055] After calculating the sum of the moments of each layer corresponding to each pending center of gravity, step S5 is executed, and the pending center of gravity with the best sum of the moments is used as the target center of gravity. In this embodiment, the pending center of gravity with the smallest sum of the moments is used as the target center of gravity. Since the smaller the sum of the moments, it means that after re-sorting according to the pending center of gravity, the layers with smaller areas are better moved together, which is more conducive to the image processor to uniformly process the layers with smaller areas, thereby reducing the power consumption of the image processor when synthesizing multiple layers.

[0056] After determining the target center of gravity, execute step S6 to reorder the layers based on the target center of gravity. When reordering, the layers with smaller areas are closer to the target center of gravity, and the layers with larger areas are farther away from the target center of gravity. In addition, if two layers have an overlapping relationship, the overlapping relationship between the two layers cannot be changed. Assuming that the relationship between the multiple layers in the initial state is as follows Figure 2 , Figure 3 As shown, after the multiple layers are reordered using the “center of gravity method” of this embodiment, the upper and lower relationships of the multiple layers are as follows: Fig. 9 As shown. After reordering multiple layers using the "center of gravity method", layer 22 with a smaller area is moved to a position close to the middle, layer 21 is also moved to a position close to the middle, and layer 24 is also moved to a position close to the middle due to its smaller area. Although layer 26 has the smallest area, it cannot be moved because it is covered by multiple layers and is still at the bottom of the multiple layers.

[0057] Then, step S7 is executed to send the multiple layers with the largest area to the image carrier. Since the intelligent electronic device is usually provided with multiple image carriers, the number of image carriers is fixed. If the number of layers to be synthesized is small, for example, the number of layers to be synthesized does not exceed the number of image carriers, the multiple layers are directly transmitted to the multiple image carriers respectively, and each image carrier obtains the data of one layer and processes the data of one layer. If the number of layers to be synthesized exceeds the number of image carriers, the remaining layers need to be sent to the image processor, that is, step S8 is executed, and the image processor synthesizes the remaining layers. It can be understood that the number of layers sent to the image carrier depends on the number of image carriers. In fact, the number of layers sent to the image carrier is equal to the number of image carriers.

[0058] Finally, step S9 is executed, where the layer processor synthesizes the reordered layers to form a layer and outputs it to the display device.

[0059] The method of this embodiment is used to synthesize multiple layers. Since two steps are performed in sequence, namely, the multiple layers are first reordered, and then the layers are allocated, that is, the layers with larger areas are allocated to the image carrier, and the remaining layers with smaller areas are sent to the image processor. Therefore, the layer synthesized by the image processor has the smallest area. This embodiment minimizes the area of ​​the layer processed by the image processor. Therefore, this embodiment reasonably allocates multiple layers, so that the allocation result is the optimal allocation result with the smallest area and the least number of layers processed by the image processor. Generally, the power consumption of the image processor is much greater than that of the display controller. Therefore, the smaller the area of ​​the layer processed by the image processor, the shorter the working time consumed by the image processor in layer synthesis, which is more conducive to reducing the power consumption of the image processor.

[0060] Computer device embodiment:

[0061] The computer device of this embodiment may be an intelligent electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements each step of the above-mentioned layer synthesis method when executing the computer program. Of course, the intelligent electronic device also includes a camera device for acquiring an initial image.

[0062] For example, a computer program may be divided into one or more modules, one or more modules are stored in a memory and executed by a processor to complete each module of the present invention. One or more modules may be a series of computer program instruction segments capable of completing a specific function, and the instruction segments are used to describe the execution process of the computer program in a terminal device.

[0063] The processor referred to in the present invention may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0064] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0065] Computer readable storage medium embodiment:

[0066] If the computer program stored in the above-mentioned computer device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement each step of the above-mentioned layer synthesis method.

[0067] Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0068] Finally, it should be emphasized that the present invention is not limited to the above-mentioned embodiments. For example, changes in the method of determining the center of gravity to be determined, or changes in the method of calculating the layer moment, etc., should also be included in the scope of protection of the claims of the present invention.

Claims

1. A layer synthesis method, comprising: Get multiple layers of the image and calculate the area size of each layer; Features: Reorder the multiple layers: obtain the pending centroids of the multiple layers, where the pending centroid is located above at least one layer and below at least one layer in the stacking direction of the multiple layers; reorder the multiple layers according to each pending centroid, where the layer with a smaller area is closer to the pending centroid; calculate the sum of the moments of the multiple layers corresponding to each pending centroid, use the pending centroid with the best sum of the moments as the target centroid, and reorder the multiple layers according to the target centroid; Sending a preset number of the layers with the largest areas to a layer carrier, and sending the remaining layers to an image processor, and the image processor synthesizing the remaining layers; When calculating the moment of a layer, the target distance between the layer and the undetermined center of gravity is calculated, and then the product of the area of ​​the layer and the target distance is calculated. The sum of the moments of each layer corresponding to the undetermined center of gravity is calculated using the following formula: Among them, S(i) and Q(i) represent the area of ​​the layer, and L(i) is the target distance of the layer; Taking the undetermined center of gravity with the best total moment as the target center of gravity includes: taking the undetermined center of gravity with the minimum total moment as the target center of gravity; The step of calculating the target distance between the layer and the to-be-determined center of gravity includes: calculating the number of layers between the to-be-determined center of gravity and the layer, and the target distance increases by one unit for each layer of the layer.

2. The layer synthesis method according to claim 1, characterized in that: Reordering the plurality of layers based on the target center of gravity includes: acquiring the covering relationship between the layers. If there is an covering relationship between the layers, the covering relationship between the layers is not changed when the layers are reordered.

3. The layer synthesis method according to claim 1, characterized in that: The distance between the layer closest to the to-be-determined center of gravity and the to-be-determined center of gravity is one unit.

4. The layer synthesis method according to claim 1 or 2, characterized in that: Acquiring the pending centroids of the plurality of layers includes: setting the area between any two adjacent layers as the pending centroid.

5. A computer device, characterized in that The method comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, each step of the layer synthesis method as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the layer synthesis method as described in any one of claims 1 to 4 are implemented.

Images