Image processing method and device

By determining the area to be superimposed in the image processing and dividing the image area of ​​the molecule, the load of the hardware processing unit is optimized, and the problem of uneven load in the traditional method is solved and the efficiency of image processing is improved.

CN119027312APending Publication Date: 2024-11-26SHANGHAI BIREN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411149430.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the traditional image processing method, task scheduling based on the height of the whole image results in uneven load of the hardware processing unit, affecting the execution efficiency of the image processing.

Method used

By determining the size and position of the area to be superimposed, the image is divided into sub-image areas, and the copy and superimposition processing tasks are performed in parallel, avoiding uneven divisions and optimizing the load of the hardware processing unit.

Benefits of technology

It improves the overall efficiency of image processing, avoids load imbalance of hardware processing units, and enhances the advantages of asynchronous parallel processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027312A_ABST
    Figure CN119027312A_ABST
Patent Text Reader

Abstract

The invention provides an image processing method and device, and the method comprises the steps: determining the size of a to-be-superposed region and the position of the to-be-superposed region in a superposed image according to a to-be-superposed first image and a to-be-superposed second image; according to the position of a to-be-superposed region in the superposed image, determining at least one sub-image region for generating the superposed image; the hardware processing unit is called to read the first image unit corresponding to the sub-image area, the copy processing task corresponding to the sub-image area is executed in parallel to obtain the corresponding sub-image, or the hardware processing unit is called to read the first image unit and the second image unit corresponding to the sub-image area, and the copy processing task is executed in parallel to obtain the corresponding sub-image. The superposition processing tasks corresponding to the sub-image areas are executed in parallel, and corresponding sub-images are obtained; and splicing is performed according to the sub-images to obtain a superposed image, so that the advantage of asynchronous parallel data processing of the hardware processing unit is better played, load imbalance of the hardware processing unit is avoided, and the overall efficiency of image processing is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method and device. Background Art

[0002] In the field of image preprocessing of artificial intelligence chips, the efficiency of task scheduling directly affects the performance of operators. In the field of traditional image processing, when processing non-Inplace scenarios, that is, the input and output data are not in the same memory block. In such scenarios, image overlay operation is one of the common tasks, which involves synthesizing one image (the underlying image) with another image (the overlay image).

[0003] In traditional image processing methods, the scheduling of image overlay tasks usually calculates the height TilingH of each image based on the height Height of the entire image, and then divides the image data and schedules the tasks. This method has disadvantages: since the task scheduling is generated based on the division of the Height of the entire image, this leads to uneven loads on different hardware processing units (SPCs) when executing tasks. Some SPCs may only be responsible for data copying, while others are responsible for both data copying and image overlaying, which results in differences in execution time. Among the SPCs executed in parallel, the slowest executing SPC will become the bottleneck of the entire task, slowing down the total execution time of image processing, thereby affecting the overall execution efficiency. Summary of the invention

[0004] The present invention provides an image processing method and device to solve the defect in the prior art that the image is cut based on the height of the image, resulting in different tasks performed by different hardware processing units, thereby affecting the execution efficiency of the entire image processing.

[0005] The present invention provides an image processing method, comprising: Determine the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the area to be superimposed is an image area generated by superimposing the first image units and the second image units; Determine, according to the position of the area to be superimposed in the superimposed image, at least one sub-image area for generating the superimposed image; wherein the sub-image area includes a first image unit area and the area to be superimposed; Calling a hardware processing unit to read a first image unit corresponding to the sub-image area, and executing a copy processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image, or calling a hardware processing unit to read a first image unit and a second image unit corresponding to the sub-image area, and executing an overlay processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image; The superimposed image is obtained by performing splicing according to the sub-images.

[0006] According to the image processing method provided by the present invention, the first image unit and the second image unit are both column units; Determining the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed, specifically includes: determining the position where the second image is superimposed in the first image; determining the first image unit and the second image unit to be superimposed according to the position; determining the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image unit and the second image unit to be superimposed.

[0007] According to the image processing method provided by the present invention, determining at least one sub-image area for generating the superimposed image according to the position of the area to be superimposed in the superimposed image specifically includes: If the area to be superimposed is located in the middle of the superimposed image, determining that the sub-image area for generating the superimposed image includes a top sub-image area, a bottom sub-image area, and the area to be superimposed; If the area to be superimposed is located at the top of the superimposed image, determining that the sub-image area for generating the superimposed image includes the bottom sub-image area and the area to be superimposed; If the area to be superimposed is located at the bottom of the superimposed image, it is determined that the sub-image area for generating the superimposed image includes the top sub-image area and the area to be superimposed.

[0008] According to the image processing method provided by the present invention, calling the hardware processing unit to read the first image unit corresponding to the sub-image area, and executing the copy processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image, includes: calling the hardware processing unit to read the first image unit corresponding to the first image unit area, and executing the copy processing task corresponding to the first image unit area in parallel to obtain the corresponding copy sub-image; Calling a hardware processing unit to read the first image unit and the second image unit corresponding to the sub-image area, and executing the overlay processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image, including: calling the hardware processing unit to read the first image unit and the second image unit corresponding to the area to be overlaid, and executing the overlay processing task corresponding to the area to be overlaid in parallel to obtain a corresponding overlay sub-image.

[0009] According to the image processing method provided by the present invention, there are multiple hardware processing units; Calling the hardware processing unit to read the first image unit corresponding to the first image unit area, and executing the copy processing task corresponding to the first image unit area in parallel to obtain the corresponding copy sub-image, specifically includes: determining at least one hardware processing unit that executes the copy processing task, and the number of first image units processed by each hardware processing unit; The hardware processing unit executing the copy processing task is called in parallel, the first image units corresponding to the first image unit area are read in parallel according to the number of the first image units, the copy processing task corresponding to the first image unit area is executed, and the corresponding copy sub-image is obtained.

[0010] According to the image processing method provided by the present invention, there are multiple hardware processing units; Calling the hardware processing unit to read the first image unit and the second image unit corresponding to the area to be superimposed, and executing the superimposition processing task corresponding to the area to be superimposed in parallel to obtain the corresponding superimposed sub-image specifically includes: Determine at least one hardware processing unit that performs the overlay processing task, and the number of overlay image units processed by each hardware processing unit; The hardware processing unit for executing the overlay processing task is called in parallel, the first image unit and the second image unit corresponding to the area to be overlaid are read in parallel according to the number of overlay image units, the corresponding overlay processing task is executed, and the corresponding overlay sub-image is obtained.

[0011] The present invention also provides an image processing device, comprising: a superimposed area determination module, configured to determine the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image to be superimposed and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the area to be superimposed is an image area generated by superimposing the first image units and the second image units; An image segmentation module, used for determining at least one sub-image region for generating the superimposed image according to the position of the area to be superimposed in the superimposed image; wherein the sub-image region includes a first image unit region and the area to be superimposed; An image processing module, used to call a hardware processing unit to read a first image unit corresponding to the sub-image area, and execute a copy processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image, or call a hardware processing unit to read a first image unit and a second image unit corresponding to the sub-image area, and execute an overlay processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image; The image stitching module is used to stitch the sub-images to obtain the superimposed image.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image processing method described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image processing method described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned image processing methods.

[0015] The image processing method and device provided by the present invention determine the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; and then determine at least one sub-image area for generating the superimposed image according to the position of the area to be superimposed in the superimposed image. Compared with the method of segmenting based on the height of the image, the method of this embodiment can divide the area to be superimposed from the superimposed image, thereby avoiding uneven division of the superimposed image. Then, the hardware processing unit is called to read the first image unit corresponding to the sub-image area, and the copy processing task corresponding to the sub-image area is executed in parallel to obtain the corresponding sub-image, or the hardware processing unit is called to read the first image unit and the second image unit corresponding to the sub-image area, and the superimposition processing task corresponding to the sub-image area is executed in parallel to obtain the corresponding sub-image; finally, the sub-images are spliced ​​to obtain the superimposed image, thereby better giving play to the advantage of the hardware processing unit in asynchronous parallel processing of data, avoiding load imbalance of the hardware processing unit, and improving the overall efficiency of image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is one of the flowchart diagrams of the image processing method provided by the present invention.

[0018] Figure 2 This is one of the schematic diagrams of image processing using the method provided by the present invention.

[0019] Figure 3 This is the second schematic diagram of applying the method provided by the present invention to perform image processing.

[0020] Figure 4 This is the second flowchart of the image processing method provided by the present invention.

[0021] Figure 5 This is the third flow chart of the image processing method provided by the present invention.

[0022] Figure 6 It is a schematic diagram of generating a superimposed image provided by the present invention.

[0023] Figure 7 It is a schematic diagram of image processing according to the conventional image processing method.

[0024] Figure 8 Schematic diagram of image processing according to the image processing method of the present invention.

[0025] Fig. 9 It is a structural schematic diagram of the image processing device provided by the present invention.

[0026] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] First, the terms involved in the embodiments of the invention are further explained.

[0029] Tiling: A method of splitting data, for example, splitting data into N parts. Each of the N parts is called Tiling data. Different Tiling methods can obtain different dimensions and sizes of split data.

[0030] SPC: Stream Processing Cluster, hardware processing unit.

[0031] ROI: Region of Interest, which refers to the area to be superimposed in this embodiment.

[0032] copy: data copy, from memory A to memory B, and A and B are not the same memory.

[0033] Combine the following Figure 1-Figure 8 The image processing method according to the embodiment of the present invention is described.

[0034] Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following: 101. Determine the size of a to-be-superimposed area and the position of the to-be-superimposed area in the superimposed image according to a first image and a second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the to-be-superimposed area is an image area generated by superimposing the first image units and the second image units.

[0035] The image processing method in this embodiment can be applied to an image processor, which includes multiple units, such as an image analysis unit, an image segmentation unit, a hardware processing unit SPC, and the like.

[0036] It should be explained that after acquiring the first image and the second image, the sizes of the first image and the second image, including height and width, are first determined. In this embodiment, in order to facilitate understanding of the solution, the first image is defined to be larger than the second image, so as to achieve superimposing the second image on the first image.

[0037] The first image is divided into a plurality of first image units, and the second image is divided into a plurality of second image units. These image units can be pixel-level, or can be larger image blocks, or image units composed of a column of image blocks, depending on processing requirements and hardware capabilities.

[0038] Analyze the content of the first image and the second image to determine the area to be superimposed between them. In an illustrative embodiment, this requires semantic analysis of the image content to ensure that the superimposed image units are visually reasonable. The position determination of the area to be superimposed can be based on a specific algorithm, such as feature point matching, edge detection or other image registration techniques.

[0039] In some cases, the area to be superimposed may also be determined by a user's operation instruction, for example, allowing the user to manually select the area to be superimposed or adjust the position of the second image in the first image to meet specific visual effects or application requirements.

[0040] Of course, after determining the size and position of the overlay area, some pre-processing operations may need to be performed on the first image and / or the second image, such as scaling, rotation or color adjustment, to ensure that they are visually consistent.

[0041] Through step 101, it is ensured that the image overlay operation has a clear starting point, laying a foundation for subsequent image segmentation, processing and stitching.

[0042] 102. Determine at least one sub-image region for generating the superimposed image according to a position of the area to be superimposed in the superimposed image; wherein the sub-image region includes a first image unit region and the area to be superimposed.

[0043] In step 102, the superimposed image is divided into a plurality of sub-image regions according to the position of the region to be superimposed, which may be a top sub-image region, a middle sub-image region and a bottom sub-image region, depending on the layout of the region to be superimposed.

[0044] Specifically, if the area to be superimposed is located in the middle of the superimposed image, determining that the sub-image area for generating the superimposed image includes the top sub-image area, the bottom sub-image area, and the area to be superimposed; If the area to be superimposed is located at the top of the superimposed image, determining that the sub-image area for generating the superimposed image includes the bottom sub-image area and the area to be superimposed; If the area to be superimposed is located at the bottom of the superimposed image, it is determined that the sub-image area for generating the superimposed image includes the top sub-image area and the area to be superimposed.

[0045] See also Figure 2 and Figure 3 , the second image is the black part in the middle, and the first image is the white part. The second image is superimposed on the middle of the first image. Then for Figure 2 The overlay image needs to be divided into three sub-image areas: the top sub-image area aboveDst, the bottom sub-image area bottomDst and the area to be overlaid roiSrc. Figure 3 The overlay image needs to be divided into two sub-image areas: the top sub-image area aboveDst and the area to be overlaid roiSrc.

[0046] In step 103, the area to be superimposed can be divided from the superimposed image. Compared with the division based on the height of the superimposed image in the prior art, the method of this embodiment can avoid the uneven division of the superimposed image, which is conducive to the subsequent parallel processing of the area to be superimposed and the non-superimposed area.

[0047] 103. Call a hardware processing unit to read the first image unit corresponding to the sub-image area, and execute the copy processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image, or call a hardware processing unit to read the first image unit and the second image unit corresponding to the sub-image area, and execute the overlay processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image.

[0048] During the parallel processing, each sub-image area is mapped to a corresponding hardware processing unit, ensuring that each hardware processing unit has a corresponding image area to be processed.

[0049] According to the type of the sub-image area, the copy processing task or the overlay processing task is assigned to the corresponding hardware processing unit.

[0050] For a sub-image area that only contains the first image unit, the hardware processing unit executes the copy processing task in parallel to copy the data of the first image unit from the original position to the target position; for a sub-image area that contains the first image unit and the second image unit, the hardware processing unit executes the overlay processing task in parallel to synthesize the data of the two image units.

[0051] During task processing, ensure that task execution of all hardware processing units is synchronized to avoid data competition and inconsistency issues.

[0052] By calling the hardware processing unit in step 103 to execute the copy processing task or the superposition processing task corresponding to the sub-image area in parallel, efficient and accurate processing of each sub-image area is ensured.

[0053] 104. Perform stitching according to the sub-images to obtain the superimposed image.

[0054] All processed sub-images are collected from step 103, including the copied sub-images and the superimposed sub-images, and the order of splicing the sub-images is determined according to the positions of the sub-images in the final superimposed image.

[0055] During the stitching process, ensure that the boundaries between the sub-images are accurately aligned to avoid seams or overlaps in the stitched image. In addition, it is also necessary to ensure the uniformity of color and brightness. Adjust the color and brightness of the sub-images to ensure that the stitched image is visually uniform. If necessary, adjust the transparency of the sub-images to achieve a smooth transition between different sub-images.

[0056] For edges, edge smoothing techniques such as feathering or blurring can also be applied to reduce the obvious boundaries between sub-images.

[0057] During the stitching process, the stitching quality needs to be constantly checked to ensure that there are no errors or unnatural effects to ensure the efficiency of the stitching operation, especially when processing high-resolution images. If user interaction is supported, allow the user to make fine adjustments during the stitching process, such as adjusting the position of the sub-images or modifying the stitching parameters.

[0058] Through step 104, accurate splicing of the sub-images is ensured, and a high-quality superimposed image is generated.

[0059] The image processing method provided by the embodiment of the present invention determines the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; and then determines at least one sub-image area for generating the superimposed image according to the position of the area to be superimposed in the superimposed image. Compared with the method of segmenting based on the height of the image, the method of this embodiment can divide the area to be superimposed from the superimposed image, thereby avoiding uneven division of the superimposed image. Then, the hardware processing unit is called to read the first image unit and / or the second image unit corresponding to the sub-image area, and the superimposition processing task or the copy processing task corresponding to the sub-image area is executed in parallel to obtain the corresponding sub-image; finally, the sub-images are spliced ​​to obtain the superimposed image, thereby better utilizing the advantage of the hardware processing unit in asynchronously processing data in parallel, avoiding load imbalance of the hardware processing unit, and improving the overall efficiency of image processing.

[0060] Further, taking the first image unit and the second image unit as an example, see Figure 4 , step 101 specifically includes: 401. Determine a position where the second image is superimposed on the first image.

[0061] 402. Determine a first image unit and a second image unit to be superimposed according to the position.

[0062] 403. Determine the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image unit and the second image unit to be superimposed.

[0063] It should be noted that the area to be superimposed is determined by the size of the first image unit and the second image unit, rather than the size of the first image and the second image. If the first image unit and the second image unit are both column units, then the area to be superimposed is also composed of column units, and the width is the same as the first image unit (the larger image unit).

[0064] Further, see Figure 5, step 103 specifically includes: 501. Call a hardware processing unit to read a first image unit corresponding to a first image unit region, and execute a copy processing task corresponding to the first image unit region in parallel to obtain a corresponding copy sub-image.

[0065] Step 501 specifically includes: determining at least one hardware processing unit that performs the copy processing task, and the number of first image units processed by each hardware processing unit; calling the hardware processing units that perform the copy processing task in parallel, reading the first image units corresponding to the first image unit area in parallel according to the number of first image units, performing the copy processing task corresponding to the first image unit area, and obtaining the corresponding copy sub-image.

[0066] It should be noted that the parallel calling of hardware processing units for processing in this embodiment refers to parallel processing of image units included in each sub-image area, and does not mean parallel processing of multiple sub-image areas respectively.

[0067] According to the size and complexity of the first image unit area, the copy task is assigned to the selected hardware processing unit. The number of first image units that each hardware processing unit needs to process is determined to ensure balanced distribution of tasks.

[0068] 502. Call a hardware processing unit to read a first image unit and a second image unit corresponding to the area to be superimposed, and execute the superimposition processing task corresponding to the area to be superimposed in parallel to obtain a corresponding superimposed sub-image.

[0069] Considering that the bandwidth utilization of the preloaded image module preLoadGmb continuously fetching data is greater than the bandwidth utilization of the preloaded image module preLoadGmb non-continuously fetching data, in this embodiment, the data of the first image unit and the second image unit are read by the hardware processing unit, and the on-chip buffer is used for continuous data copying.

[0070] Step 502 specifically includes: determining at least one hardware processing unit that performs the overlay processing task, and the number of overlay image units processed by each hardware processing unit; calling the hardware processing units that perform the overlay processing task in parallel, reading the first image unit and the second image unit corresponding to the area to be overlaid in parallel according to the number of overlay image units, performing the corresponding overlay processing task, and obtaining the corresponding overlay sub-image.

[0071] For each hardware processing unit, it is necessary to read the corresponding first image unit and second image unit data from the area to be superimposed according to the number of superimposed image units allocated. Each hardware processing unit independently performs the superimposition processing task of the image unit it reads, which may include pixel-level mixing, fusion or other image synthesis operations.

[0072] During the execution process, it is necessary to ensure that the operations of all hardware processing units are performed synchronously to maintain data consistency and integrity.

[0073] by Figure 2 Taking the image shown as an example, the processing of the sub-image can be divided into two steps and implemented through three task scheduling.

[0074] Step 1: Use aboveDstHeight to get the top sub-image area height TilingAboveH, and use bottomDstHeight to get the bottom sub-image area height TilingBottomH. After the calculation is completed, execute the copy processing tasks corresponding to the top sub-image area data and the bottom sub-image area data in turn to obtain the corresponding copy sub-image. The two copies contain two task scheduling, and there is no dependency between the data.

[0075] Specifically, the top sub-image is split into multiple row units, and the size cannot exceed the buffer memory.

[0076] The height of the top sub-image area TilingAboveH divided by the number of hardware processing units SPC is the Height that each hardware processing unit SPC needs to process, denoted as hPerSpc.

[0077] Assuming that the buffer can process hMax rows at most, if hPerSpc is less than hMax, each hardware processing unit SPC will process hPerSpc rows and complete the processing in one go; if hPerSpc is greater than hMax, each hardware processing unit SPC will process hMax rows and process them multiple times until hPerSpc is processed.

[0078] The processing process for the bottom sub-image is similar to that for the top sub-image, and will not be described in detail here.

[0079] Step 2: Use roiDstHeight to get the height of the area to be superimposed TilingRoiH, read the first image unit and the second image unit corresponding to the area to be superimposed in parallel according to the number of superimposed image units, and perform the corresponding superimposition processing task. In this step, only the internal data of ROI is calculated, and there is only one task scheduling.

[0080] In order to facilitate understanding of the technical solution of the embodiment of the present invention, the technical solution of this embodiment is schematically described below based on a specific example.

[0081] See also Figure 6 , Figure 6 Schematic diagram of generating a superimposed image dst by superimposing a first image src1 and a second image src2.

[0082] See respectively Figure 7 and Figure 8 , Figure 7 is a schematic diagram of image processing according to an image processing method of conventional technology, Figure 8 FIG. 4 is a schematic diagram of performing image processing according to the image processing method of this embodiment.

[0083] Assume that the hardware has only 4 hardware processing units SPC that can be scheduled, and 1 hardware processing unit SPC processes at least 2 lines of data. The time to copy 2 lines of data is m seconds (here only the base map data is copied), and the time to overlay 2 lines of images (because image overlay not only needs to fetch the base data, but also the overlay image, and then perform calculations, so it takes longer) is 2*m seconds.

[0084] like Figure 7 The figure shows an example of image processing in a traditional method. Since the Height of the first image src1 in the figure is 16, it is evenly distributed to the four hardware processing units SPC of the hardware, and the TilingHeight processed by each hardware processing unit spc is 4.

[0085] It can be found that spc0 and spc2 can do both data copying and image overlay, spc1 can only do image overlay, and spc3 can only do data copying. That is: spc0 time consumption: It takes m seconds to copy 2 lines of image units, and it takes 2*m seconds to superimpose 2 lines of image units, so the total time consumption is 3*m seconds.

[0086] SPC1 time consumption: It takes 2m*2 seconds to stack 4 rows of image units, and a total of 4*m seconds.

[0087] spc2 time consumption: It takes m seconds to copy 2 lines of image units, and it takes 2*m seconds to superimpose 2 lines of image units, so the total time consumption is 3*m seconds.

[0088] spc4 time consumption: copying 4 rows of image units takes m*2 seconds, and the total time consumption is 2*m seconds.

[0089] Since spc0, spc1, spc2, and spc3 are all executed in parallel, the program ends only after all spcs are executed. Therefore, the final time of the operator is determined by spc1, which runs the slowest. The traditional solution takes 4*m seconds.

[0090] like Figure 8 As shown, Figure 8 The processing of the existing image is divided into three tasks for scheduling. The three tasks need to process data separately, and 4 spcs are called for each task processing. The data processed by the three tasks have no dependency, so they can be processed asynchronously and in parallel.

[0091] Task 1: Copy 2 lines of image units. spc0 processes 2 lines of H. spc1, spc2, and spc3 are idle and do not process data. Therefore, the total time consumed is the time consumed by spc0, that is, the time consumed to copy 2 lines of image units. The total time consumed is m seconds.

[0092] Task 2: Perform image superposition of 8 rows of image units, which is evenly distributed to 4 SPCs. Each SPC processes 2 rows of image units. Since the 4 SPCs are parallel, the total time consumed is the time consumed to process the superposition of 2 rows of image units, and the total time consumed is 2*m seconds.

[0093] Task 3: Only copy 6 rows of image units. spc0, spc1, and spc2 each process 2 rows of image units. Spc3 is idle and does not process data. Therefore, the total time consumed is the time consumed by any one of spc0 to spc2, that is, the time consumed to copy 2 rows of image units. The total time consumed is m seconds.

[0094] It can be seen that even if Task 1, Task 2, and Task 3 are executed in sequence, the total time consumed is only: m+2*m+m=4*m seconds. Since the data of Task 1, Task 2, and Task 3 are not dependent on each other, they can be asynchronously parallelized, and the total time consumed will be less than 4*m seconds. For example, if Task 3 is set to spc1, spc2, and spc3 to process 2 rows of image units respectively, then Task 3 and Task 1 can be executed in parallel, thereby reducing the total time consumed. Obviously, the method of this embodiment has better performance than the traditional method.

[0095] In summary, it can be seen that the method of this embodiment has a more obvious benefit when the height of the superimposed area is moderate or relatively small. The smaller the proportion of the superimposed image to the original image, the better the optimization effect.

[0096] The image processing device provided by the embodiment of the present invention is described below. The image processing device described below and the image processing method described above can be referred to each other.

[0097] The embodiment of the present invention provides an image processing device, see Fig. 9 ,include: The superimposed region determining module 901 is used to determine the size of the region to be superimposed and the position of the region to be superimposed in the superimposed image according to the first image to be superimposed and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the region to be superimposed is an image region generated by superimposing the first image units and the second image units; An image segmentation module 902 is used to determine at least one sub-image region for generating the superimposed image according to the position of the area to be superimposed in the superimposed image; wherein the sub-image region includes a first image unit region and the area to be superimposed; The image processing module 903 is used to call the hardware processing unit to read the first image unit corresponding to the sub-image area, and execute the copy processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image, or call the hardware processing unit to read the first image unit and the second image unit corresponding to the sub-image area, and execute the overlay processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image; The image stitching module 904 is used to stitch the sub-images to obtain the superimposed image.

[0098] Optionally, the first image unit and the second image unit are both column units; The superposition area determination module 901 is specifically used for: determining a position where the second image is superimposed on the first image; Determine a first image unit and a second image unit to be superimposed according to the position; According to the first image unit and the second image unit to be superimposed, the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image are determined.

[0099] Optionally, the image segmentation module 902 is specifically configured to: If the area to be superimposed is located in the middle of the superimposed image, determining that the sub-image area for generating the superimposed image includes a top sub-image area, a bottom sub-image area, and the area to be superimposed; If the area to be superimposed is located at the top of the superimposed image, determining that the sub-image area for generating the superimposed image includes the bottom sub-image area and the area to be superimposed; If the area to be superimposed is located at the bottom of the superimposed image, it is determined that the sub-image area for generating the superimposed image includes the top sub-image area and the area to be superimposed.

[0100] Optionally, the image processing module 903 is specifically configured to: Calling the hardware processing unit to read the first image unit corresponding to the first image unit area, and executing the copy processing task corresponding to the first image unit area in parallel to obtain a corresponding copy sub-image; The hardware processing unit is called to read the first image unit and the second image unit corresponding to the area to be superimposed, and the superimposition processing task corresponding to the area to be superimposed is executed in parallel to obtain a corresponding superimposed sub-image.

[0101] Optionally, there are multiple hardware processing units; the image processing module 903 is specifically used to: determine at least one hardware processing unit that performs the copy processing task, and the number of first image units processed by each hardware processing unit; call the hardware processing units that perform the copy processing task in parallel, read the first image units corresponding to the first image unit area in parallel according to the first image unit number, execute the copy processing task corresponding to the first image unit area, and obtain the corresponding copy sub-image.

[0102] Optionally, there are multiple hardware processing units; the image processing module 903 is specifically used to: determine at least one hardware processing unit that performs the overlay processing task, and the number of overlay image units processed by each hardware processing unit; call the hardware processing units that perform the overlay processing task in parallel, read the first image unit and the second image unit corresponding to the area to be overlaid in parallel according to the number of overlay image units, perform the corresponding overlay processing task, and obtain the corresponding overlay sub-image.

[0103] The image processing device provided by the embodiment of the present invention determines the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; and then determines at least one sub-image area for generating the superimposed image according to the position of the area to be superimposed in the superimposed image. Compared with the method of segmenting based on the height of the image, the method of the embodiment can divide the area to be superimposed from the superimposed image, thereby avoiding uneven division of the superimposed image. Then, the hardware processing unit is called to read the first image unit corresponding to the sub-image area, and the copy processing task corresponding to the sub-image area is executed in parallel to obtain the corresponding sub-image, or the hardware processing unit is called to read the first image unit and the second image unit corresponding to the sub-image area, and the superimposition processing task corresponding to the sub-image area is executed in parallel to obtain the corresponding sub-image; finally, the sub-images are spliced ​​to obtain the superimposed image, thereby better giving play to the advantage of asynchronous parallel data processing of the hardware processing unit, avoiding load imbalance of the hardware processing unit, and improving the overall efficiency of image processing.

[0104] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10As shown, the electronic device may include: a processor (processor) 1010 , a communication interface (Communications Interface) 1020 , a memory (memory) 1030 and a communication bus 1040 , wherein the processor 1010 , the communication interface 1020 , and the memory 1030 communicate with each other via the communication bus 1040 . The processor 1010 can call the logic instructions in the memory 1030 to execute the image processing method, including: determining the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the area to be superimposed is an image area generated by superimposing the first image unit and the second image unit; determining at least one sub-image area for generating the superimposed image according to the position of the area to be superimposed in the superimposed image; wherein the sub-image area includes a first image unit area and the area to be superimposed; calling the hardware processing unit to read the first image unit corresponding to the sub-image area, and executing the copy processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image, or calling the hardware processing unit to read the first image unit and the second image unit corresponding to the sub-image area, and executing the superimposition processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image; and performing splicing according to the sub-images to obtain the superimposed image.

[0105] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0106] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the image processing methods provided by the above methods, including: determining the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the area to be superimposed is an image area generated by superimposing the first image unit and the second image unit; determining at least one sub-image area for generating the superimposed image according to the position of the area to be superimposed in the superimposed image; wherein the sub-image area includes a first image unit area and the area to be superimposed; calling a hardware processing unit to read the first image unit corresponding to the sub-image area, and executing the copy processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image, or calling the hardware processing unit to read the first image unit and the second image unit corresponding to the sub-image area, and executing the superimposition processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image; splicing according to the sub-images to obtain the superimposed image.

[0107] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the image processing method provided by the above methods is implemented, including: determining the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the area to be superimposed is an image area generated by superimposing the first image unit and the second image unit; determining at least one sub-image area for generating the superimposed image according to the position of the area to be superimposed in the superimposed image; wherein the sub-image area includes a first image unit area and the area to be superimposed; calling a hardware processing unit to read the first image unit corresponding to the sub-image area, and executing the copy processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image, or calling the hardware processing unit to read the first image unit and the second image unit corresponding to the sub-image area, and executing the superimposition processing task corresponding to the sub-image area in parallel to obtain the corresponding sub-image; splicing according to the sub-images to obtain the superimposed image.

[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing an image, characterized in that: include: Determine the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the area to be superimposed is an image area generated by superimposing the first image units and the second image units; Determine, according to the position of the area to be superimposed in the superimposed image, at least one sub-image area for generating the superimposed image; wherein the sub-image area includes a first image unit area and the area to be superimposed; Calling a hardware processing unit to read a first image unit corresponding to the sub-image area, and executing a copy processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image, or calling a hardware processing unit to read a first image unit and a second image unit corresponding to the sub-image area, and executing an overlay processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image; The superimposed image is obtained by performing splicing according to the sub-images.

2. The image processing method according to claim 1, characterized in that: The first image unit and the second image unit are both column units; Determining the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image and the second image to be superimposed specifically includes: determining a position where the second image is superimposed on the first image; Determine a first image unit and a second image unit to be superimposed according to the position; According to the first image unit and the second image unit to be superimposed, the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image are determined.

3. The image processing method according to claim 1, characterized in that: Determining, according to a position of the to-be-overlaid area in the overlaid image, at least one sub-image area for generating the overlaid image specifically includes: If the area to be superimposed is located in the middle of the superimposed image, determining that the sub-image area for generating the superimposed image includes a top sub-image area, a bottom sub-image area, and the area to be superimposed; If the area to be superimposed is located at the top of the superimposed image, determining that the sub-image area for generating the superimposed image includes the bottom sub-image area and the area to be superimposed; If the area to be superimposed is located at the bottom of the superimposed image, it is determined that the sub-image area for generating the superimposed image includes the top sub-image area and the area to be superimposed.

4. The image processing method according to claim 1, characterized in that: Calling a hardware processing unit to read a first image unit corresponding to the sub-image area, and executing a copy processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image, including: Calling the hardware processing unit to read the first image unit corresponding to the first image unit area, and executing the copy processing task corresponding to the first image unit area in parallel to obtain a corresponding copy sub-image; Calling a hardware processing unit to read a first image unit and a second image unit corresponding to the sub-image area, and executing a superposition processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image, including: The hardware processing unit is called to read the first image unit and the second image unit corresponding to the area to be superimposed, and the superimposition processing task corresponding to the area to be superimposed is executed in parallel to obtain a corresponding superimposed sub-image.

5. The image processing method according to claim 4, characterized in that: There are multiple hardware processing units; Calling the hardware processing unit to read the first image unit corresponding to the first image unit area, and executing the copy processing task corresponding to the first image unit area in parallel to obtain the corresponding copy sub-image specifically includes: Determining at least one hardware processing unit that performs the copy processing task, and the number of first image units processed by each hardware processing unit; The hardware processing unit executing the copy processing task is called in parallel, the first image units corresponding to the first image unit area are read in parallel according to the number of the first image units, the copy processing task corresponding to the first image unit area is executed, and the corresponding copy sub-image is obtained.

6. The image processing method according to claim 4, characterized in that: There are multiple hardware processing units; Calling the hardware processing unit to read the first image unit and the second image unit corresponding to the area to be superimposed, and executing the superimposition processing task corresponding to the area to be superimposed in parallel to obtain the corresponding superimposed sub-image specifically includes: Determine at least one hardware processing unit that performs the overlay processing task, and the number of overlay image units processed by each hardware processing unit; The hardware processing unit for executing the overlay processing task is called in parallel, the first image unit and the second image unit corresponding to the area to be overlaid are read in parallel according to the number of overlay image units, the corresponding overlay processing task is executed, and the corresponding overlay sub-image is obtained.

7. An image processing device, characterized in that: include: a superimposed area determination module, configured to determine the size of the area to be superimposed and the position of the area to be superimposed in the superimposed image according to the first image to be superimposed and the second image to be superimposed; wherein the first image is larger than the second image, the first image includes a plurality of first image units, the second image includes a plurality of second image units, and the area to be superimposed is an image area generated by superimposing the first image units and the second image units; An image segmentation module, used for determining at least one sub-image region for generating the superimposed image according to the position of the area to be superimposed in the superimposed image; wherein the sub-image region includes a first image unit region and the area to be superimposed; An image processing module, used to call a hardware processing unit to read a first image unit corresponding to the sub-image area, and execute a copy processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image, or call a hardware processing unit to read a first image unit and a second image unit corresponding to the sub-image area, and execute an overlay processing task corresponding to the sub-image area in parallel to obtain a corresponding sub-image; The image stitching module is used to stitch the sub-images to obtain the superimposed image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the image processing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.