Data processing method, device, terminal and readable storage medium
By acquiring the filter radius of the filter layer, determining the filter area and directly obtaining image data, solving the problem of repeated data transmission between filter layers in data processing, reducing hardware power consumption and improving processing efficiency.
Patent Information
- Application Number
- CN202210126015.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-10
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-02-10
AI Technical Summary
During data processing, frequent transmission of data between memory and memory leads to increased power consumption and inefficiency. This is mainly due to the difference in data requirements of different data processing processes, resulting in a large amount of repeated data transmission between filter layers and time. .
By acquiring the filter radii of the preset multiple filter layers, the filtering area of the data to be processed is determined, and the image data of the filtering area is directly obtained, reducing the need for each filter layer to independently acquire data, and reducing the data transmission amount and time.
It significantly reduces the amount of data transmission and transmission time, reduces hardware power consumption, and improves data processing efficiency.
Smart Images

Figure CN114529483B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image technology, and in particular to a data processing method, a data processing device, a terminal, and a non-volatile computer-readable storage medium. Background Art
[0002] Currently, when processing data, the data needs to be transferred back and forth between the memory and the storage. Different data processing processes require different data to be obtained. Therefore, each data processing process requires data transmission between the memory and the storage, resulting in a large amount of data transmission and a large amount of time occupied by data transmission, which increases the power consumption of the hardware and reduces the data processing efficiency. Summary of the invention
[0003] Embodiments of the present application provide a data processing method, a data processing device, a terminal, and a non-volatile computer-readable storage medium.
[0004] The data processing method of the implementation mode of the present application includes obtaining the filter radii of multiple preset filter layers; determining the filter area of the data to be processed according to the filter radii of the multiple filter layers; and obtaining image data of the filter area.
[0005] The data processing device of the embodiment of the present application includes a first acquisition module, a first determination module and a second acquisition module. The first acquisition module is used to acquire the filter radius of multiple preset filter layers; the first determination module is used to determine the filter area of the data to be processed according to the filter radius of the multiple filter layers; the second acquisition module is used to acquire the image data of the filter area.
[0006] The terminal of the embodiment of the present application includes a processor, which is used to obtain the filtering radius of multiple preset filtering layers; determine the filtering area of the data to be processed according to the filtering radius of the multiple filtering layers; and obtain image data of the filtering area.
[0007] The present application discloses a non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, enables the processors to perform a data processing method. The data processing method includes obtaining filter radii of a plurality of preset filter layers; determining a filter area of data to be processed according to the filter radii of the plurality of filter layers; and obtaining image data of the filter area.
[0008] The data processing method, data processing device, terminal and non-volatile computer-readable storage medium of the implementation mode of the present application obtain the filtering radius of multiple filtering layers of the data to be processed, and thus determine and obtain all the filtering areas required for filtering the data to be processed by multiple filtering layers according to the filtering radius of the multiple filtering layers. Compared with obtaining the data of the filtering area of each filtering layer separately when filtering each filtering layer, there are repeated parts in the filtering areas of different filtering layers, and the data transmission volume and the time occupied by data transmission are both large, resulting in increased power consumption of hardware and low data processing efficiency. The data transmission volume and the time occupied by data transmission are significantly reduced, the power consumption of hardware is reduced and the data processing efficiency is improved.
[0009] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0011] Figure 1 is a flowchart of a data processing method of certain embodiments of the present application;
[0012] Figure 2 is a schematic diagram of a module of a data processing device in some embodiments of the present application;
[0013] Figure 3 is a schematic diagram of a terminal plan of certain embodiments of the present application;
[0014] Figures 4 to 6 It is a schematic diagram of the principles of certain embodiments of the present application;
[0015] Figure 7 is a flowchart of a data processing method of certain embodiments of the present application;
[0016] Figure 8 It is a schematic diagram of the principles of certain embodiments of the present application;
[0017] Fig. 9 is a flowchart of a data processing method of certain embodiments of the present application;
[0018] Fig.10 It is a schematic diagram of the principles of certain embodiments of the present application;
[0019] Fig.11and Fig.12 is a flowchart of a data processing method of certain embodiments of the present application;
[0020] Fig.13 A schematic diagram of the connection between a processor and a computer-readable storage medium in certain embodiments of the present application. DETAILED DESCRIPTION
[0021] The following is a further description of the embodiments of the present application in conjunction with the accompanying drawings. The same or similar reference numerals in the accompanying drawings represent the same or similar elements or elements with the same or similar functions from beginning to end. In addition, the embodiments of the present application described below in conjunction with the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the present application.
[0022] See also Figures 1 to 3 The data processing method of the embodiment of the present application comprises the following steps:
[0023] 011: Get the filter radius of multiple preset filter layers;
[0024] 012: Determine a filtering area for the data to be processed according to the filtering radius of the multiple filtering layers; and
[0025] 013: Get the image data of the filter area.
[0026] The data processing device 10 of the embodiment of the present application includes a first acquisition module 11, a first determination module 12, and a second acquisition module 13. The first acquisition module 11, the first determination module 12, and the second acquisition module 13 are respectively used to execute step 011, step 012, and step 013. That is, the first acquisition module 11 is used to obtain the filter radius of the preset multiple filter layers, the first determination module 12 is used to determine the filter area of the data to be processed according to the filter radius of the multiple filter layers; the second acquisition module 13 is used to obtain the image data of the filter area.
[0027] The terminal 100 of the embodiment of the present application includes a processor 30. The processor 30 is used to obtain the filter radius of the preset multiple filter layers; determine the filter area of the data to be processed according to the filter radius of the multiple filter layers; and obtain the image data of the filter area. That is to say, step 011, step 012 and step 013 can be implemented by the processor 30.
[0028] Specifically, the terminal 100 further includes a housing 40. The terminal 100 may be a mobile phone, a tablet computer, a display device, a laptop computer, a teller machine, a gate machine, a smart watch, a head-mounted display device, a game console, etc. Figure 3As shown, the embodiment of the present application is described by taking the terminal 100 as a mobile phone as an example. It can be understood that the specific form of the terminal 100 is not limited to a mobile phone. The housing 40 can also be used to install functional modules such as a display device, an imaging device, a power supply device, and a communication device of the terminal 100, so that the housing 40 provides dustproof, drop-proof, waterproof and other protections for the functional modules.
[0029] The data to be processed may be an image captured by the camera 20 of the terminal 100, or an image downloaded from the Internet, which is not limited here. The data to be processed may also be a part of the image captured by the camera 20; or, the data to be processed may be multi-frame posture data, acceleration data, etc. acquired by the terminal 100. The embodiment of the present application is described by taking the image to be processed as the data to be processed as an example.
[0030] When filtering the image to be processed, it is generally performed based on a preset filtering algorithm. The preset filtering algorithm determines the preset size of the image to be processed, the preset number of filtering layers, and the preset filtering radius of each filtering layer.
[0031] During filtering, if the size of the captured image is larger than the preset size, the captured image can be divided into multiple images to be processed by the preset size, so that the processor 30 can directly obtain the position information of each image to be processed, such as the vertex coordinates of the image to be processed (taking the image to be processed as a rectangle as an example). At the same time, the processor 30 can also directly obtain the filtering radius of each filtering layer.
[0032] Then, the processor 30 can determine the filter area of each filter layer according to the position information of the image to be processed and the filter radius of each filter layer. For example, the filter radius is the distance from the edge of the image to be processed, and the position information includes the vertex coordinates of the image to be processed. According to the vertex coordinates of the image to be processed, the width and height of the image to be processed can be quickly determined. The processor 30 can determine the vertex coordinates of the filter area according to the vertex coordinates of the image to be processed and the filter radius.
[0033] For an example, see Figure 4 , establish an image coordinate system with the upper left corner of the image A1 to be processed as the origin. The width of the image A1 to be processed is 8 (the number of pixels along the W direction) and the height is 8 (the number of pixels along the H direction). The vertex coordinates of the image A1 to be processed are (0,0), (8,0), (0,8) and (8,8). If the filter radius is 2 pixels, two rows of pixels are added to the upper and lower edges of the image A1 to be processed, and two columns of pixels are added to the left and right edges. The vertex coordinates of the image S1 to be processed are (-2,-2), (10,2), (-2,10) and (10,10). In this way, according to the position information of the image A1 to be processed and the filter radius of each filter layer, the filter area S1 of each filter layer can be quickly determined.
[0034] In other embodiments, the filter radius includes a first filter radius (such as the top of the corresponding image), a second filter radius (such as the right side of the corresponding image), a third filter radius (such as the bottom of the corresponding image) and a fourth filter radius (such as the left side of the corresponding image), and the first filter radius, the second filter radius, the third filter radius and the fourth filter radius are the same (such as Figure 4 or the first filtering radius and the third filtering radius are the same, and the second filtering radius and the fourth filtering radius are the same; or the first filtering radius, the second filtering radius, the third filtering radius and the fourth filtering radius are different from each other.
[0035] In another example, see Figure 5 , an image coordinate system is established with the upper left corner of the image A1 to be processed as the origin, then the vertex coordinates of the image A1 to be processed are (0,0), (8,0), (0,8) and (8,8), respectively. If the first filter radius and the third filter radius are the same and are both 2 pixels, and the second filter radius and the fourth filter radius are the same and are both 1 pixel, then two rows of pixels are added to the upper and lower edges of the image A1 to be processed, and one column of pixels is added to the left and right edges, respectively. Then the vertex coordinates of the image S1 to be processed are (-1,-2), (9,-2), (-1,10) and (9,10), respectively. In this way, according to the position information of the image A1 to be processed and the filter radius of each filter layer, the filter area S1 of each filter layer can be quickly determined.
[0036] In another example, see Figure 6 , an image coordinate system is established with the upper left corner of the image A1 to be processed as the origin, then the vertex coordinates of the image A1 to be processed are (0,0), (8,0), (0,8) and (8,8), respectively. If the first filter radius, the second filter radius, the third filter radius and the fourth filter radius are 4 pixels, 3 pixels, 2 pixels and 1 pixel, respectively, then 4 rows of pixels and 2 rows of pixels are added to the upper and lower edges of the image A1 to be processed, and 1 column of pixels and 3 columns of pixels are added to the left and right edges, respectively, then the vertex coordinates of the image A1 to be processed are (-1,-4), (11,-4), (-1,10) and (11,10), respectively. In this way, according to the position information of the image A1 to be processed and the filter radius of each filter layer, the filter area S1 of each filter layer can be quickly determined.
[0037] It can be understood that when performing filtering processing, two adjacent filtering layers are generally interdependent. For example, the output image of the previous filtering layer will be used as the input image of the next filtering layer, and the input image of the next filtering layer is determined by the output image of the next filtering layer and the filtering radius of the next filtering layer.
[0038] Therefore, the filtering area of the last filtering layer can be determined according to the output image of the last filtering layer and the filtering radius of the last filtering layer, thereby determining the output image of the second-to-last filtering layer (that is, the image data of the filtering area of the last filtering layer is also the input image of the last filtering layer). Then, the filtering area of the second-to-last filtering layer can be determined according to the output image of the second-to-last filtering layer and the filtering radius of the second-to-last filtering layer, thereby determining the output image of the third-to-last filtering layer (that is, the image data of the filtering area of the second-to-last filtering layer is also the input image of the second-to-last filtering layer). In this way, the filtering area of the first filtering layer among all the filtering layers can be determined, thereby determining the filtering area of the image to be processed (that is, the image data of the filtering area of the first filtering layer is also the input image of the first filtering layer).
[0039] It can be understood that the number of filter layers can be 1 layer, 2 layers, 3 layers, 4 layers, 5 layers, etc. The above embodiment is described with 3 or more filter layers. If there are two filter layers, it is only necessary to determine the filter area of the penultimate filter layer in the above embodiment, and the filter area is the filter area of the image to be processed. If there is one filter layer, it is only necessary to determine the filter area of the first filter layer according to the output image of the first filter layer and the filter radius of the first filter layer, and the filter area is the filter area of the image to be processed.
[0040] Of course, it is also possible to obtain only the filtering radius of any two adjacent layers, three adjacent layers in sequence, four layers, etc. in the multiple filtering layers, so as to obtain the filtering areas corresponding to any two adjacent layers, three adjacent layers in sequence, four layers, etc., so that the data of the filtering area can be read once and used by any two adjacent layers, three adjacent layers in sequence, four layers, etc., thereby reducing the amount of data reading and writing during filtering.
[0041] For example, taking the filtering layer as 4 layers, after the filtering is completed in the first layer, the filtering area corresponding to the input image of the second layer can be calculated according to the filtering radius of the second layer to the fourth layer. The calculation method refers to the implementation method of calculating the filtering area of the first layer in the aforementioned implementation method, which will not be repeated here. In this way, the image data of the second layer to the fourth layer adjacent to each other only needs to be read once, thereby reducing the amount of data reading and writing during filtering.
[0042] In addition, the image size of the output of each filter layer can be fixed, such as the same as the size of the image to be processed. For example, if the captured image is 100*100 and the size of the image to be processed is 50*50, the captured image is divided into 4 images to be processed. After filtering the image to be processed, the size of the final output filtered image is also 50*50.
[0043] Finally, the processor 30 directly obtains the image data in the filter area of the image to be processed, and can obtain all the image data required for subsequent filtering of the image to be processed. Among them, the image data existing in the filter area can be directly obtained (such as the part where the filter area overlaps with the captured image), and the part where the filter area does not have image data (the part where the filter area does not overlap with the captured image) needs to be filled according to the existing image data in the filter area, so as to obtain the image data of each pixel in the filter area.
[0044] The data processing method, data processing device 10 and terminal 100 of the implementation mode of the present application obtain the filtering radius of multiple filtering layers of the data to be processed, and thus determine and obtain all the filtering areas required for filtering the data to be processed by multiple filtering layers according to the filtering radius of the multiple filtering layers. Compared with obtaining the data of the filtering area of each filtering layer separately when filtering each filtering layer, there are repeated parts in the filtering areas of different filtering layers, and the data transmission volume and the time occupied by data transmission are both large, resulting in increased power consumption of hardware and low data processing efficiency. The data transmission volume and the time occupied by data transmission are significantly reduced, the power consumption of hardware is reduced and the data processing efficiency is improved.
[0045] See also Figure 2 , Figure 3 and Figure 7 In some embodiments, the filter layer includes M layers, M is a positive integer, and the filter area of the Mth layer is determined according to the filter radius of the Mth layer and the image area where the image to be processed is located. Step 012 also includes:
[0046] 0121: Determine the filtering area of the N-1th layer according to the filtering radius of the N-1th layer and the filtering area of the Nth layer, where N is a positive integer less than or equal to M;
[0047] 0122: After determining the filter area of the N-1th layer, reduce the value of N by 1 and execute step 0121 again: determine the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer, until the filter area of the 1st layer is determined as the filter area of the image to be processed.
[0048] In some embodiments, the first determination module 12 is further used to execute step 0121 and step 0122. The first determination module 12 is further used to determine the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer, where N is a positive integer less than or equal to M; after determining the filter area of the N-1th layer, the value of N is reduced by 1, and the step of determining the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer is performed again, until the filter area of the 1st layer is determined as the filter area of the image to be processed.
[0049] In some embodiments, the processor 30 is further used to determine the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer, where N is a positive integer less than or equal to M; after determining the filter area of the N-1th layer, the value of N is reduced by 1, and the step of determining the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer is performed again, until the filter area of the first layer is determined as the filter area of the image to be processed. That is to say, steps 0121 and 0122 can be implemented by the processor 30.
[0050] Specifically, the filtering layer includes M layers, such as M is 1, 2, 3, 4, 5, etc. The Mth layer is the last layer for filtering processing, and the image data of the filtering area of the image to be processed is processed by the M filtering layers in sequence to complete the filtering to generate a filtered image.
[0051] When determining the filtering area of the image to be processed, the processor 30 may first determine the filtering area of the Mth layer according to the filtering radius of the Mth layer and the image area where the image to be processed is located.
[0052] The size of the filtered image can be the same as that of the image to be processed. After filtering the filter area of the Mth layer, only the image data in the image area where the image to be processed is located is retained to generate the filter area. Therefore, the filter area of the Mth layer can be determined according to the image area where the image to be processed is located and the filter radius of the Mth layer. For example, if the filter radius is 1 pixel, a row of pixels is added to the upper and lower edges of the image area where the image to be processed is located, and a column of pixels is added to the left and right edges to serve as the filter area of the Mth layer. It can be understood that after filtering of each filter layer, the image data of the image area where the image to be processed is located may change.
[0053] Then, the processor 30 determines the filtering area of the N-1th layer according to the filtering radius of the N-1th layer and the filtering area of the Nth layer; see Figure 8 , taking M as 4 as an example, the first layer L1 to the fourth layer L4 are respectively. Initially, the value of N is equal to the value of M, that is, the processor 30 determines the filter area of the third layer according to the filter radius of the third layer and the filter area of the fourth layer; after determining the filter area of the third layer, the value of N is reduced by 1 to 3, and the processor 30 determines the filter area of the second layer according to the filter radius of the second layer and the filter area of the third layer; after determining the filter area of the second layer, the value of N is reduced by 1 again to 2, and the processor 30 determines the filter area of the first layer according to the filter radius of the first layer and the filter area of the second layer. After determining the filter area of the first layer, it can be used as the filter area of the image to be processed.
[0054] Please refer again Figure 2 , Figure 3 and Fig. 9 In some embodiments, the filter layer includes a filter operator, and the filter area of the filter operator of the Mth layer is determined according to the filter radius of the filter operator of the Mth layer and the image area where the image to be processed is located. Step 0121 also includes:
[0055] 01211: Determine the filtering area of each filtering operator in the N-1th layer according to the filtering radius of the filtering operator in the N-1th layer and the filtering area of the filtering operator in the Nth layer.
[0056] In some implementations, the first determination module 12 is further used to execute step 01211. That is, the first determination module 12 is further used to determine the filtering area of each filtering operator of the N-1th layer according to the filtering radius of the filtering operator of the N-1th layer and the filtering area of the filtering operator of the Nth layer.
[0057] In some embodiments, the processor 30 is further configured to determine the filtering area of each filtering operator of the N-1th layer according to the filtering radius of the filtering operator of the N-1th layer and the filtering area of the filtering operator of the Nth layer. That is, step 01211 can be implemented by the processor 30.
[0058] Specifically, each filter layer may include one or more filter operators, and each filter operator may filter the output image input by the previous filter layer separately to generate an intermediate filter image. Figure 8 , there are 4 filter layers, the first layer L1 includes 1 filter operator (ie, OP1), the second layer L2 includes 3 filter operators (ie, OP2, OP3 and OP4), the third layer L3 includes 2 filter operators (ie, OP5 and OP6), and the fourth layer L4 includes 1 filter operator (ie, OP7). 109*109, 107*107
[0059] The first layer of filter operator OP1 is used to process the image data of the filter area of the image to be processed (i.e. Figure 8After filtering the image P0 in the image, a first intermediate filtered image P1 is output; the first intermediate filtered image P1 is respectively input to OP2, OP3 and OP4 of the second layer, OP2, OP3 and OP4 can process the first intermediate filtered image P1 respectively, and three second intermediate filtered images P2 can be output; the three second intermediate filtered images P2 are respectively input to OP5 (processing the second intermediate filtered images P2 of OP2 and OP3) and OP6 (processing the second intermediate filtered image P2 of OP4) of the third layer, OP5 and OP6 can process the second intermediate filtered image P2 respectively, and two third intermediate filtered images P3 can be output; the two third intermediate filtered images P3 are both input to OP7 of the fourth layer, OP7 processes the two third intermediate filtered images P3, and a fourth intermediate filtered image P4 can be output, and the fourth intermediate filtered image P4 can be output as the final filtered image.
[0060] according to Figure 8 As shown, there are associations between different filter operators, OP1 and OP2, OP1 and OP3, OP1 and OP4, OP2 and OP5, OP3 and OP5, OP4 and OP6, OP5 and OP7, OP6 and OP7. The associated filter operator means that the output of the filter operator of the previous layer is used as the input of the filter operator of the next layer.
[0061] When calculating the filtering area of each filtering operator of the N-1th layer, first determine the filtering area of the filtering operator of the Mth layer (i.e., the filtering area of OP7) according to the filtering radius of the filtering operator of the Mth layer and the image area where the image to be processed is located. Then, after determining the filtering operator of the Nth layer associated with each filtering operator of the N-1th layer, the filtering area of each filtering operator of the N-1th layer can be determined according to the filtering radius of the filtering operator of the N-1th layer and the filtering area of the filtering operator of the Nth layer.
[0062] For example, if the value of N is equal to the value of M of 4, the filter area of OP5 is determined according to the filter area of OP7 and the filter radius of OP5; the filter area of OP6 is determined according to the filter area of OP7 and the filter radius of OP6. In this way, the filter area of each filter operator in each filter layer is calculated layer by layer until the filter area of the filter operator OP1 of the first layer is calculated, which is the filter area of the image to be processed.
[0063] It can be understood that a filter operator of the N-1th layer may be associated with multiple filter operators of the Nth layer. The association of the first filter operator of the N-1th layer with the second filter operator of the Nth layer includes that the output data of the filter operator of the N-1th layer is the input data of the filter operator of the Nth layer, the first filter operator is any one of the filter operators of the N-1th layer, and the second filter operator is any one of the filter operators of the Nth layer.
[0064] For example, Figure 8 As shown, the first filter operator (i.e., OP1) of the N-1th layer (taking the 1st layer as an example) is associated with multiple second filter operators (i.e., OP2, OP3, and OP4) of the Nth layer (i.e., the 2nd layer). The first intermediate filter image P1 output by OP1 is the input image of OP2, OP3, and OP4. In order to ensure that the first intermediate filter image P1 contains all the image data required by OP2, OP3, and OP4, after determining the filter area of OP2, OP3, and OP4, the intermediate filter area can be determined according to the filter area of each second filter operator of the Nth layer (i.e., the filter area of OP2, OP3, and OP4).
[0065] like Fig.10 As shown, the union of the filter areas of all filter operators of the current filter layer (such as the Nth layer) can be used as the intermediate filter area S2, such as the union of the filter areas of OP2, OP3 and OP4 (that is, the second intermediate filter images P2 output by OP2, OP3 and OP4 respectively) can be used as the intermediate filter area S2. Since the filter area is generally rectangular, the rectangular area corresponding to the union of the filter areas of OP2, OP3 and OP4 can be used as the intermediate filter area S2.
[0066] Finally, the filtering area of the first filtering operator is determined according to the filtering radius of the intermediate filtering area and the first filtering operator. Thus, the image data output by the first filtering operator after filtering processing contains all the image data required by OP2, OP3 and OP4. In this way, the image data of all filtering operators of the current filtering layer can be read at one time.
[0067] Alternatively, the union of the filter areas of multiple filter operators (at least two) of the current filter layer can be used as the intermediate filter area S2, such as the union of the filter areas of OP2 and OP3, the union of the filter areas of OP2 and OP4, or the union of the filter areas of OP3 and OP4.
[0068] Then, the filtering area of the first filtering operator is determined according to the intermediate filtering area and the filtering radius of the first filtering operator, so that the image data output by the first filtering operator after filtering processing contains all the image data required by multiple filtering operators, thereby obtaining the image data of multiple filtering operators through one reading.
[0069] For the case where there are multiple first filter operators (i.e., OP5 and OP6) in the N-1th layer (taking the 3rd layer as an example) and they are associated with OP7 of the Nth layer (i.e., the 4th layer), the two third intermediate filter images P3 generated by the multiple OP5 and OP6 can be input into OP7, and OP7 respectively filters the two third intermediate filter images P3 to obtain two fourth intermediate filter images P4, and then fuses the two fourth intermediate filter images P4, such as weighted fusion of pixels in the same position in the two fourth intermediate filter images P4, and finally outputs a fused fourth intermediate filter image P4.
[0070] See also Figure 2 , Figure 3 and Fig.11 In some embodiments, the data processing method further comprises the following steps:
[0071] 014: determining the coordinate offset of each second filtering operator according to the vertex coordinates of the filtering area of each second filtering operator and the vertex coordinates of the middle filtering area;
[0072] 015: when filtering the Nth layer, determining the vertex coordinates of the filtering area of each second filtering operator according to the coordinate offset of each second filtering operator and the vertex coordinates of the output image of the N-1th layer;
[0073] 016: From the output image, obtain and process the image data corresponding to the filtering area of each second filtering operator.
[0074] In some embodiments, the data processing device 10 further includes a second determination module 14, a third determination module 15, and a first processing module 16. The second determination module 14, the third determination module 15, and the first processing module 16 are used to execute step 014, step 015, and step 016, respectively. That is, the second determination module 14 is used to determine the coordinate offset of each second filter operator according to the vertex coordinates of the filter area of each second filter operator and the vertex coordinates of the intermediate filter area; the third determination module 15 is used to determine the vertex coordinates of the filter area of each second filter operator according to the coordinate offset of each second filter operator and the vertex coordinates of the output image of the N-1th layer when filtering the Nth layer; the first processing module 16 is used to obtain and process the image data corresponding to the filter area of each second filter operator from the output image.
[0075] In some embodiments, the processor 30 is further configured to determine the coordinate offset of each second filter operator according to the vertex coordinates of the filter area of each second filter operator and the vertex coordinates of the intermediate filter area; when filtering the Nth layer, determine the vertex coordinates of the filter area of each second filter operator according to the coordinate offset of each second filter operator and the vertex coordinates of the output image of the N-1th layer; and obtain and process the image data corresponding to the filter area of each second filter operator from the output image. That is, steps 014, 015, and 016 can be implemented by the processor 30.
[0076] Specifically, since a first filter operator may be associated with multiple second filter operators, and the filter area of the first filter operator is determined according to the filter areas of multiple second filter operators (such as the union of the filter areas of multiple second filter operators, that is, the middle filter area) and the filter radius of the first filter operator.
[0077] In order to ensure that the second filter operator can accurately obtain the required image data from the first intermediate filter image P1 (corresponding to the intermediate filter area) output by the first filter operator, it is necessary to determine the position of the filter area of the second filter operator in the intermediate filter area.
[0078] Therefore, the processor 30 can determine the coordinate offset of each second filtering operator based on the vertex coordinates of the filtering area of each second filtering operator and the vertex coordinates of the intermediate filtering area; so that when the first filtering operator (such as the first filtering operator of the first layer) outputs the first intermediate filtering image P1 (i.e., the intermediate filtering area), the second filtering operator can obtain the image data required for filtering from the first intermediate filtering image P1.
[0079] For example, when filtering is performed at the Nth layer (such as the 2nd layer), the second filtering operator can calculate the vertex coordinates of the filtering area of the second filtering operator based on its corresponding coordinate offset and the vertex coordinates of the first intermediate filtering image P1, thereby obtaining image data corresponding to the filtering area of the second filtering operator in the first intermediate filtering image P1, and performing filtering processing to ensure the accuracy of data acquisition and processing.
[0080] See also Figure 2 , Figure 3 and Fig.12 In some embodiments, the filter radius includes a fifth filter radius and a sixth filter radius, and the data processing method further includes the following steps:
[0081] 017: Determine a first difference between a fifth filter radius of a filter area of each second filter operator and a fifth filter radius of a middle filter area, and a second difference between a sixth filter radius of a filter area of each second filter operator and a sixth filter radius of a middle filter area;
[0082] 018: when filtering the Nth layer, determine the width of the filtering area of each second filtering operator according to the first difference value of each second filtering operator and the width of the output image of the N-1th layer, and determine the height of the filtering area of each second filtering operator according to the second difference value of each second filtering operator and the height of the output image of the N-1th layer;
[0083] 019: Determine a filtering area of each second filtering operator according to the width and height of each second filtering operator; and
[0084] 020: From the output image, obtain and process image data corresponding to the filtering area of each second filtering operator.
[0085] In some embodiments, the data processing device 10 further includes a fourth determination module 17, a fifth determination module 18, a sixth determination module 19 and a second processing module 20. The fourth determination module 17, the fifth determination module 18, the sixth determination module 19 and the second processing module 20 are used to execute step 017, step 018, step 019 and step 020 respectively. That is, the fourth determination module 17 is used to determine the first difference between the fifth filter radius of the filter area of each second filter operator and the fifth filter radius of the intermediate filter area, and the second difference between the sixth filter radius of the filter area of each second filter operator and the sixth filter radius of the intermediate filter area; the fifth determination module 18 is used to determine the width of the filter area of each second filter operator according to the first difference of each second filter operator and the width of the output image of the N-1 layer when filtering the Nth layer, and to determine the height of the filter area of each second filter operator according to the second difference of each second filter operator and the height of the output image of the N-1 layer; the sixth determination module 19 is used to determine the filter area of each second filter operator according to the width and height of each second filter operator; the second processing module 20 is used to obtain and process the image data corresponding to the filter area of each second filter operator from the output image.
[0086] In some embodiments, the processor 30 is further used to determine the first difference between the fifth filter radius of the filter area of each second filter operator and the fifth filter radius of the intermediate filter area, and the second difference between the sixth filter radius of the filter area of each second filter operator and the sixth filter radius of the intermediate filter area; when filtering the Nth layer, the width of the filter area of each second filter operator is determined according to the first difference of each second filter operator and the width of the output image of the N-1th layer, and the height of the filter area of each second filter operator is determined according to the second difference of each second filter operator and the height of the output image of the N-1th layer; the filter area of each second filter operator is determined according to the width and height of each second filter operator; and the image data corresponding to the filter area of each second filter operator is obtained and processed from the output image. That is, steps 017, 018, 019 and 020 can be implemented by the processor 30.
[0087] Specifically, when the filtering radius of the filtering operator includes only the fifth filtering radius and the sixth filtering radius, that is, the filtering radii on the upper and lower sides of the filtering area are the same, both are the fifth filtering radius, and the filtering radii on the left and right are the same, both are the sixth filtering radius.
[0088] The processor 30 can determine the first difference between the fifth filter radius of the filter area of each second filter operator and the fifth filter radius of the intermediate filter area, and the second difference between the sixth filter radius of the filter area of each second filter operator and the sixth filter radius of the intermediate filter area; thereby determining the offset of the filter radius of the filter area of each second filter operator relative to the filter radius of the intermediate filter area, so that when the output image of the first filter operator (such as the first filter operator of the first layer) (i.e., the first intermediate filter image P1, corresponding to the intermediate filter area) is obtained, the second filter operator can obtain the image data required for filtering from the first intermediate filter image P1.
[0089] For example, when filtering is performed at the Nth layer (such as the 2nd layer), the second filter operator can be used to perform a difference between the height and width of the first intermediate filter image P1 according to its corresponding first difference and second difference, thereby obtaining the height and width of each second filter operator. Then, the processor 30 determines the filtering area of each second filter operator according to the height and width of the second filter operator. For example, if the first intermediate filter image P1 is 106*106, the first difference and the second difference are 2 and 1 respectively, then the width and height of the second filter operator are 104*102 respectively, that is,
[0090] Therefore, after determining the width and height of the second filter operator, the filter area of each second filter operator can be quickly determined. Finally, the processor 30 obtains the image data corresponding to the filter area of the second filter operator from the first intermediate filter image P1 and performs filtering processing to ensure the accuracy of data acquisition and processing.
[0091] In another embodiment, the filter radius of the four sides of the filter area is the same, then it is only necessary to calculate the difference between the filter radius of the filter area of each second filter operator and the filter radius of the middle filter area. This facilitates the subsequent filtering process, and each filter operator quickly obtains all the image data required for filtering from the output image output by the previous layer.
[0092] In another embodiment, the filter radii of the four sides of the filter area are different, that is, the filter area includes the first filter radius to the fourth filter radius mentioned above. When calculating the difference in the filter radius, it is necessary to calculate the four differences from the first filter radius to the fourth filter radius of each second filter operator and the first filter radius to the fourth filter radius of the middle filter area respectively, so as to facilitate subsequent filtering processing, and each filter operator can quickly obtain all the image data required for filtering from the output image output by the previous layer.
[0093] In an example, see again Figure 8 When the above data processing algorithm is used for data processing, the input data and output data of each filter operator are not downsampled or upsampled; the filter radius of each filter operator is 5 pixels, and in the general method, the output of each filter operator is also planned as [w, h], and it is assumed that w = 128, h = 64.
[0094] For the existing filtering algorithm, data is read and written from the memory 50 (such as dynamic random access memory, DRAM) of the terminal 100 in each filtering layer. The read and write amount of the first to fourth layers of the memory 50 is as shown in Table 1:
[0095] Table 1
[0096] DRAM Read DRAM Write L1 (128+2*5)*(64+2*5)=10212 128*64=8192 L2 10212 128*64*3=8192*3 L3 10212*3 128*64*2=8192*2 L4 10212*2 128*64=8192 total 71484 57344
[0097] When the data processing method of the present application is used for data processing, the read and write volume of the memory 50 is as shown in Table 2:
[0098] Table 2
[0099] DRAM Read DRAM Write total (128+2*5*4)*(64+2*5*4)=17472 w*h=8192
[0100] It can be seen that the use of this method reduces the amount of data read by DRAM from 71484 to 17472, and the amount of data written by DRAM from 57344 to 8192. The amount of reading and writing required by the algorithm is greatly reduced, so that the algorithm efficiency and power consumption performance can also be significantly improved.
[0101] See also Fig.13 , a non-volatile computer-readable storage medium 300 storing a computer program 302 according to an embodiment of the present application, when the computer program 302 is executed by one or more processors 30, enables the processor 30 to execute the data processing method of any of the above embodiments.
[0102] For example, please combine Figure 1 , when the computer program 302 is executed by one or more processors 30, the processor 30 performs the following steps:
[0103] 011: Get the filter radius of multiple preset filter layers;
[0104] 012: Determine a filtering area for the data to be processed according to the filtering radius of the multiple filtering layers; and
[0105] 013: Get the image data of the filter area.
[0106] For another example, please combine Figure 7 When the computer program 302 is executed by one or more processors 30, the processor 30 may further perform the following steps:
[0107] 0121: Determine the filtering area of the N-1th layer according to the filtering radius of the N-1th layer and the filtering area of the Nth layer, where N is a positive integer less than or equal to M;
[0108] 0122: After determining the filter area of the N-1th layer, reduce the value of N by 1 and execute step 0121 again: determine the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer, until the filter area of the 1st layer is determined as the filter area of the image to be processed.
[0109] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0110] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of a program code that includes one or more steps for implementing a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0111] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A data processing method, characterized in that: include: Obtaining the filter radii of the preset multiple filter layers; Determining a filtering area for data to be processed according to the filtering radii of the plurality of filtering layers; and Acquiring image data of the filtering area; The step of determining the filtering area of the data to be processed according to the filtering radius of the plurality of filtering layers comprises: Determining the filtering area according to the filtering radius of the plurality of filtering layers and the position information of the data to be processed; The filtering layer includes M layers, where M is a positive integer, and the filtering area of the Mth layer is determined according to the filtering radius of the Mth layer and the image area where the data to be processed is located. The determining the filtering area of the data to be processed according to the filtering radius and the position information includes: Determine the filtering area of the N-1th layer according to the filtering radius of the N-1th layer and the filtering area of the Nth layer, where N is a positive integer less than or equal to M; After determining the filter area of the N-1th layer, the value of N is reduced by 1, and the step of determining the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer is performed again, until the filter area of the 1st layer is determined as the filter area of the data to be processed.
2. The data processing method according to claim 1, characterized in that: The filtering radius includes a first filtering radius, a second filtering radius, a third filtering radius and a fourth filtering radius, and the first filtering radius, the second filtering radius, the third filtering radius and the fourth filtering radius are all the same; or the first filtering radius and the third filtering radius are the same, and the second filtering radius and the fourth filtering radius are the same; or the first filtering radius, the second filtering radius, the third filtering radius and the fourth filtering radius are different from each other.
3. The data processing method according to claim 1, characterized in that: The filtering layer includes a filtering operator, a filtering area of the filtering operator of the Mth layer is determined according to a filtering radius of the filtering operator of the Mth layer and an image area where the to-be-processed data is located, and determining the filtering area of the N-1th layer according to the filtering radius of the N-1th layer and the filtering area of the Nth layer includes: The filtering area of each of the filtering operators in the N-1th layer is determined according to the filtering radius of the filtering operator in the N-1th layer and the filtering area of the filtering operator in the Nth layer.
4. The data processing method according to claim 3, characterized in that: The first filter operator of the N-1th layer is associated with the second filter operator of the Nth layer, and the association between the first filter operator of the N-1th layer and the second filter operator of the Nth layer includes that the output data of the filter operator of the N-1th layer is the input data of the filter operator of the Nth layer, the first filter operator is any one of the filter operators of the N-1th layer, and the second filter operator is any one of the filter operators of the Nth layer; The determining the filtering area of each filtering operator of the N-1th layer according to the filtering radius of the filtering operator of the N-1th layer and the filtering area of the filtering operator of the Nth layer comprises: Determine an intermediate filtering area according to the filtering area of each second filtering operator of the Nth layer; and The filtering area of the first filtering operator is determined according to the intermediate filtering area and the filtering radius of the first filtering operator.
5. The data processing method according to claim 4, characterized in that: There are multiple second filter operators associated with the first filter operator, and determining the intermediate filter area according to the filter area of each second filter operator of the Nth layer includes: The intermediate filtering area is determined according to a union of filtering areas of a plurality of the second filtering operators.
6. The data processing method according to claim 4, characterized in that: Also includes: Determining a coordinate offset of each second filtering operator according to vertex coordinates of a filtering area of each second filtering operator and vertex coordinates of the intermediate filtering area; When filtering the Nth layer, determining the vertex coordinates of the filtering area of each second filtering operator according to the coordinate offset of each second filtering operator and the vertex coordinates of the output image of the N-1th layer; Image data corresponding to the filtering area of each second filtering operator is obtained and processed from the output image.
7. The data processing method according to claim 4, characterized in that: The filtering radius includes a fifth filtering radius and a sixth filtering radius; and the data processing method further includes: Determine a first difference between the fifth filter radius of the filter area of each second filter operator and the fifth filter radius of the intermediate filter area, and a second difference between the sixth filter radius of the filter area of each second filter operator and the sixth filter radius of the intermediate filter area; When filtering the Nth layer, determining the height of the filtering area of each second filtering operator according to the first difference of each second filtering operator and the width of the output image of the N-1th layer, and determining the width of the filtering area of each second filtering operator according to the second difference of each second filtering operator and the height of the output image of the N-1th layer; Determine a filtering area of each of the second filtering operators according to the width and height of each of the second filtering operators; and Image data corresponding to the filtering area of each second filtering operator is obtained and processed from the output image.
8. The data processing method according to claim 1, characterized in that: Also includes: The image data in the filtering area is filtered to generate a filtered image.
9. A data processing device, characterized in that: include: A first acquisition module, used to acquire the filter radius of a plurality of preset filter layers; A first determining module, configured to determine a filtering area of the data to be processed according to the filtering radius of the plurality of filtering layers; and A second acquisition module, used to acquire image data of the filtering area; The first determining module is used to determine the filtering area according to the filtering radius of the plurality of filtering layers and the position information of the data to be processed; The filter layer includes M layers, where M is a positive integer, and the filter area of the Mth layer is determined according to the filter radius of the Mth layer and the image area where the data to be processed is located; the first determination module is also used to determine the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer, where N is a positive integer less than or equal to M; after determining the filter area of the N-1th layer, the value of N is reduced by 1, and the step of determining the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer is performed again, until the filter area of the 1st layer is determined as the filter area of the data to be processed.
10. A terminal, characterized in that: The method comprises a processor, wherein the processor is used to obtain filter radii of a plurality of preset filter layers; determine a filter area of data to be processed according to the filter radii of the plurality of filter layers; and obtain image data of the filter area; The processor is used to determine the filtering area according to the filtering radius of the plurality of filtering layers and the position information of the data to be processed; The filter layer includes M layers, where M is a positive integer, and the filter area of the Mth layer is determined according to the filter radius of the Mth layer and the image area where the data to be processed is located; the processor is also used to determine the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer, where N is a positive integer less than or equal to M; after determining the filter area of the N-1th layer, the value of N is reduced by 1, and the step of determining the filter area of the N-1th layer according to the filter radius of the N-1th layer and the filter area of the Nth layer is performed again, until the filter area of the 1st layer is determined as the filter area of the data to be processed.
11. A non-volatile computer-readable storage medium comprising a computer program, wherein when the computer program is executed by a processor, the processor executes the data processing method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Panoramic video region of interest description method and coding method
CN106412582A
Image processing method and device
CN112639867A
