A picture processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a picture processing method and device, electronic equipment and storage medium, wherein the method comprises: obtaining a source picture with source pixel values and a target picture without target pixel values; splitting the target picture based on a preset split size to obtain a plurality of target sub-pictures; determining a source sub-picture corresponding to the target sub-picture in the source picture based on the target sub-picture, and judging whether the pixel amount contained in the source sub-picture is consistent with a preset threshold; if yes, determining each target pixel value in the target sub-picture based on each source pixel value in the source sub-picture. The present disclosure splits based on the preset split size and then performs picture processing, so that the pixel amount contained in the source sub-picture corresponding to the split target sub-picture can better meet the preset threshold set for SARM, thereby ensuring the conversion relationship between the source picture and the target picture in the case of effective splitting, and the overall picture processing performance is better.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of picture processing, and in particular, to a picture processing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In a system on chip (SOC) design for processing pictures, a hardware accelerator is generally added to complete various transformation processing of pictures, such as affine transformation and size transformation.

[0003] In the process of processing pictures by using a hardware accelerator, data of a source picture is generally stored in a double data rate (DDR) memory. At the same time, in order to reduce the bandwidth pressure of the DDR, an internal static random-access memory (SRAM) module is generally added to the hardware accelerator, and the source picture is read from the DDR to the SRAM for processing.

[0004] How to better ensure the processing performance of picture processing based on the SARM is a problem to be solved. SUMMARY

[0005] Embodiments of the present disclosure provide at least a picture processing method, device, electronic device, and storage medium to improve the performance of picture processing.

[0006] In a first aspect, embodiments of the present disclosure provide a picture processing method, comprising:

[0007] obtaining a source picture with source pixel values, and obtaining a target picture without target pixel values;

[0008] splitting the target picture based on a preset split size to obtain a plurality of target sub-pictures;

[0009] determining a source sub-picture corresponding to the target sub-picture in the source picture based on the target sub-picture, and determining whether a pixel amount contained in the source sub-picture is consistent with a preset threshold;

[0010] if yes, determining each target pixel value in the target sub-picture based on each source pixel value in the source sub-picture.

[0011] In a possible implementation, the method further comprises:

[0012] if no, adjusting the preset split size to re-split the target picture.

[0013] In a possible implementation, the method further comprises:

[0014] If the pixel amount contained in the source subgraph is greater than a preset threshold, the preset splitting size is reduced; if the pixel amount contained in the source subgraph is less than a preset threshold, the preset splitting size is increased.

[0015] In a possible implementation, the splitting of the target graph based on the preset splitting size to obtain a plurality of target subgraphs comprises:

[0016] The first preset splitting size is determined by a splitting square method, and the second preset splitting size is determined by a splitting strip method.

[0017] The target graph is split based on the first preset splitting size and the second preset splitting size, respectively, to obtain a splitting square target subgraph and a splitting strip target subgraph.

[0018] In a possible implementation, the width and the height of the first preset splitting size are both less than the width and the height of the target graph, and the width or the height of the second preset splitting size corresponds to the width or the height of the target graph.

[0019] In a possible implementation, the method further comprises:

[0020] The splitting square source subgraph and the splitting strip source subgraph corresponding to the splitting square target subgraph and the splitting strip target subgraph are determined, and when the pixel amount contained in the splitting square source subgraph and the splitting strip source subgraph is consistent with a preset threshold, the target pixel value in each target subgraph is determined by selecting the source subgraph with the least quantity from the splitting square source subgraph and the splitting strip source subgraph.

[0021] In a possible implementation, the pixel amount contained in the splitting square source subgraph and the splitting strip source subgraph is consistent with a preset threshold, and the number of the splitting square source subgraph and the splitting strip source subgraph is consistent, and the target pixel value in the splitting strip target subgraph is determined by selecting the splitting strip source subgraph.

[0022] In a possible implementation, the first preset splitting size is determined by the splitting square method, comprising:

[0023] At least one first split line in a picture row direction and at least one second split line in a picture column direction are determined.

[0024] The target graph is split according to the at least one first split line and the at least one second split line to obtain a plurality of splitting square target subgraphs.

[0025] In a possible implementation, the second preset splitting size is determined by the splitting strip method, comprising:

[0026] At least one third split line in a picture row direction is determined.

[0027] According to at least one third split line, the target image is split to obtain a plurality of split target sub-images; or

[0028] A fourth split line in a picture column direction is determined.

[0029] According to at least one fourth split line, the target image is split to obtain a plurality of split target sub-images.

[0030] In a possible implementation, the source sub-image corresponding to the target sub-image in the source image is determined by:

[0031] A conversion relationship between a first image coordinate system in which the source image is located and a second image coordinate system in which the target image is located is obtained.

[0032] The source sub-image corresponding to the target sub-image in the source image is determined based on the conversion relationship.

[0033] In a possible implementation, the amount of pixels contained in the source sub-image is determined by:

[0034] For each to-be-determined target pixel point in the target sub-image, a source pixel point corresponding to the to-be-determined target pixel point is determined from the source sub-image based on the conversion relationship.

[0035] The amount of pixels contained in the source sub-image corresponding to the target sub-image is determined based on the source pixel points respectively corresponding to the to-be-determined target pixel points.

[0036] In a possible implementation, the target pixel value in the target sub-image is determined based on the source pixel value in the source sub-image by:

[0037] For each to-be-determined target pixel point in the target sub-image, a source pixel value of the source pixel point corresponding to the to-be-determined target pixel point is determined; and the source pixel value is determined as the target pixel value of the to-be-determined target pixel point.

[0038] In a second aspect, the disclosure also provides a picture processing device, comprising:

[0039] An acquisition module is configured to acquire a source image with source pixel values and a target image without target pixel values.

[0040] A splitting module is configured to split the target image based on a preset splitting size to obtain a plurality of target sub-images.

[0041] A judging module is configured to determine a source sub-image corresponding to the target sub-image in the source image based on the target sub-image, and judge whether the amount of pixels contained in the source sub-image is consistent with a preset threshold.

[0042] determining module, configured to determine each target pixel value in the target subgraph based on each source pixel value in the source subgraph.

[0043] In a third aspect, the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the picture processing method in any one of the first aspect and various embodiments thereof.

[0044] In a fourth aspect, the present disclosure provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a processor, the picture processing method in any one of the first aspect and various embodiments thereof is executed.

[0045] By using the picture processing method, device, electronic device and storage medium, in the case of obtaining a source graph with source pixel values and a target graph without target pixel values, the target graph is split based on a preset split size, then the source subgraph corresponding to the split target subgraph is determined, and it is judged whether the pixel amount contained in the source subgraph is consistent with the preset threshold, in the case of consistency, each target pixel value in the target subgraph is determined based on each source pixel value in the source subgraph. The present disclosure splits based on the preset split size and then processes the picture, which mainly considers that the size of SRAM is usually not particularly large, so that the pixel amount contained in the source subgraph corresponding to the split target subgraph can better meet the preset threshold set for SARM, so as to ensure the conversion relationship between the source graph and the target graph in the case of effective splitting, and the overall picture processing performance is better.

[0046] Other advantages of the present disclosure will be described in more detail in conjunction with the following description and drawings.

[0047] It should be understood that the above description is only a summary of the technical solutions of the present disclosure, so as to enable a clearer understanding of the technical means of the present disclosure, so that the content of the specification can be implemented. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following specific embodiments of the present disclosure are described. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly introduced below. The drawings incorporated into the specification and form a part of the specification, which show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show some of the embodiments of the present disclosure, and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor. Moreover, the same reference numbers are used to represent the same components throughout the drawings. In the drawings:

[0049] Figure 1 A schematic diagram of a disassembled side view is shown according to an embodiment of the present disclosure;

[0050] Figure 2 A flowchart of a picture processing method is shown according to an embodiment of the present disclosure;

[0051] Figure 3 A schematic diagram of a disassembled strip view is shown according to an embodiment of the present disclosure;

[0052] Figure 4 A schematic diagram of a picture processing device is shown according to an embodiment of the present disclosure;

[0053] Figure 5 A schematic diagram of an electronic device is shown according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0054] The exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While the exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms without being limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0055] In the description of the embodiments of the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate that there is existence of the features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts or combinations thereof exist.

[0056] Unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" herein is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone.

[0057] The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0058] Research has shown that during image processing using hardware accelerators, the source image data is typically stored in Double Data Rate (DDR) memory. Simultaneously, to reduce the bandwidth pressure on DDR, an internal Static Random-Access Memory (SRAM) module is usually added to the hardware accelerator to read the source image from DDR into SRAM before processing.

[0059] Considering that SRAM is generally not very large, it is often necessary to split the source image and perform transformation processing based on the split results. In the transformed target image, continuous coordinate points are discontinuous points distributed in different rows / columns in the source image. For example, using the bilinear interpolation algorithm, calculating one point in the target image requires reading the data information of 4 pixels from the source image, i.e., DDR, which places higher demands on the access efficiency of DDR.

[0060] Therefore, how to read the source image data more quickly while ensuring the splitting effect becomes a problem that needs to be solved.

[0061] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, this disclosure provides at least one image processing method, apparatus, electronic device and storage medium to split the image by a preset split size and to compare the split source sub-images with a preset threshold, so as to read the corresponding source pixel values ​​more quickly for image processing and to achieve better processing performance.

[0062] To clearly illustrate the embodiments of this disclosure, some concepts that may appear in subsequent embodiments will be introduced first.

[0063] 1. Slice splitting:

[0064] like Figure 1 As shown, the rectangular section in the middle of the left image is the source image before transformation, and the large black box section (including four smaller black boxes) in the right image is the target image after transformation. The diamond-shaped part within the large black box is the deformed shape of the source image, and the other parts are fill values. Each small black box section is called a Slice, or simply a Slice.

[0065] 2. Graphic Transformation:

[0066] The graphic transformation here can be an affine transformation, a size transformation, etc. Take the affine transformation as an example: that is, transforming the source graph of the left rectangular part into the target graph of the right large black frame part (including the middle diamond part and other filled parts).

[0067] To facilitate the understanding of the present embodiment, first, a picture processing method disclosed by the present embodiment is described in detail. The execution subject of the picture processing method provided by the present embodiment is generally an electronic device with certain computing capability, which includes, for example, a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the picture processing method can be realized by a processor calling computer readable instructions stored in a memory.

[0068] Referring to FIG. 1, Figure 2 The picture processing method provided by the present embodiment includes steps S201-S204, wherein:

[0069] S201: obtaining a source graph with source pixel values, and obtaining a target graph without target pixel values;

[0070] S202: splitting the target graph based on a preset split size to obtain a plurality of target subgraphs;

[0071] S203: determining a source subgraph corresponding to the target subgraph in the source graph based on the target subgraph, and judging whether the pixel amount contained in the source subgraph is consistent with a preset threshold;

[0072] S204: if yes, determining each target pixel value in the target subgraph based on each source pixel value in the source subgraph.

[0073] To facilitate the understanding of the picture processing method provided by the present embodiment, the application scenario of the method is described in detail as follows. The picture processing method in the present embodiment can be mainly applied to the field of picture processing, especially for the transformation processing of large-size high-definition pictures such as 4K size. This is mainly to better adapt to the size limitation of SRAM, and at the same time, it is also necessary to ensure the continuity of SRAM reading as much as possible. In this way, after the source graph is split into a plurality of Slice small blocks, the pixel information of each target subgraph after transformation can be calculated respectively to obtain the transformation result of the whole source graph.

[0074] Considering that the pixels of the source sub-image to be read are not continuous during the image transformation process, for example, the source sub-image corresponding to the coordinate point (0,0) in the target image may not be the coordinate point (0,0), but the coordinate point (3,4). Therefore, in the actual splitting process, it is necessary not only to consider the splitting size, but also to consider whether the number of pixels of the corresponding source sub-image after splitting is suitable for reading from SRAM.

[0075] Taking affine transformation as an example, when actually determining the number of pixels in the source subimage, such as... Figure 1 As shown, in the left image, the slice portion (corresponding to a small diamond) represents the position of the target image slice on the source image. There is a circumscribed rectangle outside the source image slice. The dashed box represents the actual size of the source image slice to be read after removing invalid areas, thus determining the pixel count of each source sub-image. This is mainly because the source and target image data are stored contiguously in DDR; therefore, both reading from the source image and writing to the target image generally only support... Figure 1 The rectangle is split into sections. In the left image, the area outside the dotted line represents regions where data cannot be retrieved and needs to be discarded during reading. In the right image, the area outside the rhombus within the rectangle represents data that needs to be filled according to specific requirements. This data cannot be discarded and must be written sequentially in memory. In practical applications, this can be achieved by reading the corresponding source pixel values ​​from the source sub-image to determine the target pixel values ​​of the target sub-image.

[0076] In image processing such as affine transformations, the correspondence between the target sub-image and the source sub-image is known in advance. Then, the corresponding source sub-image can be determined for each target sub-image. Thus, given the processing result of the target sub-image corresponding to each source sub-image, the processing result of each target sub-image is combined to determine the processing result of the entire target image. This ensures the processing performance of the split sub-images while ensuring the splitting effect.

[0077] Image segmentation can be achieved by using a preset segmentation size. This segmentation size can be limited to the row direction, the column direction, or both; there are no specific limitations. Furthermore, the actual segmentation size should not be too large or too small. A segmentation size that is too small may result in too many sub-images and fail to effectively utilize SRAM's read performance. A segmentation size that is too large may not meet SRAM's storage capacity. Therefore, the segmentation size can be determined based on the actual size of the SRAM and the number of pixels required to be read per slice.

[0078] In the embodiments of the present disclosure, the target picture can be split in a slice manner and a strip manner, and meanwhile, the source picture can be split synchronously, that is, the source picture is the picture before transformation, the target picture is the picture after transformation, and consistent splitting manners are adopted, and in a case where a plurality of target sub-pictures are obtained by splitting, for each target sub-picture, a corresponding source sub-picture can be determined.

[0079] The slice manner refers to splitting in both the row direction and the column direction of the picture, that is, the height and the width of the Slice split from the source picture are close to each other, and both the height and the width are not equal to the height and the width of the source picture, and in actual application, the height and the width of the Slice can be increased simultaneously to make the height and the width close to each other, and the number of Slices can be reduced in a case where the rotation angle of the affine transformation is large; the strip manner refers to splitting in the row direction or the column direction of the picture, that is, the height or the width of the Slice split from the source picture is equal to the height or the width of the source picture in one direction.

[0080] For the split source sub-picture, the number of pixels (corresponding to the pixel amount) required to be read for each source sub-picture can be determined, and in a case where the pixel amount of the source sub-picture is consistent with a preset threshold, the pixel information of the target sub-picture after transformation processing can be obtained.

[0081] The preset threshold can be a specific value or a value range, and in actual application, the storage size based on SRAM can be determined, and no specific limitation is made here.

[0082] In the process of determining the pixel information of the target sub-picture, the transformation relationship between the source picture and the target picture can be mainly based on, in a case where the transformation relationship is determined, the corresponding coordinate projection relationship can be obtained, and then the target pixel value of the target sub-picture is determined based on the source pixel value of the source sub-picture.

[0083] In the picture processing method provided by the embodiments of the present disclosure, in a case where the pixel amount contained in the source sub-picture is different from the preset threshold, it is indicated to some extent that the SRAM cannot meet the reading requirement, at this time, the preset splitting size needs to be adjusted, and the target picture needs to be split again, for example, another splitting manner can be used, and for another example, the specific splitting size can be adjusted.

[0084] In actual application, if the pixel amount contained in the source sub-picture is greater than the preset threshold, it is indicated to some extent that the data amount required to be read by the SRAM exceeds the load amount of the SRAM, at this time, the preset splitting size needs to be reduced to reduce the pixel amount so as to more possibly meet the performance of the SRAM; if the pixel amount contained in the source sub-picture is less than the preset threshold, it is indicated to some extent that the utilization rate of the SRAM is low, at this time, the preset splitting size needs to be increased to increase the pixel amount so as to more possibly improve the utilization rate of the SRAM.

[0085] In reducing or increasing the preset split size, the specific size to be reduced or increased can be searched for local optimization to determine the size of the final required transformation.

[0086] Considering the key role of picture splitting in the whole picture processing, the specific process of picture splitting can be described as follows, including the following steps:

[0087] Step one, determine the first preset split size by splitting the square way, and determine the second preset split size by splitting the strip way;

[0088] Step two, based on the first preset split size and the second preset split size, respectively split the target graph to obtain the square target subgraph and the strip target subgraph.

[0089] The splitting strategy can be: first set the height (dest_h) and width (dest_w) of the target subgraph slice to 1, calculate the total number of split slices as ceil(H' / dest_h)*ceil(W' / dest_w) according to the total size H' and W' of the target graph, obtain the coordinates of the four vertices for each slice of the target subgraph, and calculate the corresponding four vertices on the source graph by the known transformation formula. The four vertices can determine the outer rectangle of the corresponding source subgraph. Since the outer rectangle may include invalid areas, the intersection rectangle with the source graph is taken as the slice of the source subgraph, and then the size of this slice is calculated to see if it exceeds the SRAM size. When all slices complete the above calculation and meet the SRAM size, i.e. match the preset threshold, it means that the height (dest_h) and width (dest_w) of the target subgraph slice are set effective. Then add 1 to dest_h or dest_w, repeat the above round of calculation, until the maximum slice in the source subgraph is close to the total SRAM size, and the dest_h and dest_w at this time are the final split search results.

[0090] In the above, the height (dest_h) and width (dest_w) of the target subgraph slice are the preset split size, which can be determined according to the size of the target graph and the source graph, or set according to human experience or neural network learning. In this disclosure, the specific selection of the split size is not specifically limited, but the form of splitting is limited.

[0091] Here, in order to more likely ensure the splitting effect, the first preset split size can be determined by splitting the square way, and the second preset split size can be determined by splitting the strip way, and then based on the first preset split size and the second preset split size, the target graph is split to obtain the square target subgraph and the strip target subgraph.

[0092] The first preset split size determined by the splitting method has a width and height smaller than the width and height of the target drawing, such as... Figure 1 As shown, the splitting method splits the image in both directions simultaneously, so that the width of the split sub-image is smaller than the width of the target image, and the height of the split sub-image is smaller than the height of the target image.

[0093] For the second preset split size determined by the splitting method, when splitting in the row direction, its width is equal to the width of the target image, such as... Figure 3 The diagram shown illustrates the strip splitting method. In addition, when splitting in the column direction, its height is equal to the height of the target image, but no illustration is provided here.

[0094] Considering that in the process of transforming the various sub-images of a source image, more source sub-images will generally require a longer image processing time, and vice versa. Figure 1 To a certain extent, the required image processing time is shorter. Based on this, if both the square and strip splitting methods can meet the preset thresholds, then the source sub-image with the fewest number of square and strip splitting source sub-images can be selected, and the target pixel value of the corresponding target sub-image can be determined based on this.

[0095] In the actual process of demolition, such as Figure 1 As shown, changes in the size or rotation angle of the source image lead to uneven splitting. Except for the first slice, which can fully utilize the SRAM, the utilization rate of other slices is not high, which may result in a larger number of slices. In addition, since the splitting is performed in the width direction of the image, when reading data from DDR to SRAM for each slice, the continuity of the source image in the width direction is disrupted (i.e., the beginning of each row is not continuous with the end of the previous row), thus reducing the reading efficiency.

[0096] However, for slice splitting, the width of the source image slice is fixed to the width of the source image. Therefore, only the height of the slice needs to be searched. The specific search method can be referenced from the slice splitting method. For example... Figure 3 As shown, by using the splitting scheme, the total number of slices is reduced from 4 to 3. In addition, the read portion of each slice is continuous in the width direction, which also improves the read efficiency of the slice from DDR to SRAM.

[0097] In this embodiment of the disclosure, when performing image segmentation search, both square segmentation and strip segmentation searches are performed simultaneously, and the total number of slices obtained by each scheme on the source image is calculated. When a certain scheme yields a small number of slices, the corresponding search scheme is selected for segmentation. When the number of slices is the same, the strip segmentation scheme can be selected first because the strip segmentation scheme has higher efficiency in reading DDR memory.

[0098] In this embodiment of the disclosure, in the process of determining the first preset split size by splitting the image into squares, at least one first dividing line in the image row direction and at least one second dividing line in the image column direction can be determined first. Then, the target image is divided based on the first dividing line and the second dividing line to obtain multiple split target sub-images.

[0099] like Figure 1 As shown, there is one first dividing line and one second dividing line. The resulting image is then divided into four target sub-images, corresponding to four source sub-images. This is just a specific example; in practical applications, there can be multiple first dividing lines and multiple second dividing lines, for example, two first dividing lines and three second dividing lines, resulting in six target sub-images.

[0100] In this embodiment of the disclosure, in the process of determining the second preset split size by the strip splitting method, at least one third dividing line in the image row direction or at least one fourth dividing line in the image column direction can be determined first, and then the target image is divided based on the third dividing line or the fourth dividing line to obtain multiple strip split target sub-images.

[0101] like Figure 3 As shown, there are two first dividing lines, and then the image is split to obtain three target sub-images. Correspondingly, the source image is split to obtain three source sub-images. This is just a specific example. In practical applications, there may be other first dividing lines, such as five first dividing lines, thus splitting the image to obtain six target sub-images.

[0102] In addition to splitting in the row direction, it can also be split into strips using the fourth dividing line in the column direction. The splitting process described above will not be repeated here.

[0103] The image processing method provided in this disclosure can determine the specific splitting method for splitting the target image by combining various factors. For example, if the storage method of the source image pixel values ​​is determined to be from left to right and from front to back, the pixel information in the row direction is arranged continuously. In this case, the splitting method in the row direction can be preferred. Or, if it is determined that both splitting into squares and splitting into strips can meet the splitting requirements, the splitting method with fewer sub-images can be preferred.

[0104] In combination Figure 1 And Figure 3 It can be known that there is a direct correspondence between the target subgraph and the source subgraph, and the correspondence is determined by the transformation between the two, that is, different transformation determines different correspondence. In the embodiment of the present disclosure, the source subgraph corresponding to the target subgraph can be determined according to the following steps:

[0105] Step one, obtain the conversion relationship between the first image coordinate system where the source graph is located and the second image coordinate system where the target graph is located;

[0106] Step two, determine the source subgraph corresponding to the target subgraph in the source graph based on the conversion relationship.

[0107] Here, the source graph and the target graph have a corresponding conversion relationship in the corresponding image coordinate system, so for any pixel point in the target subgraph, the corresponding pixel point in the source subgraph can be determined based on the above conversion relationship, and then the corresponding source subgraph is determined.

[0108] Specifically, for each to-be-determined target pixel point in the target subgraph, the source pixel point corresponding to the to-be-determined target pixel point is determined from the source subgraph based on the conversion relationship, so based on the source pixel points corresponding to each to-be-determined target pixel point, the pixel amount contained in the source subgraph corresponding to the target subgraph can be determined.

[0109] In actual application, due to the existence of the conversion relationship, the coordinate point of the target subgraph may not be mapped to the actual coordinate point in the source subgraph, based on which, based on whether the projected source pixel point falls within the coordinate range of the source subgraph, the pixel points can be pruned after determining the source pixel points corresponding to all to-be-determined target pixel points, so as to determine the final pixel amount.

[0110] Here, the source pixel value of the source pixel point corresponding to the to-be-determined target pixel point can be used to determine each target pixel value in the target subgraph, which can be obtained in combination with the conversion relationship, which will not be described here.

[0111] In the embodiment of the present disclosure, the process of determining the target pixel value of each target subgraph is equivalent to determining the transformation result of the source subgraph, and then the transformation results of each source subgraph are summarized, that is, the transformation result of the entire source graph is obtained, so as to better meet the cache demand of SRAM and other memories, and significantly improve the processing performance.

[0112] In the description of the specification, the description referring to the terms "some possible embodiments", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0113] Regarding the method flowchart of the embodiments of the present disclosure, some operations are described as different steps executed in a certain order. Such flowcharts are illustrative rather than limiting. Some steps described herein can be grouped together and executed in a single operation, some steps can be divided into multiple sub-steps, and some steps can be executed in an order different from that shown herein. Each step shown in the flowchart can be implemented in any way by any circuit structure and / or tangible mechanism (for example, by software running on a computer device, hardware (for example, processor or chip implemented logic function) and the like, and / or any combination thereof) in any manner.

[0114] The person skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible inherent logic.

[0115] Based on the same inventive concept, the present disclosure also provides a picture processing device corresponding to the picture processing method. Since the principle of solving problems of the device in the embodiments of the present disclosure is similar to the above-mentioned picture processing method of the embodiments of the present disclosure, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described.

[0116] Referring to Figure 4 As shown in the figure, a schematic diagram of a picture processing device provided by the embodiments of the present disclosure is shown, the device comprises: an acquisition module 401, a splitting module 402, a judgment module 403 and a determination module 404; wherein,

[0117] The acquisition module 401 is configured to acquire a source picture with source pixel values and acquire a target picture without target pixel values;

[0118] The splitting module 402 is configured to split the target picture based on a preset splitting size to obtain a plurality of target sub-pictures;

[0119] The determining module 403 is configured to determine a source subgraph corresponding to the target subgraph in the source graph based on the target subgraph, and determine whether the pixel quantity contained in the source subgraph is consistent with a preset threshold.

[0120] The determining module 404 is configured to determine each target pixel value in the target subgraph based on each source pixel value in the source subgraph.

[0121] With the above picture processing device, in the case that the source graph with source pixel values and the target graph without target pixel values are obtained, the target graph is split based on a preset split size, and then the source subgraph corresponding to the split target subgraph is determined, and it is determined whether the pixel quantity contained in the source subgraph is consistent with a preset threshold. In the case of consistency, each target pixel value in the target subgraph is determined based on each source pixel value in the source subgraph. The present disclosure splits based on the preset split size and then performs picture processing, which mainly considers that the size of SRAM is usually not particularly large. Thus, the pixel quantity contained in the source subgraph corresponding to the split target subgraph can be more consistent with the preset threshold set for SARM, so as to ensure the conversion relationship between the source graph and the target graph in the case of effective splitting, and the overall picture processing performance is better.

[0122] In a possible implementation, the above device further includes:

[0123] The adjusting module 405 is configured to adjust the preset split size and re-split the target graph.

[0124] In a possible implementation, the adjusting module 405 is further configured to:

[0125] If the pixel quantity contained in the source subgraph is greater than the preset threshold, the preset split size is reduced; and if the pixel quantity contained in the source subgraph is less than the preset threshold, the preset split size is increased.

[0126] In a possible implementation, the splitting module 402 is configured to split the target graph based on the preset split size according to the following steps to obtain a plurality of target subgraphs:

[0127] The first preset split size is determined by a splitting square method, and the second preset split size is determined by a splitting strip method.

[0128] The target graph is split based on the first preset split size and the second preset split size, respectively, to correspondingly obtain a split square target subgraph and a split strip target subgraph.

[0129] In a possible implementation, the width and height of the first preset split size are both less than the width and height of the target graph, and the width or height of the second preset split size corresponds to the width or height of the target graph.

[0130] In a possible implementation, the determining module 404 is further configured to:

[0131] When the pixel amounts of the disassembled side source subgraphs and the disassembled strip source subgraphs are consistent with the preset threshold, and the number of the disassembled side source subgraphs and the disassembled strip source subgraphs is consistent, the target pixel values in the disassembled strip target subgraph are determined by selecting the disassembled strip source subgraph.

[0132] In a possible implementation, the determining module 404 is further configured to:

[0133] When the pixel amounts of the disassembled side source subgraphs and the disassembled strip source subgraphs are consistent with the preset threshold, and the number of the disassembled side source subgraphs and the disassembled strip source subgraphs is consistent, the target pixel values in the disassembled strip target subgraph are determined by selecting the disassembled strip source subgraph.

[0134] In a possible implementation, the splitting module 402 is configured to determine the first preset splitting size by the disassembled side manner according to the following steps:

[0135] Determine at least one first split line in a picture row direction and at least one second split line in a picture column direction;

[0136] Disassemble the target graph according to the at least one first split line and the at least one second split line to obtain a plurality of disassembled side target subgraphs.

[0137] In a possible implementation, the splitting module 402 is configured to determine the second preset splitting size by the disassembled strip manner according to the following steps:

[0138] Determine at least one third split line in a picture row direction;

[0139] Disassemble the target graph according to the at least one third split line to obtain a plurality of disassembled strip target subgraphs; or

[0140] Determine at least one fourth split line in a picture column direction;

[0141] Disassemble the target graph according to the at least one fourth split line to obtain a plurality of disassembled strip target subgraphs.

[0142] In a possible implementation, the determining module 403 is configured to determine the source subgraph corresponding to the target subgraph in the source graph according to the following steps:

[0143] Obtain a conversion relationship between a first image coordinate system in which the source graph is located and a second image coordinate system in which the target graph is located;

[0144] Determine the source subgraph corresponding to the target subgraph in the source graph based on the conversion relationship.

[0145] In one possible implementation, the determination module 403 is used to determine the number of pixels contained in the source sub-image according to the following steps:

[0146] For each target pixel to be determined in the target sub-image, the source pixel corresponding to the target pixel to be determined is determined from the source sub-image based on the transformation relationship;

[0147] Based on the source pixels corresponding to each target pixel to be determined, determine the number of pixels contained in the source sub-image corresponding to the target sub-image.

[0148] In one possible implementation, the determining module 404 is configured to determine each target pixel value in the target sub-image based on each source pixel value in the source sub-image according to the following steps:

[0149] For each target pixel to be determined in the target sub-image, determine the source pixel value of the corresponding source pixel; and set the source pixel value as the target pixel value of the target pixel to be determined.

[0150] It should be noted that the apparatus in this embodiment can implement the various processes of the aforementioned method and achieve the same effects and functions, which will not be elaborated here.

[0151] This disclosure also provides an electronic device, such as... Figure 5 The diagram shown is a schematic representation of an electronic device structure provided in this embodiment of the present disclosure, including: a processor 501, a memory 502, and a bus 503. The memory 502 stores machine-readable instructions executable by the processor 501 (e.g., ...). Figure 4 The device includes an acquisition module 401, a splitting module 402, a judgment module 403, and a determination module 404 (for the corresponding execution instructions, etc.). When the electronic device is running, the processor 501 and the memory 502 communicate via the bus 503. When a machine-readable instruction is executed by the processor 501, the following processing is performed:

[0152] Obtain the source image with source pixel values ​​and the target image without target pixel values;

[0153] Based on the preset split size, the target image is split into multiple target sub-images;

[0154] Identify the source sub-image corresponding to the target sub-image and determine whether the number of pixels contained in the source sub-image is consistent with the preset threshold.

[0155] If so, then the target pixel values ​​in the target sub-image are determined based on the source pixel values ​​in the source sub-image.

[0156] The embodiment of the present disclosure further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, steps of the picture processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0157] The embodiment of the present disclosure further provides a computer program product, and the computer program product carries a program code. The program code includes instructions for executing steps of the picture processing method described in the above method embodiment. For details, refer to the above method embodiment, which will not be described here.

[0158] The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0159] The embodiments in the present disclosure are described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment mainly describes the difference from other embodiments. Especially, the device, equipment and computer readable storage medium embodiments are basically similar to the method embodiments, so the description is simplified, and the related parts can be referred to the part of the method embodiment.

[0160] The device, equipment and computer readable storage medium provided by the embodiments of the present disclosure correspond to the method, so the device, equipment and computer readable storage medium also have the similar beneficial technical effects as the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device, equipment and computer readable storage medium will not be described here.

[0161] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, device (equipment or system), or computer readable storage medium. Therefore, the present disclosure can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can be in the form of a computer readable storage medium implemented on one or more computer readable storage media containing computer usable program code, including but not limited to disk memory, CD-ROM, optical memory, etc.

[0162] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination of flowcharts and / or blocks in the flowcharts can be implemented by computer program instructions. Figure 1 an apparatus with a processor-based platform adapted to perform the functions described in a flowchart block or blocks.

[0163] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination of flowcharts and / or blocks in the flowcharts can be implemented by computer program instructions. Figure 1 an apparatus with a processor-based platform adapted to perform the functions described in a flowchart block or blocks.

[0164] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination of flowcharts and / or blocks in the flowcharts can be implemented by computer program instructions. Figure 1 an apparatus with a processor-based platform adapted to perform the functions described in a flowchart block or blocks.

[0165] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0166] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM) for storing structural information and / or instruction sets. Both can be within one or more memory devices 1625. It is considered that the memory can also include other alternatives, such as flash memory, disk drives, disk arrays, tape storage, etc.

[0167] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology for storing information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. In addition, although the operations of the methods of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in that particular order, or that all of the shown operations must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps.

[0168] Although the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not mean that the features in these aspects cannot be combined for benefit, but is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims.

Claims

1. A picture processing method, characterized by, The method comprises: obtaining a source image with source pixel values and a target image without target pixel values; based on a preset split size, splitting the target image by a split-square method and a split-strip method respectively to obtain a plurality of split-square target sub-images and a plurality of split-strip target sub-images; the split-square method refers to splitting in both the picture row direction and the picture column direction, and the split-strip method refers to splitting in the picture row direction or the picture column direction; determining a plurality of split-square source sub-images corresponding to the plurality of split-square target sub-images, and a plurality of split-strip source sub-images corresponding to the plurality of split-strip target sub-images, and judging whether the pixel amount contained in the split-square source sub-images and the split-strip source sub-images is consistent with a preset threshold value; the preset threshold value is determined based on the storage size of a static random access memory (SRAM); if consistent, selecting the source sub-image with the least quantity from the split-square source sub-images and the split-strip source sub-images, and determining the target pixel values in the target sub-images based on the source pixel values in the source sub-image; if the quantities are the same, selecting the split-strip source sub-images to determine the target pixel values in the split-strip target sub-images.

2. The method of claim 1, wherein, The method further comprises: if not, adjusting the preset split size and re-splitting the target image.

3. The method of claim 2, wherein, The method further comprises: if the pixel amount contained in the source sub-images is greater than the preset threshold value, reducing the preset split size; if the pixel amount contained in the source sub-images is less than the preset threshold value, increasing the preset split size.

4. The method of claim 1, wherein, The splitting of the target image by the split-square method and the split-strip method respectively comprises: determining a first preset split size by the split-square method and a second preset split size by the split-strip method; based on the first preset split size and the second preset split size, splitting the target image to obtain the split-square target sub-images and the split-strip target sub-images respectively.

5. The method of claim 4, wherein, The width and height of the first preset split size are both smaller than the width and height of the target image, and the width or height of the second preset split size corresponds to the width or height of the target image.

6. The method of claim 4, wherein, The method further comprises: determining the split-square source sub-images corresponding to the split-square target sub-images and the split-strip source sub-images corresponding to the split-strip target sub-images, and selecting the source sub-image with the least quantity from the split-square source sub-images and the split-strip source sub-images to determine the target pixel values in the corresponding target sub-images when the pixel amount contained in the split-square source sub-images and the split-strip source sub-images is consistent with the preset threshold value.

7. The method of claim 6, wherein, The method further comprises: when the pixel amount contained in the split-square source sub-images and the split-strip source sub-images is consistent with the preset threshold value and the quantities of the split-square source sub-images and the split-strip source sub-images are the same, selecting the split-strip source sub-images to determine the target pixel values in the split-strip target sub-images.

8. The method of claim 4, wherein, The determination of the first preset split size by the split-square method comprises: determining at least one first split line in the picture row direction and at least one second split line in the picture column direction; splitting the target image according to the at least one first split line and the at least one second split line to obtain a plurality of split-square target sub-images.

9. The method of claim 4, wherein, The determination of the second preset split size by the split-strip method comprises: determining at least one third split line in the picture row direction; According to at least one of the third split lines, the target image is split to obtain a plurality of split target sub-images; or At least one fourth split line in the picture column direction is determined; According to at least one of the fourth split lines, the target image is split to obtain a plurality of split target sub-images.

10. The method of claim 1, wherein, The determination of the plurality of split source sub-images corresponding to the plurality of split target sub-images comprises: A conversion relationship between a first image coordinate system in which the source image is located and a second image coordinate system in which the target image is located is obtained; Based on the conversion relationship, the source sub-image corresponding to the target sub-image in the source image is determined.

11. The method of claim 10, wherein, The determination of the pixel amount contained in the source sub-image comprises: For each to-be-determined target pixel point in the target sub-image, a source pixel point corresponding to the to-be-determined target pixel point is determined from the source sub-image based on the conversion relationship; Based on the source pixel points respectively corresponding to each to-be-determined target pixel point, the pixel amount contained in the source sub-image corresponding to the target sub-image is determined.

12. The method according to claim 10 or 11, characterized in that, The determination of each target pixel value in the target sub-image comprises: For each to-be-determined target pixel point in the target sub-image, a source pixel value of the source pixel point corresponding to the to-be-determined target pixel point is determined; and the source pixel value is determined as the target pixel value of the to-be-determined target pixel point.

13. An image processing apparatus, characterized by comprising: Comprise: An acquisition module is used to acquire a source image with source pixel values and acquire a target image without target pixel values; A splitting module is used to split the target image by a split-by-square manner and a split-by-strip manner based on a preset split size, to obtain a plurality of split-by-square target sub-images and a plurality of split-by-strip target sub-images; the split-by-square manner refers to splitting in both the picture row direction and the picture column direction, and the split-by-strip manner refers to splitting in the picture row direction or the picture column direction; A judgment module is used to determine a plurality of split-by-square source sub-images corresponding to the plurality of split-by-square target sub-images, and a plurality of split-by-strip source sub-images corresponding to the plurality of split-by-strip target sub-images, and judge whether the pixel amounts contained in the split-by-square source sub-images and the split-by-strip source sub-images are consistent with a preset threshold value; The preset threshold value is determined based on a storage size of a static random access memory (SRAM); A determination module is used to, if consistent, select a source sub-image with the least quantity from the split-by-square source sub-images and the split-by-strip source sub-images, and determine each target pixel value in the target sub-image based on each source pixel value in the source sub-image; If the quantities are the same, the split-by-strip source sub-images are selected to determine each target pixel value in the split-by-strip target sub-images.

14. An electronic device, comprising: Comprise: A processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to execute the picture processing method in any one of claims 1 to 12.

15. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, the computer program is executed by the processor to execute the picture processing method in any one of claims 1 to 12.

Images