Image processing method, computer device and storage medium

Abstract

The present application discloses an image processing method, a computer device and a storage medium. The method comprises: obtaining an original image; dividing the region based on the original image to obtain a plurality of sub-regions; sampling the plurality of sub-regions to obtain sampled images of a plurality of channels; wherein the sampled image of each channel contains sampled pixels obtained by sampling the vector direction corresponding to the channel in the plurality of sub-regions; combining the sampled images of the plurality of channels to obtain a target image; wherein the resolution of the target image is smaller than that of the original image. The above scheme can reduce the amount of computation for image processing without reducing the accuracy of image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, a computer device and a storage medium. Background Art

[0002] With the development of computer technology, more and more image-based application tasks are being performed.

[0003] In some application scenarios, network models can be used to process images to achieve image application tasks. However, the scale of the input image has a great impact on the model's computational complexity. For example, for some large-resolution images, a large amount of computation is required. In order to reduce the model's computational complexity, the image can usually be cropped into a small-scale image, but this method will reduce the model's processing accuracy for the image. Therefore, there are currently problems such as large image computational complexity or low accuracy. Summary of the invention

[0004] The main technical problem solved by the present application is to provide an image processing method, a computer device and a storage medium, which can reduce the amount of calculation for image processing without reducing the accuracy of image processing.

[0005] The first aspect of the present application provides an image processing method, which includes: acquiring an original image; performing region division based on the original image to obtain a plurality of sub-regions; sampling the plurality of sub-regions to obtain sampled images of multiple channels; wherein the sampled image of each channel includes sampled pixels obtained by sampling the vector directions corresponding to the channel in the plurality of sub-regions; combining the sampled images of the plurality of channels to obtain a target image; wherein the resolution of the target image is smaller than that of the original image.

[0006] The second aspect of the present application provides an image processing method, which includes: acquiring a target image; the target image is obtained by using the above-mentioned image processing method; and processing the target image by using an image processing model to obtain an image processing result.

[0007] A third aspect of the present application provides a computer device, which includes a memory and a processor coupled to each other, wherein program data is stored in the memory, and the processor is used to execute the program data to implement any step of the above-mentioned image processing method.

[0008] A fourth aspect of the present application provides a computer-readable storage medium, which stores program data that can be executed by a processor, and the program data is used to implement any step of the above-mentioned image processing method.

[0009] The above scheme obtains an original image, divides the region based on the original image, obtains several sub-regions, samples the several sub-regions, and obtains sampled images of multiple channels. Since the sampled image of each channel contains sampled pixels obtained by sampling the vector directions corresponding to the channel in the several sub-regions, each sampled image can contain feature information of the corresponding vector direction. Then, the sampled images of multiple channels are combined to obtain a target image. Compared with the original image, a target image with a smaller resolution of multiple channels is obtained. Since the resolution of the target image is smaller than that of the original image, the amount of calculation for image processing can be reduced. Moreover, the target image is a multi-channel image, and its data volume is the same as that of the original image, so that the accuracy of image processing can be not reduced.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in this application, the following briefly introduces the drawings required in the description of the embodiments. Obviously, the drawings described below are only some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0012] Figure 1 It is a flowchart of the first embodiment of the image processing method of the present application;

[0013] Figure 2 This application Figure 1 A schematic diagram of a flow chart of an embodiment of step S12;

[0014] Figure 3 is a structural schematic diagram of an embodiment of an original image after alignment of the present application;

[0015] Figure 4 This application Figure 1 A schematic diagram of a flow chart of an embodiment of step S13;

[0016] Figure 5 This is an example schematic diagram of an embodiment of the vector direction of the present application;

[0017] Figure 6 It is an example schematic diagram of a sampling image of a channel of the present application;

[0018] Figure 7 It is an example schematic diagram of an embodiment of a multi-channel sampling image of the present application;

[0019] Figure 8 is a flowchart of the second embodiment of the image processing method of the present application;

[0020] Fig. 9 This is an example schematic diagram of an embodiment of the defect detection result of the present application;

[0021] Fig.10 is a structural schematic diagram of the first embodiment of the image processing device of the present application;

[0022] Fig.11 is a structural schematic diagram of a second embodiment of the image processing device of the present application;

[0023] Fig.12 It is a structural schematic diagram of an embodiment of a computer device of the present application;

[0024] Fig.13 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The terms "first" and "second" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0027] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0028] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] The present application provides the following embodiments, and each embodiment is described in detail below.

[0030] It is understandable that the image processing method in the present application can be executed by a computer device, and the computer device can be any device with processing capabilities, such as a mobile device, a computer, a server, a network device, etc., and the present application does not limit this. In some possible implementations, the image processing method can also be implemented by a processor calling program data stored in a memory.

[0031] See also Figure 1 , Figure 1 1 is a flow chart of the first embodiment of the image processing method of the present application. The method may include the following steps:

[0032] S11: Acquire the original image.

[0033] Obtain an original image to be processed. The original image may be any image captured by a camera device, for example, any image captured of a target, scene, etc. This application does not impose any limitation on this original image.

[0034] Optionally, the original image may be an image that needs to be input for processing by the image processing model. The image processing model may be a deep learning model, a neural network model, etc. Exemplarily, the image processing model includes but is not limited to: an image detection model, an image classification model, an image encoding and decoding model, etc. This application does not limit the image processing model.

[0035] Optionally, the original image may be an image of larger size or resolution, such as an image that requires a larger amount of computation relative to the image processing model. This application does not impose any restrictions on the size of the original image.

[0036] S12: Perform region division based on the original image to obtain a plurality of sub-regions.

[0037] The original image may be partitioned to divide the original image into regions to obtain a number of sub-regions, and each sub-region may be used as a sampling unit.

[0038] Optionally, the original image may be divided into a preset number of regions according to a preset number of sub-regions to be divided, thereby obtaining a plurality of sub-regions.

[0039] Optionally, the original image may be divided into a plurality of regions of preset scales according to preset scales of the sub-regions to be divided, to obtain a plurality of sub-regions.

[0040] In some embodiments, see Figure 2 , the step S12 of the above embodiment can be further expanded. Based on the original image, the region is divided to obtain a plurality of sub-regions. This embodiment may include the following steps:

[0041] S121: Determine the vector scale for region division.

[0042] The vector scale can represent the scale information of the divided sub-region. The vector scale can represent the number of horizontal vectors and vertical vectors in the sub-region, etc. The vector scale can represent the scale information of pixels in the sub-region, etc.

[0043] Optionally, the vector scale of the region division may be determined according to a preset number.

[0044] Optionally, the vector scale of the region division may be determined according to a preset scale.

[0045] In some embodiments, the vector scale of the region division is determined by using the ratio of the original scale of the original image to the preset scale. The preset scale includes the scale of the image input to the image processing model. The scale includes the width and height of the image. The preset scale can be set according to the application scenario of the image processing model, the scale of the input image, etc. Optionally, the preset scale can be a pre-set scale, and the present application does not limit the preset scale.

[0046] In some embodiments, the ratio of the original scale (such as width w, height h) of the original image to the preset scale (such as width sw, height sh) can be obtained, and the ratio is rounded up to obtain the vector scale of the region division. The vector scale includes a first vector scale vx and a second vector scale vy, which can also be expressed as the width and height of the sub-region, respectively, or can represent information such as the number of horizontal vectors and vertical vectors in the sub-region. For example, the first vector scale vx can represent the scale of the horizontal vector, and the second vector scale vy can represent the scale of the vertical vector.

[0047] Among them, the vector scale (vx, vy) can be calculated using the following formula:

[0048]

[0049] Optionally, after obtaining the vector scale, the original image may be directly divided into regions. Optionally, after obtaining the vector scale, the alignment of the original image is determined, and then, the region division is performed according to the alignment of the original image.

[0050] S122: Determine whether the original image meets the scale alignment condition.

[0051] The scale alignment condition includes that the original scale of the original image and the vector scale satisfy a preset multiple relationship. The preset multiple relationship may be an integer multiple relationship. For example, if the original scale is an integer multiple of the vector scale, or the original scale can divide the vector scale, it can be indicated that the preset multiple relationship is satisfied.

[0052] Since the vector scale includes the first vector scale vx and the second vector scale vy, the scale alignment condition may include that the width w of the original image and the first vector scale vx satisfy a preset multiple relationship, and the width h of the original image and the first vector scale vy satisfy a preset multiple relationship.

[0053] In some implementations, in response to the original image not satisfying the scale alignment condition, the following step S123 is executed to align the original image to obtain an aligned original image, and then the original image is divided into regions.

[0054] In some implementations, in response to the original image satisfying the scale alignment condition, the following step S124 is performed to divide the original image into regions.

[0055] S123: Acquire an alignment scale of the original image, and use the alignment scale to perform a preset interpolation process on the original image to obtain an aligned original image.

[0056] The original image may be aligned so that the aligned original image meets the scale alignment condition. First, the alignment scale of the original image may be obtained, and the alignment scale represents the scale after the original scale is adjusted, that is, the scale (such as width and height) of the aligned original image may be represented.

[0057] In some implementations, the alignment scale is obtained using the vector scale and the original scale. The expansion scale can be determined using the ratio of the original scale to the vector scale, and then the alignment scale is obtained based on the expansion scale and the vector scale.

[0058] Exemplarily, the ratio of the original scale to the vector scale can be obtained, the ratio is rounded down to obtain an integer, and then the product of the integer and the vector scale is obtained to obtain the product number (that is, the extended scale), and then the sum of the extended scale and the vector scale is obtained to obtain the alignment scale.

[0059] The alignment scale can include width and height, and the width and height of the original image alignment can be obtained respectively. The alignment scale can be expressed by the following formula:

[0060]

[0061] Among them, nw and nh represent the width and height of the alignment scale respectively, vx and vy represent the first vector scale and the second vector scale of the vector scale respectively, and w and h represent the width and height of the original image respectively.

[0062] Then, the original image is interpolated by the preset method using the alignment scale (nw, nh) to obtain the aligned original image. Optionally, the original image can be resized (such as resize) using a bilinear interpolation algorithm based on the alignment scale to obtain an image larger than the original image, so as to achieve alignment of the scale of the original image and the vector scale of the sub-region (such as the horizontal vector and the vertical vector) to integer multiples.

[0063] See also Figure 3 , Figure 3 It can represent the original image after alignment, where the multiple regions corresponding to the vector scale (vx, vy) represent several sub-regions respectively, and the region corresponding to the original scale (w, h) represents the original image. The region corresponding to the alignment scale (nw, nh) represents the original image after alignment. For example, the original scale of the original image is (13, 9), the vector scale of the sub-region is (3, 2), and the alignment scale of the aligned original image S is (15, 10).

[0064] S124: Based on the vector scale, the original image is divided into regions to obtain a plurality of sub-regions.

[0065] Based on the vector scale, the original image or the aligned original image can be divided into regions, and multiple regions can be divided according to the vector scale (vx, vy) of the sub-region to obtain several sub-regions, so that each sub-region can be used as a sampling unit.

[0066] Continue reading Figure 1 , after the above step S12, the following steps are also included:

[0067] S13: Sampling the plurality of sub-regions to obtain sampling images of a plurality of channels; wherein the sampling image of each channel includes sampling pixels obtained by sampling the vector directions corresponding to the channel in the plurality of sub-regions.

[0068] Each sub-region is taken as a sampling unit. Each sub-region is sampled for each vector direction to obtain the sampled pixels of each sub-region. The same vector direction corresponds to a channel, that is, the sampled pixels of the channel in each sub-region can be obtained. In this way, by sampling multiple vector directions, sampled images of multiple channels can be obtained.

[0069] In some embodiments, see Figure 4 , step S13 of the above embodiment can be further expanded. Sampling a plurality of sub-regions to obtain sampled images of multiple channels, this embodiment may include the following steps:

[0070] S131: Determine multiple vector directions based on the multiple sub-regions.

[0071] Based on the plurality of sub-regions, for example, by using the vector scales corresponding to the plurality of sub-regions, a set of vector directions is determined to obtain a plurality of vector directions. For each sub-region, there is a set of vector directions corresponding to the sub-region, and the number of vector directions may be vx*vy, which is the product of the width and the height in the vector scale.

[0072] In some embodiments, each sub-region corresponds to a vector scale, and the vector direction can be determined using the vector scale and the vector sequence number, wherein the vector sequence number represents the sequence number of the vector direction. In this way, each vector direction can be determined. Optionally, the vector sequence number can be sorted by a preset order of the sub-regions, such as from left to right, from top to bottom, etc., and the present application does not limit the vector sequence number.

[0073] In some embodiments, the vector scale corresponding to each sub-region, each vector direction includes a first component and a second component, each vector direction can be used as the current vector, the current vector corresponds to the current sequence number, and the second component of the current vector is obtained using the current sequence number and the vector scale; the first component of the current vector is obtained using the current sequence number, the second component and the vector scale.

[0074] Optionally, the vector scale corresponding to each sub-region includes a first vector scale vx and a second vector scale vy, and one of vx and vy may be used as the vector scale to obtain the first component or the second component of the current vector.

[0075] In some embodiments, the vector scale corresponding to each sub-region includes a first vector scale vx and a second vector scale vy. Correspondingly, each vector direction V i It can contain the first component V i x and the second component V i y Therefore, each vector direction can be used as the current vector V i , the current vector V iCorresponding to the current sequence number i. For each vector direction V i , using the current sequence number i and the first vector scale vx, get the second component V of the current vector i y Then, using the current sequence number i and the second component V i y And the first vector scale vx, get the first component V of the current vector i x .

[0076] Exemplarily, the ratio of the current sequence number i and the first vector scale vx may be rounded down to obtain the second component Then, get the first vector scale vx and the second component The product of the current sequence number i and the difference between the product is taken as the first component Thus, we get the vector direction

[0077] Among them, for the i-th vector direction V i , where i∈[0,vx*vy), and satisfies the following:

[0078]

[0079]

[0080] In the above manner, a set of vector directions of the sub-region can be obtained, and for each sub-region, vx*vy vector directions can be included. In some application scenarios, it can be understood that each sub-region can also include vx*vy pixel points.

[0081] See also Figure 5 For example, the alignment scale of the aligned original image S is (15, 10), the vector scale of each sub-region is (3, 2), and for each sub-region, 6 vector directions can be obtained, namely (V0, V1, V2, V3, V4, V5). Therefore, the pixel points of each sampling unit can be sampled using multiple vector directions, and the sampled pixels of different vector directions can be stored in different sampling images.

[0082] S132: For each vector direction, determine the sampling position of each sub-region respectively, and obtain the sampling pixels corresponding to the vector direction.

[0083] For each vector direction, sampling can be performed in each sub-region to determine the sampling position of each sub-region, thereby obtaining the sampling pixel corresponding to the vector direction. In this way, sampling can be performed for each vector direction, and in each sub-region, the pixel points corresponding to the same vector direction can be used as sampling pixels.

[0084] S133: Based on the sampling pixels corresponding to each vector direction, a sampling image of multiple channels is obtained.

[0085] By sampling each vector direction of each sub-region to obtain the sampling pixels of each vector direction in each sub-region, the sampling pixels corresponding to the same vector direction can be stored in the corresponding position of the sub-region in the same sampling image. For example, the sampling pixels are stored and arranged in the order of each sub-region to obtain the sampling image of the channel corresponding to the vector direction. In this way, sampling images of multiple channels can be obtained.

[0086] In some embodiments, each coordinate position (x, y) in the sampled image stores a sampled pixel of a corresponding sub-region, and the sub-region corresponding to the coordinate position (x, y) is determined by the ratio of the coordinate position (x, y) to the vector scale (vx, vy).

[0087] In some implementations, each sub-region is sampled separately, and the sampled image of each channel can be determined by the following formula:

[0088]

[0089] in, Represents the vector direction V i The corresponding pixel value of the sampled image R at the coordinate position (x, y), S is the aligned original image. Indicates the original image S after alignment at coordinate position The pixel value at , that is, the sampled pixel.

[0090] in, Indicates the number of sub-regions of the aligned original image S from left to right in the x direction where sampling is performed. Indicates the number of sub-regions of the aligned original image S from top to bottom in the y direction to be sampled, through That is, the position of the sub-region corresponding to each sampling pixel can be determined. Multiply by the corresponding vector scale vx, vy to obtain the starting pixel coordinates of the sampled pixel in the current sub-area.

[0091] In the above manner, the sampled image R corresponding to each channel can be obtained. For each sampled image R, it represents the result of sampling in a specific vector direction, so different sampled images R store the pixel values ​​of the sub-region in different vector directions. When all the sampled images R are arranged in order, they can retain the structural feature information of the aligned original image S.

[0092] See also Figure 6For example, the alignment scale of the aligned original image S is (15,10), and the vector scale of each sub-region is (3,2). For each sub-region, it contains pixel points corresponding to each vector direction, such as vector direction V5. In each sub-region, vector direction V5 is sampled. A pixel point of vector direction V5 can be selected in each sub-region as a sampling pixel. Thus, the sampled pixels of each sub-region are stored in the sampling image R5, and the sampling image R5(5,5) of the channel corresponding to vector direction V5 is obtained. In the above manner, each vector direction V5 can be obtained. i The sampled image Ri of the corresponding channel, where i is 0-5.

[0093] Continue reading Figure 1 , after the above step S13, the following steps are also included:

[0094] S14: integrating the sampled images of multiple channels to obtain a target image; wherein the resolution of the target image is smaller than that of the original image.

[0095] The target image can be obtained by combining the sampled images of multiple channels, such as performing channel merging and fusion processing on the sampled images of multiple channels. Since the sampled images are obtained by partitioning the original images, the resolution of the sampled images is smaller than that of the original images. After the channels of the sampled images are merged, the resolution of the target image is smaller than that of the original images.

[0096] See also Figure 7 For example, in a sub-region of the original image S, sampling is performed in six vector directions (V0, V1, V2, V3, V4, V5) respectively, and the sampled images (R0, R1, R2, R3, R4, R5) corresponding to each channel can be obtained. Then, the sampled images (R0, R1, R2, R3, R4, R5) are merged to obtain the target image, that is, a multi-channel output image. This image can be used as the input of the image processing model. Each channel of the image records the spatial features of a certain direction of the original image.

[0097] The above scheme obtains an original image, divides the region based on the original image, obtains several sub-regions, samples the several sub-regions, and obtains sampled images of multiple channels. Since the sampled image of each channel contains sampled pixels obtained by sampling the vector directions corresponding to the channel in the several sub-regions, each sampled image can contain feature information of the corresponding vector direction. Then, the sampled images of multiple channels are combined to obtain a target image. Compared with the original image, a target image with a smaller resolution of multiple channels is obtained. Since the resolution of the target image is smaller than that of the original image, the amount of calculation for image processing can be reduced. Moreover, the target image is a multi-channel image, and its data volume is the same as that of the original image, so that the accuracy of image processing can be not reduced.

[0098] In some embodiments, the target image can be used as an input image of the image processing model, so that the target image is processed by the image processing model to obtain an image processing result. In this way, the problem that using a large-resolution image as the input of an image processing model (such as a deep learning model) will result in excessive model calculations, and the problem that downsampling or cutting the input image into a small-size image will result in reduced model performance can be solved.

[0099] In some embodiments, the low-resolution target image can be restored to a high-resolution original image through an inverse algorithm. That is, a multi-channel low-resolution image can be adjusted to a single-channel high-resolution image through an inverse algorithm. The inverse algorithm may include the inverse processing of the above-mentioned image processing method, such as performing channel splitting on the target image to obtain split images of each channel, and then mapping the pixels of each split image to the original image according to the vector direction in the split images of each channel to obtain the original image. The present application does not limit the inverse algorithm.

[0100] In some implementations, the above-mentioned image processing method may be applied to an image encoder to implement a lossless image encoding and decoding.

[0101] In some embodiments, the above-mentioned image processing method can be applied to various image processing fields, such as adjusting the resolution of the original image before inputting it into the image processing model so that the adjusted target image is more suitable for the performance of the image processing model.

[0102] See also Figure 8 , Figure 8 1 is a flow chart of the second embodiment of the image processing method of the present application. The method may include the following steps:

[0103] S21: Acquire a target image.

[0104] The target image may be obtained by processing the original image. The target image is obtained by using the image processing method of any of the above embodiments. For details, reference may be made to the implementation process of the above embodiments, which will not be described in detail in this application.

[0105] S22: Process the target image using the image processing model to obtain an image processing result.

[0106] The image processing model may be a deep learning model, a neural network model, etc. Exemplarily, the image processing model includes but is not limited to: an image detection model, an image classification model, an image encoding and decoding model, etc. This application does not limit the image processing model.

[0107] The target image is used as the input image of the image processing model, and the image processing model can be used to process the target image to obtain an image processing result. For example, a defect detection model, a target recognition model, etc. can process its target image to obtain a defect detection result and a target recognition result. It can be understood that the image processing model of the present application can be any image model, and the present application does not limit this.

[0108] In the above scheme, the input image of the image processing model is a multi-channel image, and its data volume is the same as that of the original image. However, due to its low resolution, the length and width of each feature layer of the convolutional neural network are relatively low when performing inference. The computational amount and storage amount of the entire inference process are also relatively low, which can reduce the computational amount of the image processing model without reducing the accuracy of image processing.

[0109] Exemplarily, taking the defect detection model as an example, the scale of its original image is 1024×1024×1. Through the above-mentioned image processing method, the original image is processed to obtain the target image, that is, a multi-channel small-resolution image with a scale of 512×512×4 is reconstructed. If a defect detection model with the same network structure (such as including 16 convolutional layers) is used, and only the input image of the model is changed, the model calculation amount with the original image as the input image is about 8.983bflops, and the 512×512×4 multi-channel small-resolution target image obtained by the above-mentioned method of the present application is used as the input image, then the model calculation amount is about 2.473bflops. There is a lower model calculation amount for large-resolution image input. For scenarios where the large-resolution image needs to be downsampled or cropped due to limited calculation amount, the target image can be directly used as the model input by processing the large-resolution image, which ensures a lower calculation amount while avoiding the degradation of model detection performance due to downsampling or cropping of the large-resolution image.

[0110] Illustratively, the present application provides a specific embodiment for illustration below.

[0111] Taking the defect detection model as an example, the test environment is the same (such as the execution equipment, equipment processing performance, test training parameters, number of iterations, etc. are the same). For example, the test training parameter batchsize is 32, the number of training iterations is 50,000, and the learning rate can adopt the momentum descent strategy and the cosine annealing strategy.

[0112] (1) Model testing with the same input scale:

[0113] In this test, both defect detection models use 512*512 resolution images as input images, where the input of the first model is a 512*512*3 image obtained by bilinear interpolation downsampling of the original image of 1024*1024*1 (its three channels have the same data). In addition, the second model uses the above-mentioned image processing method of this application to process the original image to obtain a 512*512*4 image as input. The computational complexity of the two models is comparable, at 2.397bflops and 2.473bflops, respectively.

[0114] The test results of the two models are as follows:

[0115]

[0116] Among them, the average IOU (Mean Intersection over Union) is a common indicator for measuring the similarity between the predicted results and the true labels in semantic segmentation tasks. The target image of this application is used as the model input, which can reduce the image calculation time and improve the recall rate, average IOU, false alarm rate and other aspects.

[0117] See also Fig. 9 As for the defect detection results, it can be seen from the defect detection results of the first model that its defect detection ability for small pixel features is poor, while this application uses the target image as input, and the second model has a higher defect detection accuracy.

[0118] (2) Model test with the same detection performance:

[0119] In this test, for the same original image, the input scale of the first model is 608*608*3, and the input scale of the second model is 448*448*4 after being processed by the above method of the present application. The two models process the input images respectively to obtain their own test results.

[0120] Among them, the calculation amount of the second model is 1.892bflops, while the calculation amount of the first model is 3.387bflops. The target image obtained in this application is used as the model input, which can greatly reduce the calculation amount of the image.

[0121] The test results of the two models are as follows:

[0122]

[0123] The target image of this application is used as the model input, which can greatly reduce the amount of calculation, reduce the image calculation time and video memory usage, and maintain the recall rate, average IOU, false alarm rate, etc. The target image obtained by this application is used as the model input, and under the same model input size, the model's detection accuracy for the defect can be greatly improved and the false alarm rate can be reduced. If a higher detection accuracy is met, the first model requires an input size of 608*608, while the second model requires an input size of 448*448, which reduces the amount of calculation and video memory usage by about half.

[0124] It is understandable that in the above method of the specific implementation mode, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0125] In some embodiments, the present application also provides an image processing device for implementing the image processing method of any of the above embodiments.

[0126] See also Fig.10 , Fig.10 3 is a schematic diagram of the structure of the first embodiment of the image processing device of the present application. The image processing device 30 includes an acquisition module 31, a partitioning module 32, a sampling module 33 and a synthesis module 34. The modules are interconnected.

[0127] The acquisition module 31 is used to acquire the original image.

[0128] The partitioning module 32 is used to perform region division based on the original image to obtain a plurality of sub-regions.

[0129] The sampling module 33 is used to sample several sub-regions to obtain sampled images of multiple channels; wherein the sampled image of each channel includes sampled pixels obtained by sampling the vector directions corresponding to the channel in the several sub-regions.

[0130] The integration module 34 is used to integrate the sampled images of multiple channels to obtain a target image; wherein the resolution of the target image is smaller than that of the original image.

[0131] It should be noted that the image processing device of this embodiment and the image processing method of the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the image processing device of this embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this application does not limit this.

[0132] See also Fig.11 , Fig.11 4 is a schematic diagram of the structure of the second embodiment of the image processing device of the present application. The image processing device 40 includes an image module 41 and a processing module 42. The modules are interconnected. The image module 41 is used to obtain a target image. The target image is obtained using the above-mentioned image processing method. The processing module 42 is used to process the target image using an image processing model to obtain an image processing result.

[0133] It should be noted that the image processing device of this embodiment and the image processing method of the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the image processing device of this embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this application does not limit this.

[0134] For the above embodiment, the present application provides a computer device, see Fig.12 , Fig.12 1 is a schematic diagram of the structure of an embodiment of a computer device of the present application. The computer device 50 includes a memory 51 and a processor 52, wherein the memory 51 and the processor 52 are coupled to each other, the memory 51 stores program data, and the processor 52 is used to execute the program data to implement the steps of any embodiment of the above-mentioned image processing method.

[0135] In this embodiment, the processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having the ability to process signals. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or the processor 52 may also be any conventional processor, etc.

[0136] The method of the above embodiment can be implemented in the form of a computer program, so the present application proposes a computer readable storage medium, see Fig.13 , Fig.13The computer-readable storage medium 60 stores program data 61 that can be run by a processor, and the program data 61 can be executed by the processor to implement the steps of any embodiment of the above-mentioned image processing method.

[0137] The computer-readable storage medium 60 of this embodiment can be a medium that can store program data 61, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the program data 61. The server can send the stored program data 61 to other devices for execution, or it can also execute the stored program data 61 by itself.

[0138] In some embodiments, the functions or modules included in the device provided in the above embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, the present application will not go into details here.

[0139] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, this application will not go into details here.

[0140] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0142] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium, which is a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of each implementation method of the present application.

[0144] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a computer-readable storage medium and executed by a computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present application is not limited to any specific combination of hardware and software.

[0145] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: include: Get the original image; Performing region division based on the original image to obtain a plurality of sub-regions; Sampling the plurality of sub-regions to obtain sampled images of a plurality of channels; wherein the sampled image of each channel includes sampled pixels obtained by sampling the vector directions corresponding to the channels in the plurality of sub-regions; The sampled images of the multiple channels are integrated to obtain a target image; wherein the resolution of the target image is smaller than that of the original image.

2. The method according to claim 1, characterized in that The step of sampling the plurality of sub-regions to obtain sampled images of multiple channels includes: Based on the plurality of sub-areas, determining a plurality of vector directions; For each vector direction, the sampling positions of each sub-region are determined respectively to obtain the sampling pixels corresponding to the vector direction; Based on the sampling pixels corresponding to each vector direction, sampling images of multiple channels are obtained.

3. The method according to claim 2, characterized in that Each sub-region corresponds to a vector scale; The vector direction is determined using the vector scale and the vector sequence number; and / or, The sampling pixels of the corresponding sub-region are stored at each coordinate position in the sampled image, and the sub-region corresponding to the coordinate position is determined by using the ratio of the coordinate position to the vector scale.

4. The method according to claim 2, characterized in that: The vector scale corresponding to each sub-region; each vector direction includes a first component and a second component; and the determining of multiple vector directions based on the plurality of sub-regions includes: Each vector direction is used as a current vector, and the current vector corresponds to a current sequence number; Using the current sequence number and the vector scale, obtaining a second component of the current vector; The first component of the current vector is obtained by using the current sequence number, the second component and the vector scale.

5. The method according to claim 1, characterized in that The region division is performed based on the original image to obtain a plurality of sub-regions, including: Determine the vector scale for region division; Based on the vector scale, the original image is divided into regions to obtain the plurality of sub-regions.

6. The method according to claim 5, characterized in that The determining of the vector scale of the region division includes: The vector scale of the region division is determined by using the ratio of the original scale of the original image to a preset scale, wherein the preset scale includes the scale of the image input to the image processing model.

7. The method according to claim 5, characterized in that Before dividing the original image into regions based on the vector scale to obtain the plurality of sub-regions, the method includes: In response to the original image not satisfying the scale alignment condition, acquiring an alignment scale of the original image, and performing a preset interpolation process on the original image using the alignment scale to obtain an aligned original image; The scale alignment condition includes: the original scale of the original image and the vector scale satisfy a preset multiple relationship; and / or the alignment scale is obtained by using the vector scale and the original scale.

8. An image processing method, characterized in that: include: Get the target image; Processing the target image using an image processing model to obtain an image processing result; Wherein, the target image is obtained using the method described in any one of claims 1 to 7.

9. A computer device, characterized in that: The invention comprises a memory and a processor coupled to each other, wherein the memory stores program data, and the processor is used to execute the program data to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: Program data that can be executed by a processor is stored, and the program data is used to implement the method according to any one of claims 1 to 8.

Images