Image processing method and device, storage medium and electronic device

Abstract

The present invention provides an image processing method and device, a storage medium, and an electronic device. The method comprises: obtaining an original image to be processed in response to a fuzzy image recognition instruction; preprocessing the original image to obtain an image to be recognized; processing the image to be recognized using multiple preset convolution kernels to obtain a feature map of the image to be recognized; calculating the variance of each pixel in the feature map; and determining that the original image is a fuzzy image when the variance is less than or equal to a preset fuzzy threshold. The image processing method provided by the present invention can process the image to be recognized using multiple convolution kernels to obtain a feature map, and then compare the variance of each pixel in the feature map with a preset fuzzy threshold, thereby quickly and accurately determining whether the original image is a fuzzy image.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an image processing method and device, a storage medium, and an electronic device. Background Art

[0002] In some business scenarios, for example, in after-sales service scenarios, technicians are usually required to upload pictures related to after-sales service, and the pictures are manually reviewed by the backend.

[0003] However, when technicians use the after-sales app to take pictures, blurry pictures are often produced due to camera shake. In this case, the technicians usually judge whether the picture is blurry subjectively, so it is easy to upload blurry pictures, resulting in the background being unable to complete the review of the picture content. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an image processing method that can accurately identify fuzzy images.

[0005] The present invention also provides an image processing device to ensure the implementation and application of the above method in practice.

[0006] An image processing method, comprising:

[0007] In response to the fuzzy image recognition instruction, obtaining an original image to be processed;

[0008] Preprocessing the original image to obtain an image to be recognized;

[0009] Processing the image to be identified using a plurality of preset convolution kernels to obtain a feature map of the image to be identified;

[0010] Calculating the variance of each pixel in the feature map;

[0011] When the variance is less than or equal to a preset blur threshold, the original image is determined to be a blurred image.

[0012] Optionally, the above method uses a plurality of preset convolution kernels to process the image to be identified to obtain a feature map of the image to be identified, including:

[0013] Determining a convolution kernel template corresponding to the image to be identified; the channel order of the image to be identified is consistent with the image channel processing order corresponding to the convolution kernel template; the convolution kernel template includes a plurality of preset single-channel convolution kernels and a plurality of preset multi-channel convolution kernels;

[0014] Processing the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map;

[0015] The initial feature map is processed using each multi-channel convolution kernel in the convolution kernel template to obtain a feature map of the image to be recognized.

[0016] Optionally, the method described above processes the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map, including:

[0017] Using the N×M convolution kernels in the convolution kernel template, a convolution operation is performed in a first direction of each channel of the image to be identified to obtain a first feature subgraph corresponding to each channel of the image to be identified; N and M are positive integers;

[0018] Combining the first feature subgraphs corresponding to each channel of the image to be identified in the channel order to obtain a first feature graph;

[0019] Performing a convolution operation on each channel of the first feature map in a second direction using the M×N convolution kernel in the convolution kernel template to obtain a second feature submap corresponding to each channel of the first feature map;

[0020] The second feature sub-graphs corresponding to each channel of the first feature graph are combined in the channel order to obtain an initial feature graph.

[0021] Optionally, the above method uses each multi-channel convolution kernel in the convolution kernel template to process the image to be identified to obtain a feature map of the original image, including:

[0022] Performing a convolution operation on each channel of the initial feature map using an M×M×N convolution kernel in the convolution kernel template to obtain a second feature map; N and M are positive integers;

[0023] A convolution operation is performed on each channel of the second feature map using the N×N×M convolution kernel in the convolution kernel template to obtain a feature map of the image to be recognized.

[0024] In the above method, optionally, preprocessing the original image to obtain the image to be recognized includes:

[0025] The order of each channel of the original image is adjusted to obtain an image to be recognized.

[0026] The above method may optionally further include:

[0027] In a case where the variance is greater than the blur threshold, the original image is sent.

[0028] Optionally, the above method, after determining that the original image is a blurred image, further includes:

[0029] Outputting prompt information, wherein the prompt information is used to prompt the user that the original image is a blurred image.

[0030] Optionally, the above method, after determining that the original image is a blurred image, further includes:

[0031] The transmission of said original image is prohibited.

[0032] An image processing device, comprising:

[0033] An acquisition unit, configured to respond to a fuzzy image recognition instruction and acquire an original image to be processed;

[0034] A preprocessing unit, configured to preprocess the original image to obtain an image to be recognized;

[0035] an execution unit, configured to process the image to be identified using a plurality of preset convolution kernels to obtain a feature map of the image to be identified;

[0036] A calculation unit, configured to calculate the variance of each pixel in the feature map;

[0037] The determining unit is configured to determine that the original image is a blurred image when the variance is less than or equal to a preset blur threshold.

[0038] A storage medium includes storage instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned image processing method.

[0039] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to execute the above-mentioned image processing method by one or more processors.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] The present invention provides an image processing method and apparatus, a storage medium, and an electronic device. In response to a fuzzy image recognition instruction, an original image to be processed is obtained; the original image is preprocessed to obtain an image to be recognized; the image to be recognized is processed using multiple preset convolution kernels to obtain a feature map of the image to be recognized; the variance of each pixel in the feature map is calculated; and if the variance is less than or equal to a preset fuzziness threshold, the original image is determined to be a fuzzy image. This method can quickly and accurately identify whether the original image is a fuzzy image. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0043] Figure 1 A flowchart of an image processing method provided by the present invention;

[0044] Figure 2 A flowchart of a process for obtaining a feature map of an image to be identified provided by the present invention;

[0045] Figure 3 An example diagram of an implementation scenario provided by the present invention;

[0046] Figure 4 A schematic structural diagram of an image processing device provided by the present invention;

[0047] Figure 5 This is a structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0050] An embodiment of the present invention provides an image processing method, which can be applied to electronic devices. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:

[0051] S101: In response to a fuzzy image recognition instruction, an original image to be processed is acquired.

[0052] In this embodiment, the blurred image recognition instruction can be an instruction triggered by the user clicking a preset control when the user needs to upload the original image, or it can be an instruction automatically triggered after the user selects the original image to be uploaded.

[0053] Optionally, the native image may be various types of images, for example, an image of a damaged part, a repair order image, or a face image taken by a user.

[0054] S102: Preprocess the original image to obtain an image to be recognized.

[0055] In the embodiment of the present invention, the original image is pre-processed by adjusting at least one of the size, occupied space size, and channel order of the image, and the original image may also be filtered.

[0056] S103: Processing the image to be identified using a plurality of preset convolution kernels to obtain a feature map of the image to be identified.

[0057] In an embodiment of the present invention, a feature map containing features of the image to be identified can be obtained by processing the image to be identified through multiple convolution kernels, and the feature map contains multiple pixel points.

[0058] S104: Calculate the variance of each pixel in the feature map.

[0059] In the embodiment of the present invention, the average value of each pixel may be calculated first, and then the variance of each pixel may be calculated based on the average value of each pixel.

[0060] S105: When the variance is less than or equal to a preset blur threshold, determine that the original image is a blurred image.

[0061] In the embodiment of the present invention, the blur threshold can be set to any value, for example, it can be set to 0. When the variance of the feature map is less than or equal to 0, the original image is determined to be a blurred image.

[0062] Optionally, the blurred image may be an image whose content cannot be recognized.

[0063] An embodiment of the present invention provides an image processing method, comprising: obtaining an original image to be processed in response to a fuzzy image recognition instruction; preprocessing the original image to obtain an image to be recognized; processing the image to be recognized using multiple preset convolution kernels to obtain a feature map of the image to be recognized; calculating the variance of each pixel in the feature map; and determining that the original image is a fuzzy image if the variance is greater than a preset fuzziness threshold. This method can quickly and accurately identify whether the original image is a fuzzy image.

[0064] In the embodiment of the present invention, based on the above solution, optionally, the image to be identified is processed using a plurality of preset convolution kernels to obtain a feature map of the image to be identified, such as Figure 2 As shown, specifically including:

[0065] S201: Determine the convolution kernel template corresponding to the image to be identified; the channel order of the image to be identified is consistent with the image channel processing order corresponding to the convolution kernel template; the convolution kernel template includes multiple preset single-channel convolution kernels and multiple preset multi-channel convolution kernels.

[0066] In this embodiment, the channel order of the image to be identified is consistent with the image channel processing order of the convolution kernel template. For example, the channel order of the image to be identified can be that the image to be identified can be a BGR (blue, green, red) format image, that is, the image to be identified can include 3 channels: B channel, G channel and R channel.

[0067] In some embodiments, the convolution kernel template corresponding to the image to be identified can be determined from each alternative convolution kernel template based on the channel order of the image to be identified. Each alternative convolution kernel template has its own corresponding image channel processing order, and the alternative convolution kernel template can process the image according to its image channel processing order.

[0068] S202: Processing the image to be recognized using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map.

[0069] In this embodiment, the dimension of the image to be identified can be [W, H, 3], where "W" refers to the width of the image to be identified, "H" refers to the height of the image to be identified, and "3" may refer to the number of channels of the image to be identified being 3. The width of the image corresponds to the X-axis direction, and the height of the image corresponds to the Y-axis direction.

[0070] In this embodiment, the initial feature map can be processed using at least one pair of single-channel convolution kernels, each pair of single-channel convolution kernels includes a first single-channel convolution kernel and a second single-channel convolution kernel; the second single-channel convolution kernel is a flipped convolution kernel of the first single-channel convolution kernel.

[0071] Optionally, the single-channel convolution kernel may include at least one of an N×M convolution kernel and an M×N convolution kernel, and the multi-channel convolution kernel may include at least one of an M×M×N convolution kernel and an N×N×M convolution kernel, where N and M are positive integers; N and M may be the same or different.

[0072] Specifically, one possible implementation method of processing the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain the initial feature map is:

[0073] performing a convolution operation in a first direction of each channel of the image to be identified using an N×M convolution kernel in the convolution kernel template to obtain a first feature subgraph corresponding to each channel of the image to be identified; combining the first feature subgraphs corresponding to each channel of the image to be identified in the order of the channels to obtain a first feature graph;

[0074] Performing a convolution operation on each channel of the first feature map in a second direction using the M×N convolution kernel in the convolution kernel template to obtain a second feature submap corresponding to each channel of the first feature map;

[0075] The second feature sub-graphs corresponding to each channel of the first feature graph are combined in the channel order to obtain an initial feature graph.

[0076] In this embodiment, the first direction may be one of the X-axis direction and the Y-axis direction; the second direction may be a direction other than the first direction between the X-axis direction and the Y-axis direction; for example, the first direction may be the X-axis direction, and the second direction may be the Y-axis direction; or, the first direction may be the Y-axis direction, and the second direction may be the X-axis direction.

[0077] Convolution is performed in the X-axis direction of each channel of the image to be identified using an N×M convolution kernel to obtain a first feature subgraph corresponding to each channel; in some embodiments, N may be 3 and M may be 1.

[0078] In this embodiment, the first feature sub-images corresponding to the channels can be combined according to the channel order of the channels to obtain the first feature image. The channel order can be BGR, and the first feature image is an image in BGR format. In some embodiments, the channel order can also be RGB, RBG, BRG, GBR, GRB, etc.

[0079] In this embodiment, the second feature sub-images corresponding to the channels may be combined in the order of the channels, for example, BGR, to obtain an initial feature image, which is an image in the BGR format.

[0080] S203: Utilize each multi-channel convolution kernel in the convolution kernel template to process the initial feature map to obtain a feature map of the image to be recognized.

[0081] In this embodiment, the initial feature map can be processed using at least one pair of multi-channel convolution kernels, each pair of multi-channel convolution kernels includes a first multi-channel convolution kernel and a second multi-channel convolution kernel; the second multi-channel convolution kernel is a flipped convolution kernel of the first multi-channel convolution kernel.

[0082] Specifically, a feasible way to process the image to be identified using each multi-channel convolution kernel in the convolution kernel template to obtain the feature map of the original image is:

[0083] Performing a convolution operation on each channel of the initial feature map using an M×M×N convolution kernel in the convolution kernel template to obtain a second feature map; N and M are positive integers;

[0084] A convolution operation is performed on each channel of the second feature map using the N×N×M convolution kernel in the convolution kernel template to obtain a feature map of the image to be recognized.

[0085] By applying the method provided in the embodiment of the present invention, the image to be identified is first processed by a single-channel convolution kernel, which can reduce the amount of calculation, improve the calculation speed, and quickly obtain the initial feature map. The initial feature map is then convolved with a multi-channel convolution kernel, so that the obtained feature map has global features, so that it can be accurately judged whether the original image is a blurred image based on the feature map.

[0086] In an embodiment of the present invention, based on the above solution, optionally, preprocessing the original image to obtain the image to be recognized includes:

[0087] The order of each channel of the original image is adjusted to obtain an image to be recognized.

[0088] In this embodiment, the original image may be an RGB image, i.e., the channel order of the original image is RBG. By adjusting the channel order of the original image, the image to be identified is obtained. The image to be identified may be a BGR image, with the channel order being BGR. By converting the RGB image to BGR, the hardware dependency of the image during the convolution process can be effectively reduced.

[0089] In some embodiments, the format of the image may be adjusted to GRB, GBR, BRG, etc., or the order of the channels of the original image may not be adjusted. The specific format may be set according to actual needs.

[0090] In an embodiment of the present invention, based on the above solution, optionally, the following is further included:

[0091] In a case where the variance is greater than the blur threshold, the original image is sent.

[0092] In this embodiment, the blur threshold may be set to 0. When the variance is greater than the blur threshold, it is determined that the original image is not a blurry image, and the original image may be transmitted.

[0093] Optionally, the original image may be sent to a server, which processes the original image.

[0094] By applying the method provided by the embodiment of the present invention, the original image can be sent to the server when the original image is not blurred, so that the server can accurately obtain the image content in the original image.

[0095] In an embodiment of the present invention, based on the above solution, optionally, after determining that the original image is a blurred image, the method further includes:

[0096] Outputting prompt information, wherein the prompt information is used to prompt the user that the original image is a blurred image.

[0097] In this embodiment, the prompt information can be output in the form of a message pop-up window to prompt the user that the original image is a blurred image, so that the user can take a new blurred image.

[0098] In an embodiment of the present invention, based on the above solution, optionally, after determining that the original image is a blurred image, the method further includes:

[0099] Transfer of original images is prohibited.

[0100] In this embodiment, after detecting that the original image is a blurred image, the transmission of the original image is prohibited, thereby avoiding sending the original image to the server and causing waste of server resources.

[0101] The image processing method provided by the embodiment of the present invention can be applied in a variety of image recognition scenarios. For example, it can be applied in after-sales service scenarios. The work order system assigns service tasks to after-sales technicians, and the technicians perform the corresponding after-sales service at the time and place assigned by the system. After the technicians complete the after-sales service, the technicians submit images related to the after-sales service to the work order system through electronic devices, such as images of damaged parts, images of repair orders, and images of the faces of after-sales technicians, etc. These images are uploaded to the database, and the work order is manually reviewed by the backend. The details are as follows:

[0102] See also Figure 3 , which is an example diagram of an implementation scenario provided by an embodiment of the present invention. The implementation scenario provided by an embodiment of the present invention is a work order system including a terminal 301 and a server 302.

[0103] When implementing, Figure 3 The terminal 301 shown can be a personal computer, a handheld or portable device, a tablet device, a multi-processor device, a distributed computing environment including any of the above devices or devices, and the device where the server 302 is located can be a server or a server cluster consisting of multiple servers, or a cloud service platform. The terminal 301 establishes a communication connection with the server 302 via a network.

[0104] The network involved in the embodiments of the present invention is a medium that provides a communication link. The network may include various connection types, such as wired or wireless communication links, etc.

[0105] Step 1: The terminal can obtain the original image to be uploaded, where the width of the original image is W, corresponding to the x-axis direction, and the height is H, corresponding to the y-axis direction; and the original image is a 3-channel RGB image. The feature map dimension refers to [width, height, number of channels], so the dimension of the original image is [W, H, 3].

[0106] Step 2: After obtaining the original image, the terminal can convert the original image from RGB format to BGR format.

[0107] Step 3: Perform a 3×1 convolution operation in the x-axis direction on each channel of the image converted to BGR format (corresponding feature map dimension is [W, H, 1]).

[0108] Optionally, the 3×1 convolution kernel is [k1, k2, k3] = [0.25, 0.5, 0.25].

[0109] In an example, assuming that the pixel values ​​on the R channel of an image converted to BGR format are [x1, x2, x3, x4] = [1, 2, 3, 4], then a 3×1 convolution on the x-axis means multiplying three adjacent pixels by the convolution kernel coefficient and then adding them together. The specific calculation process is as follows:

[0110] z1=x1*k1+x2*k2+x3*k3=1*0.25+2*0.5+3*0.25=2;

[0111] z2=x2*k1+x3*k2+x4*k3=2*0.25+3*0.5+4*0.25=3.

[0112] Among them, the output result is [z1, z2] = [2, 3], and from the example, it can be seen that the output feature map dimension is [W-2, H, 1].

[0113] Step 4: Merge the first feature subgraphs corresponding to each channel obtained in step 3 according to BGR in the channel dimension to obtain the first feature graph. The feature dimension of the first feature graph is [W-2, H, 3].

[0114] Step 5: Perform a 1×3 convolution operation in the y-axis direction on each channel of the first feature map (corresponding to the feature map dimension of [W-2, H, 1]);

[0115] Optionally, the 1×3 convolution kernel is

[0116] In one example, suppose the pixel value of the image on the R channel is Then the 1×3 convolution on the y-axis refers to multiplying three adjacent pixels according to the convolution kernel coefficient and then adding them together. Assuming that the output results are z3 and z4, the calculation process is as follows:

[0117] z3=y1*k1+y2*k2+y3*k3=1*0.25+2*0.5+3*0.25=2;

[0118] z4=y2*k1+y3*k2+y4*k3=2*0.25+3*0.5+4*0.25=3.

[0119] The output is And from the example, we can see that the output feature map dimension is [W-2, H-2, 1].

[0120] Step 6: Merge the second feature sub-graphs corresponding to each channel obtained in step 5 according to BGR in the channel dimension to obtain the initial feature graph. The feature dimension of the initial feature graph becomes [W-2, H-2, 3].

[0121] Step 7: Perform 1×1×3 convolution on the channel of the initial feature map.

[0122] Optionally, the convolution kernel is [k4, k5, k6] = [0.114, 0.587, 0.299].

[0123] In an example image, assuming that the pixel value at the (x, y) coordinate point is [B, G, R] = [1, 2, 3], then the 1×1×3 convolution is the output pixel value P at each pixel point, specifically:

[0124] P=B*k1+G*k2+R*k3=1*0.114+2*0.587+3*0.299=2.185.

[0125] The above convolution operation is performed on each (x, y) coordinate point of the initial feature map to obtain a second feature map, and the feature map dimension of the second feature map is [W-2, H-2, 1].

[0126] Step 8: Perform convolution on the third feature map according to 3×3×1.

[0127] Optionally, the convolution kernel is

[0128] In one example, suppose the current feature map pixel value is Then the 3×3×1 convolution refers to multiplying and adding the adjacent 3*3=9 pixels according to the convolution kernel coefficient. Assuming the output result is z5, the calculation formula is:

[0129]

[0130] Among them, the output result is [z5]=[0]. By performing the convolution operation on the second feature map, a feature map can be obtained, and it can be seen from the example that the output feature map dimension is [W-4, H-4, 1].

[0131] Step 9: Calculate the mean M and variance Var of the feature map; compare the variance with the preset threshold. The details are as follows:

[0132]

[0133] Among them, θ is the threshold for determining whether the feature map is blurred.

[0134] If the image is judged to be blurred, the after-sales technician will be prompted to retake the image to avoid uploading the blurred image to the server, which would cause unnecessary waste of resources.

[0135] and Figure 1 Corresponding to the method described above, an embodiment of the present invention further provides an image processing device for Figure 1 The specific implementation of the method in the embodiment of the present invention is that the image processing device provided by the embodiment of the present invention can be applied to electronic devices, and its structural diagram is as follows Figure 4 As shown, specifically including:

[0136] An acquisition unit 401 is configured to respond to a fuzzy image recognition instruction and acquire an original image to be processed;

[0137] A preprocessing unit 402 is used to preprocess the original image to obtain an image to be recognized;

[0138] The execution unit 403 is configured to process the image to be identified using a plurality of preset convolution kernels to obtain a feature map of the image to be identified;

[0139] A calculation unit 404 is used to calculate the variance of each pixel in the feature map;

[0140] The determining unit 405 is configured to determine that the original image is a blurred image when the variance is less than or equal to a preset blur threshold.

[0141] An embodiment of the present invention provides an image processing device that can respond to a fuzzy image recognition instruction, obtain an original image to be processed, preprocess the original image to obtain an image to be recognized, process the image to be recognized using multiple preset convolution kernels to obtain a feature map of the image to be recognized, calculate the variance of each pixel in the feature map, and determine that the original image is a fuzzy image if the variance is greater than a preset fuzziness threshold. This device can quickly and accurately identify whether an original image is a fuzzy image.

[0142] In an embodiment of the present invention, based on the above solution, optionally, the execution unit 403 includes:

[0143] a determination subunit, configured to determine a convolution kernel template corresponding to the image to be identified; the channel order of the image to be identified is consistent with the image channel processing order corresponding to the convolution kernel template; the convolution kernel template includes a plurality of preset single-channel convolution kernels and a plurality of preset multi-channel convolution kernels;

[0144] A first execution subunit is configured to process the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map;

[0145] The second execution subunit is used to process the initial feature map using each multi-channel convolution kernel in the convolution kernel template to obtain the feature map of the image to be recognized.

[0146] In an embodiment provided by the present invention, based on the above solution, optionally, the first execution subunit includes:

[0147] A first convolution submodule is configured to perform a convolution operation in a first direction of each channel of the image to be identified using an N×M convolution kernel in the convolution kernel template to obtain a first feature subgraph corresponding to each channel of the image to be identified; N and M are positive integers;

[0148] A first execution submodule, configured to combine each channel of the image to be identified with the first feature subgraph in the channel order to obtain a first feature graph;

[0149] A second convolution submodule is configured to perform a convolution operation in a second direction of each channel of the first feature map using the M×N convolution kernel in the convolution kernel template to obtain a second feature submap corresponding to each channel of the first feature map;

[0150] The second execution submodule is configured to combine the second feature subgraphs corresponding to each channel of the first feature graph in the order of the channels to obtain an initial feature graph.

[0151] In an embodiment provided by the present invention, based on the above solution, optionally, the second execution subunit includes:

[0152] A third convolution submodule is configured to perform a convolution operation on each channel of the initial feature map using an M×M×N convolution kernel in the convolution kernel template to obtain a second feature map; N and M are positive integers;

[0153] The fourth convolution submodule is used to use the N×N×M convolution kernel in the convolution kernel template to perform a convolution operation on each channel of the second feature map to obtain the feature map of the image to be recognized.

[0154] In an embodiment provided by the present invention, based on the above solution, optionally, the pre-processing unit includes:

[0155] The adjustment subunit is used to adjust the order of the channels of the original image to obtain the image to be recognized.

[0156] In an embodiment provided by the present invention, based on the above solution, optionally, the image processing device further includes:

[0157] A sending unit is configured to send the original image if the variance is greater than the blur threshold.

[0158] In an embodiment provided by the present invention, based on the above solution, optionally, the image processing device further includes:

[0159] The output unit is used to output prompt information, where the prompt information is used to prompt the user that the original image is a blurred image.

[0160] In an embodiment provided by the present invention, based on the above solution, optionally, the image processing device further includes:

[0161] A processing unit is configured to prohibit transmission of the original image.

[0162] The specific principles and execution processes of each unit and module in the image processing device disclosed in the above embodiment of the present invention are the same as those of the image processing method disclosed in the above embodiment of the present invention. Please refer to the corresponding parts of the image processing method provided in the above embodiment of the present invention, and no further details will be given here.

[0163] The embodiment of the present invention further provides an electronic device, the structural diagram of which is shown in FIG. Figure 5 As shown, for example, the electronic device 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0164] Reference Figure 5 , the electronic device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output (I / O) interface 512 , a sensor component 514 , and a communication component 516 .

[0165] The processing component 502 generally controls the overall operation of the electronic device 500, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 502 may include one or more modules to facilitate interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate interaction between the multimedia component 508 and the processing component 502.

[0166] The memory 504 is configured to store various types of data to support operations on the device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phone book data, messages, pictures, videos, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0167] The power supply assembly 506 provides power to the various components of the electronic device 500. The power supply assembly 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 500.

[0168] The multimedia component 508 includes a screen that provides an output interface between the electronic device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0169] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 500 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 504 or transmitted via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0170] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0171] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 can detect the open / closed state of the device 500, the relative positioning of components, such as the display and keypad of the electronic device 500. The sensor assembly 514 can also detect changes in the position of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, the orientation or acceleration / deceleration of the electronic device 500, and temperature changes of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0172] The communication component 516 is configured to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0173] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned image processing method.

[0174] In an exemplary embodiment, a storage medium including instructions is further provided, such as a memory 504 including instructions. The instructions may be executed by the processor 520 of the electronic device 500 to perform the above-mentioned image processing method, which specifically includes:

[0175] In response to the fuzzy image recognition instruction, obtaining an original image to be processed;

[0176] Preprocessing the original image to obtain an image to be recognized;

[0177] Processing the image to be identified using a plurality of preset convolution kernels to obtain a feature map of the image to be identified;

[0178] Calculating the variance of each pixel in the feature map;

[0179] When the variance is less than or equal to a preset blur threshold, the original image is determined to be a blurred image.

[0180] Optionally, the method described above processes the image to be identified using a plurality of preset convolution kernels to obtain a feature map of the image to be identified, including:

[0181] Optionally, the above method uses a plurality of preset convolution kernels to process the image to be identified to obtain a feature map of the image to be identified, including:

[0182] Determining a convolution kernel template corresponding to the image to be identified; the channel order of the image to be identified is consistent with the image channel processing order corresponding to the convolution kernel template; the convolution kernel template includes a plurality of preset single-channel convolution kernels and a plurality of preset multi-channel convolution kernels;

[0183] Processing the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map;

[0184] The initial feature map is processed using each multi-channel convolution kernel in the convolution kernel template to obtain a feature map of the image to be recognized.

[0185] Optionally, the method described above processes the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map, including:

[0186] Using the N×M convolution kernels in the convolution kernel template, a convolution operation is performed in a first direction of each channel of the image to be identified to obtain a first feature subgraph corresponding to each channel of the image to be identified; N and M are positive integers;

[0187] Combining the first feature subgraph corresponding to each channel of the image to be identified in the channel order to obtain a first feature graph;

[0188] Performing a convolution operation on each channel of the first feature map in a second direction using the M×N convolution kernel in the convolution kernel template to obtain a second feature submap corresponding to each channel of the first feature map;

[0189] The second feature sub-graphs corresponding to each channel of the first feature graph are combined in the channel order to obtain an initial feature graph.

[0190] Optionally, the above method uses each multi-channel convolution kernel in the convolution kernel template to process the image to be identified to obtain a feature map of the original image, including:

[0191] Performing a convolution operation on each channel of the initial feature map using an M×M×N convolution kernel in the convolution kernel template to obtain a second feature map; N and M are positive integers;

[0192] A convolution operation is performed on each channel of the second feature map using the N×N×M convolution kernel in the convolution kernel template to obtain a feature map of the image to be recognized.

[0193] In the above method, optionally, preprocessing the original image to obtain the image to be recognized includes:

[0194] The order of each channel of the original image is adjusted to obtain an image to be recognized.

[0195] The above method may optionally further include:

[0196] In a case where the variance is greater than the blur threshold, the original image is sent.

[0197] Optionally, the above method, after determining that the original image is a blurred image, further includes:

[0198] Outputting prompt information, wherein the prompt information is used to prompt the user that the original image is a blurred image.

[0199] Optionally, the above method, after determining that the original image is a blurred image, further includes:

[0200] The transmission of said original image is prohibited.

[0201] Alternatively, the storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0202] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similarities between the various embodiments can be referred to in conjunction with each other. For device embodiments, since they are generally similar to method embodiments, their description is relatively simple, and for relevant details, reference can be made to the description of the method embodiments.

[0203] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0204] For the convenience of description, the above device is described as being divided into various units according to their functions. Of course, when implementing the present invention, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0205] From the above description of the embodiments, it can be seen that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0206] The above is a detailed introduction to an image processing method provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An image processing method, characterized in that: include: In response to the fuzzy image recognition instruction, obtaining an original image to be processed; Preprocessing the original image to obtain an image to be recognized; Processing the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map; Performing a convolution operation on each channel of the initial feature map using the M×M×N convolution kernel in the convolution kernel template to obtain a second feature map; N and M are positive integers; Performing a convolution operation on each channel of the second feature map using an N×N×M convolution kernel in the convolution kernel template to obtain a feature map of the image to be recognized; Calculating the variance of each pixel in the feature map; When the variance is less than or equal to a preset blur threshold, the original image is determined to be a blurred image.

2. The method according to claim 1, characterized in that Also includes: Determining a convolution kernel template corresponding to the image to be identified; the channel order of the image to be identified is consistent with the image channel processing order corresponding to the convolution kernel template; The convolution kernel template includes multiple preset single-channel convolution kernels and multiple preset multi-channel convolution kernels.

3. The method according to claim 1, characterized in that The method of processing the image to be identified by using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map includes: Using the N×M convolution kernels in the convolution kernel template, a convolution operation is performed in a first direction of each channel of the image to be identified to obtain a first feature subgraph corresponding to each channel of the image to be identified; N and M are positive integers; Combining the first feature subgraphs corresponding to each channel of the image to be identified in the channel order to obtain a first feature graph; Performing a convolution operation on each channel of the first feature map in a second direction using the M×N convolution kernel in the convolution kernel template to obtain a second feature submap corresponding to each channel of the first feature map; The second feature sub-graphs corresponding to each channel of the first feature graph are combined in the channel order to obtain an initial feature graph.

4. The method according to claim 1, wherein The preprocessing of the original image to obtain the image to be recognized includes: The order of each channel of the original image is adjusted to obtain an image to be recognized.

5. The method according to claim 1, wherein Also includes: In a case where the variance is greater than the blur threshold, the original image is sent.

6. The method according to claim 1, characterized in that After determining that the original image is a blurred image, the method further includes: Outputting prompt information, wherein the prompt information is used to prompt the user that the original image is a blurred image.

7. An image processing device, characterized in that include: An acquisition unit, configured to respond to a fuzzy image recognition instruction and acquire an original image to be processed; A preprocessing unit, configured to preprocess the original image to obtain an image to be recognized; The execution unit includes: a first execution subunit and a second execution subunit; The first execution subunit is configured to process the image to be identified using each single-channel convolution kernel in the convolution kernel template to obtain an initial feature map; The second execution subunit includes: a third convolution submodule and a fourth convolution submodule; The third convolution submodule is used to perform a convolution operation on each channel of the initial feature map using the M×M×N convolution kernel in the convolution kernel template to obtain a second feature map; N and M are positive integers; The fourth convolution submodule is configured to perform a convolution operation on each channel of the second feature map using the N×N×M convolution kernel in the convolution kernel template to obtain the feature map of the image to be recognized; A calculation unit, configured to calculate the variance of each pixel in the feature map; The determining unit is configured to determine that the original image is a blurred image when the variance is less than or equal to a preset blur threshold.

8. A storage medium, characterized in that: The storage medium includes storage instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The system comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to execute the image processing method according to any one of claims 1 to 6 by one or more processors.

Images