Image motion blur processing method and device, equipment and medium

Abstract

The invention discloses an image motion blur processing method and device, equipment and a medium. The method comprises the following steps: acquiring a to-be-processed clear image, and determining detection frame image position information and category labels of each object in the to-be-processed clear image; determining a moving object in each object; obtaining an image slice of each moving object; for each moving object, randomly generating a filtering kernel corresponding to the moving object, and using the corresponding filtering kernel to perform convolution processing on the image slice of the moving object to obtain a motion blurred image slice of each moving object; and covering the detection frame image of each moving object in the to-be-processed clear image with the motion blurred image slice of each moving object, and carrying out smooth processing to obtain a corresponding motion blurred image. According to the embodiment of the invention, different processing can be carried out on each moving object in the clear image, and the motion blurred image which is blurred due to the movement of each moving object is effectively simulated.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to an image motion blur processing method, device, equipment and medium. Background Art

[0002] The business system of a financial institution often needs to collect and use relevant images in the process of processing the business of the financial institution. The images collected by the business system of the financial institution may include motion blurred images. Motion blurred images refer to images that appear blurred due to the movement of the target object during the image acquisition process. In order to prevent motion blurred images from affecting the effectiveness of the business processing process, the business system of the financial institution needs to perform motion blur processing on the clear images in the image database to obtain simulated motion blurred images, and then use the simulated motion blurred images to train the machine learning model to obtain an image processing model for deblurring the motion blurred images. Motion blur processing of a clear image means processing the clear image so that the clear image appears blurred, thereby simulating a motion blurred image that is blurred due to motion.

[0003] In the related art, the commonly used image motion blur processing scheme is: by flipping, translating, rotating, adding noise and Gaussian blurring the clear image in the image database to blur the clear image, and obtain a simulated motion blurred image. The clear image in the image database will contain multiple objects with motion properties. Each object with motion properties moves in different ways when it moves. The image motion blur processing scheme in the related art processes the clear image as a whole, and cannot perform different processing on each moving object in the clear image. It cannot effectively simulate the motion blurred image that is blurred due to the movement of each moving object, and cannot provide high-quality training samples for the image processing model. Summary of the invention

[0004] The present invention provides an image motion blur processing method, device, equipment and medium to solve the problem that the image motion blur processing scheme in the related art cannot perform different processing for each moving object in the clear image, cannot effectively simulate the motion blurred image caused by the movement of each moving object, and cannot provide high-quality training samples for the image processing model.

[0005] According to one aspect of the present invention, there is provided a method for processing image motion blur, comprising:

[0006] Acquire a clear image to be processed, perform target detection on the clear image to be processed, and determine detection frame image position information and category labels of each object in the clear image to be processed;

[0007] Determine the moving objects among the objects according to the detection frame image position information and category labels of the objects;

[0008] Determine the detection frame image of each moving object according to the detection frame image position information of each moving object, copy the detection frame image of each moving object, and obtain the image slice of each moving object;

[0009] For each moving object, a filter kernel corresponding to the moving object is randomly generated, and the image slices of the moving object are convolved using the filter kernel corresponding to the moving object to obtain motion blurred image slices of each moving object;

[0010] The motion blurred image slices of each moving object are overlaid on the detection frame image of each moving object in the clear image to be processed, and the edge area of ​​the detection frame image of each moving object after overlay is smoothed to obtain a motion blurred image corresponding to the clear image to be processed.

[0011] According to another aspect of the present invention, there is provided an image motion blur processing device, comprising:

[0012] The target detection module is used to obtain a clear image to be processed, perform target detection on the clear image to be processed, and determine the detection frame image position information and category label of each object in the clear image to be processed;

[0013] A moving object determination module, used to determine the moving object among the objects according to the detection frame image position information and category labels of the objects;

[0014] A slice extraction module is used to determine the detection frame image of each moving object according to the detection frame image position information of each moving object, and to copy the detection frame image of each moving object to obtain the image slice of each moving object;

[0015] A slice processing module is used to randomly generate a filter kernel corresponding to each moving object, and use the filter kernel corresponding to the moving object to perform convolution processing on the image slice of the moving object to obtain motion blurred image slices of each moving object;

[0016] The slice overlay module is used to overlay the motion blurred image slices of each moving object onto the detection frame image of each moving object in the clear image to be processed, and to smooth the edge area of ​​the detection frame image of each moving object after overlay to obtain a motion blurred image corresponding to the clear image to be processed.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor;

[0019] and a memory communicatively coupled to the at least one processor;

[0020] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image motion blur processing method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image motion blur processing method described in any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the image motion blur processing method described in any embodiment of the present invention is implemented.

[0023] The technical solution of the embodiment of the present invention obtains a clear image to be processed, performs target detection on the clear image to be processed, and determines the detection frame image position information and category labels of each object in the clear image to be processed; then, according to the detection frame image position information and category labels of each object, determines the moving objects in each object; according to the detection frame image position information of each moving object, determines the detection frame image of each moving object, copies the detection frame image of each moving object, and obtains the image slice of each moving object; for each moving object, randomly generates a filter kernel corresponding to the moving object, and uses the filter kernel corresponding to the moving object to perform convolution processing on the image slice of the moving object, and obtains the motion blurred image slice of each moving object; finally, overlays the motion blurred image slices of each moving object on the detection frame image of each moving object in the clear image to be processed, and smoothes the edge area of ​​the covered detection frame image of each moving object to obtain the motion blurred image corresponding to the clear image to be processed, thereby solving the problem of image motion blur processing in the related art. The solution cannot perform different processing on each moving object in the clear image, cannot effectively simulate the motion blurred image caused by the movement of each moving object, and cannot provide high-quality training samples for the image processing model. The moving objects in the clear image can be determined based on the detection frame image position information and category labels of each object in the clear image. The image slices of each moving object can be convolved based on the randomly generated filter kernel corresponding to the moving object to obtain image slices of each moving object with motion blur effect. The image slices of each moving object with motion blur effect are overlaid on the detection frame image of each moving object in the clear image, and the edge area of ​​the detection frame image is smoothed to obtain a smooth and natural motion blurred image caused by the movement of each moving object. Different processing can be performed on each moving object in the clear image, and the motion blurred image caused by the movement of each moving object can be effectively simulated, so as to provide high-quality training samples for the image processing model.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flowchart of an image motion blur processing method provided in Embodiment 1 of the present invention.

[0027] Figure 2 A flowchart of an image motion blur processing method provided in Embodiment 2 of the present invention.

[0028] Figure 3 A schematic diagram of the structure of an image motion blur processing device provided in Embodiment 3 of the present invention.

[0029] Figure 4 A schematic diagram of the structure of an electronic device for implementing the image motion blur processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "target", "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise", "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data comply with relevant laws, regulations and standards in relevant regions.

[0033] Embodiment 1

[0034] Figure 1This is a flow chart of a method for image motion blur processing provided in the first embodiment of the present invention. This embodiment can be applied to the situation where a clear image in an image database is subjected to motion blur processing, so that the clear image appears blurred, and a simulated motion blurred image is obtained. The method can be executed by an image motion blur processing device, which can be implemented in the form of hardware and / or software, and can be configured in the business system of a financial institution. The business system of a financial institution can be a server set up in the financial institution for processing the business of the financial institution. Figure 1 As shown, the method includes:

[0035] Step 101: Acquire a clear image to be processed, perform target detection on the clear image to be processed, and determine the detection frame image position information and category label of each object in the clear image to be processed.

[0036] Optionally, the image database may be a pre-set database for storing images that are not blurred and collected by the business system of a financial institution. A plurality of clear images are stored in the image database. Each clear image contains a plurality of moving objects. A moving object may refer to an object with motion properties, that is, an object that can move. For example, a dog, a cat, etc. Each clear image may also contain one or more stationary objects. A stationary object may refer to an object without motion properties, that is, an object that usually does not move. For example, the sky, a tree, a building, etc. Each clear image stored in the image database is an image authorized by the user or fully authorized by all parties, and the collection, use and processing of each clear image comply with the relevant laws, regulations and standards of the relevant region.

[0037] Optionally, the clear image to be processed is a clear image that needs to be subjected to motion blur processing. Acquiring the clear image to be processed includes: taking out a clear image from an image database as the clear image to be processed.

[0038] Optionally, each object in the clear image to be processed refers to each object contained in the clear image to be processed. For each object, the detection frame image of the object may refer to a rectangular image area containing the object in the clear image to be processed. The detection frame image position information of the object may be composed of the horizontal coordinate of the center point of the detection frame image of the object, the vertical coordinate of the center point of the detection frame image of the object, the width of the detection frame image of the object, and the height of the detection frame image of the object. The center point of the detection frame image may refer to the intersection of two diagonals of the detection frame image. The horizontal coordinate of the center point of the detection frame image may refer to the horizontal coordinate of the center point of the detection frame image in the image coordinate system of the clear image to be processed. The vertical coordinate of the center point of the detection frame image may refer to the vertical coordinate of the center point of the detection frame image in the image coordinate system of the clear image to be processed. The category label of the object may be a word used to describe the object. Exemplarily, the objects that may be contained in the clear image to be processed include: dog, cat, sky, tree, building. The category label of the dog is dog. The category label of the cat is cat. The category label of the sky is sky. The category label of the tree is tree. The category label of the building is building.

[0039] Optionally, target detection is performed on the clear image to be processed to determine the detection box image position information and category label of each object in the clear image to be processed, including: inputting the clear image to be processed into a pre-trained target detection model, and obtaining the detection box image position information and category label of each object in the clear image to be processed output by the target detection model.

[0040] Therefore, based on the pre-trained target detection model, the detection frame image position information and category label of each object in the clear image to be processed can be quickly determined.

[0041] Optionally, a pre-trained target detection model is provided in the business system of the financial institution. The pre-trained target detection model is used to perform target detection on the image and determine the detection frame image position information and category label of each object in the image. The input of the target detection model is the image, and the output of the target detection model is the detection frame image position information and category label of each object in the image. After the image is input into the pre-trained target detection model, the pre-trained target detection model will analyze the image, determine the detection frame image position information and category label of each object in the image, and then output the detection frame image position information and category label of each object in the image. The clear image to be processed can be input into the pre-trained target detection model. The pre-trained target detection model analyzes the clear image to be processed, determines the detection frame image position information and category label of each object in the clear image to be processed, and then outputs the detection frame image position information and category label of each object in the clear image to be processed. The detection frame image position information and category label of each object in the clear image to be processed output by the pre-trained target detection model can be obtained, thereby determining the detection frame image position information and category label of each object in the clear image to be processed.

[0042] Optionally, a set number of images, as well as the detection frame image position information and category labels of each object in each image can be used to train the machine learning model to obtain a target detection model, and then the target detection model can be set in the business system of the financial institution. The machine learning model can be a deep learning model.

[0043] Step 102: Determine the moving objects among the objects according to the detection frame image position information and category labels of the objects.

[0044] Optionally, the moving objects among the objects are determined based on the detection frame image position information and category labels of each object, including: performing the following operations for each object: detecting whether there is a category label of the object in the stationary object label list; if it is detected that the category label of the object exists in the stationary object category label set, determining that the object is not a moving object.

[0045] Therefore, it is possible to determine whether each object in the clear image to be processed is a moving object based on the stationary object category label set.

[0046] Optionally, the stationary object label list may be a pre-set list for storing category labels of stationary objects. The stationary object label list stores category labels of multiple stationary objects. If the category label of the object exists in the stationary object category label set, it indicates that the object is a stationary object, and it can be determined that the object is not a moving object. For each object in the clear image to be processed, it can be detected whether the category label of the object exists in the stationary object label list. When it is detected that the category label of the object exists in the stationary object category label set, it can be determined that the object is not a moving object.

[0047] Optionally, detecting whether there is a category label of the object in the stationary object label list includes: detecting whether each category label stored in the stationary object label list is the same as the category label of the object; if it is detected that any category label stored in the stationary object label list is the same as the category label of the object, determining that the category label of the object exists in the stationary object label list; if it is detected that all category labels stored in the stationary object label list are different from the category label of the object, determining that the category label of the object does not exist in the stationary object label list.

[0048] Optionally, determining a moving object among the objects based on the detection frame image position information and category label of each object also includes: if it is detected that the category label of the object does not exist in the stationary object category label set, calculating the product of the width and height in the detection frame image position information of the object to obtain the area of ​​the detection frame image of the object; calculating the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed, and judging whether the ratio is greater than a target value; if the ratio is greater than the target value, determining that the object is not a moving object; if the ratio is greater than or equal to the target value, determining that the object is a moving object.

[0049] Therefore, it is possible to determine whether each object in the clear image to be processed is a moving object based on the stationary object category label set and the area of ​​the detection frame image of each object in the clear image to be processed.

[0050] Optionally, the detection frame image position information of the object includes the width of the detection frame image of the object and the height of the detection frame image of the object. The product of the width and height of the detection frame image in the detection frame image position information of the object is the area of ​​the detection frame image of the object. The target value may be a preset value. Normally, after determining that the category label of the object does not exist in the stationary object category label set, if the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed is greater than the target value, it can be determined that the object is not a moving object. If the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed is less than or equal to the target value, it can be determined that the object is a moving object. Exemplarily, the target value is 0.4.

[0051] Optionally, for each object in the clear image to be processed, when the category label of the object does not exist in the set of static object category labels, the product of the width and height in the detection frame image position information of the object can be calculated to obtain the area of ​​the detection frame image of the object. The width and height of the clear image to be processed can be detected to obtain the width and height of the clear image to be processed, and then the product of the width and height of the clear image to be processed can be calculated to obtain the area of ​​the clear image to be processed. The ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed can be calculated to determine whether the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed is greater than the target value. If the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed is greater than the target value, it can be determined that the object is not a moving object. If the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed is less than or equal to the target value, it can be determined that the object is a moving object.

[0052] Step 103: determine the detection frame image of each moving object according to the detection frame image position information of each moving object, copy the detection frame image of each moving object, and obtain the image slice of each moving object.

[0053] Optionally, the image slices of each moving object may refer to the detection frame images of the moving objects extracted from the clear image to be processed. For each moving object in the clear image to be processed, the detection frame image of the moving object in the clear image to be processed may be determined according to the position information of the detection frame image of the moving object, and the detection frame image of the moving object in the clear image to be processed may be copied to obtain the image slices of the moving object.

[0054] Step 104: for each moving object, randomly generate a filter kernel corresponding to the moving object, and use the filter kernel corresponding to the moving object to perform convolution processing on the image slices of the moving object to obtain motion blurred image slices of each moving object.

[0055] Optionally, for each moving object in the clear image to be processed, the filter kernel corresponding to the moving object is a convolution kernel used to perform convolution processing on the image slice of the moving object so that the image slice of the moving object appears blurred, thereby simulating the image blurred due to the motion of the moving object. The motion blurred image slice of the moving object is an image slice of the moving object that appears blurred after the convolution processing, and is an image slice of the moving object with a motion blur effect.

[0056] Optionally, for each moving object, a filter kernel corresponding to the moving object is randomly generated, and the filter kernel corresponding to the moving object is used to perform convolution processing on image slices of the moving object to obtain motion blurred image slices of each moving object, including: performing the following operations for each moving object: randomly generating an angle; determining a unit matrix corresponding to the moving object; performing an affine transformation on the unit matrix according to the angle, rotating the unit matrix by the angle to obtain the filter kernel corresponding to the moving object, and normalizing the filter kernel; and using the filter kernel to perform convolution processing on image slices of the moving object to obtain motion blurred image slices of the moving object.

[0057] Therefore, based on a random angle, a filter kernel can be generated for performing convolution processing on image slices of a moving object to make the image slices of the moving object appear blurred, thereby simulating the blurred image caused by the movement of the moving object. The generated filter kernel can be used to perform convolution processing on the image slices of the moving object to obtain image slices of the moving object with a motion blur effect.

[0058] Optionally, randomly generating an angle includes: generating a random decimal between 0 and 1 through a random number generator, and multiplying the generated decimal by 360° to obtain an angle. The obtained angle is the randomly generated angle. The random number generator may be a software module or hardware module pre-set to generate a random decimal between 0 and 1. A random decimal between 0 and 1 may be generated through a random number generator, and then the generated decimal is multiplied by 360° to obtain a randomly generated angle.

[0059] Optionally, the unit matrix corresponding to the moving object may be a unit matrix for generating a filter kernel corresponding to the moving object. The unit matrix corresponding to the moving object is a 3×3 unit matrix, a 5×5 unit matrix, a 7×7 unit matrix, or a 9×9 unit matrix. Determining the unit matrix corresponding to the moving object includes: generating a random decimal between 0 and 1 through a random number generator; if the generated decimal is in a first numerical interval, determining the 3×3 unit matrix as the unit matrix corresponding to the moving object; if the generated decimal is in a second numerical interval, determining the 5×5 unit matrix as the unit matrix corresponding to the moving object; if the generated decimal is in a third numerical interval, determining the 7×7 unit matrix as the unit matrix corresponding to the moving object; if the generated decimal is in a fourth numerical interval, determining the 9×9 unit matrix as the unit matrix corresponding to the moving object. The first numerical interval, the second numerical interval, the third numerical interval, and the fourth numerical interval are four different numerical intervals that are preset. Exemplarily, the first numerical interval is greater than 0 and less than or equal to 0.25. The second numerical interval is greater than 0.25 and less than or equal to 0.5. The third numerical interval is greater than 0.5 and less than or equal to 0.75. The fourth numerical interval is greater than 0.75 and less than 1. The larger the dimension of the unit matrix corresponding to the moving object, the more significant the motion blur effect of the filter kernel corresponding to the moving object generated based on the unit matrix corresponding to the moving object, and the greater the equivalent simulated object motion speed.

[0060] Optionally, an affine transformation can be performed on the unit matrix corresponding to the moving object according to a randomly generated angle, the unit matrix corresponding to the moving object can be rotated by the randomly generated angle, and the rotated unit matrix corresponding to the moving object can be determined as the filter kernel corresponding to the moving object, thereby obtaining the filter kernel corresponding to the moving object.

[0061] Optionally, a target rotation matrix may be generated based on the randomly generated angle. The target rotation matrix is ​​a matrix used to rotate a specified matrix by a randomly generated angle. An affine transformation may be performed on the unit matrix corresponding to the moving object based on the target rotation matrix to rotate the unit matrix corresponding to the moving object by the randomly generated angle.

[0062] Optionally, the filter kernel is normalized, including: calculating the product of the number of rows and the number of columns of the filter kernel; and for each element in the filter kernel, updating the value of the element to the ratio of the value to the calculated product.

[0063] Optionally, the filter kernel corresponding to the moving object can be used as a convolution kernel for convolution processing of the image slice of the moving object, and the image slice of the moving object is convolution processed using the filter kernel corresponding to the moving object. Specifically, the filter kernel corresponding to the moving object is placed on each pixel point of the image slice of the moving object in sequence, so that the center point of the filter kernel coincides with the pixel point, the value of each element in the filter kernel is multiplied by the pixel value of the pixel point located at the same position, and then all the products are added to obtain a value, and the pixel value of the pixel point is updated to the obtained value until all the pixel points of the image slice of the moving object are processed. After the image slice of the moving object is convoluted using the filter kernel corresponding to the moving object, the image slice of the moving object will appear blurred, thereby simulating an image blurred due to the movement of the moving object. The image slice of the moving object after the convolution processing using the filter kernel corresponding to the moving object is the image slice of the moving object with motion blur effect. The image slice of the moving object after the convolution processing using the filter kernel corresponding to the moving object is determined as the motion blurred image slice of the moving object. The image slice of the moving object is convolved with the filter kernel corresponding to the moving object, and the motion blurred image slice of the moving object can be expressed as: Among them, D i is the motion blurred image slice of the moving object, K i is the filter kernel corresponding to the moving object, C i is the image slice of the moving object, Represents a convolution operation.

[0064] Step 105: Overlay the motion blurred image slices of each moving object onto the detection frame image of each moving object in the clear image to be processed, and smooth the edge area of ​​the detection frame image of each moving object after covering to obtain a motion blurred image corresponding to the clear image to be processed.

[0065] Optionally, the motion blurred image corresponding to the clear image to be processed is a blurred clear image to be processed obtained by processing the clear image to be processed, that is, a motion blurred image blurred due to motion simulated based on the clear image to be processed.

[0066] Optionally, the motion blurred image slices of each moving object are overlaid on the detection frame image of each moving object in the clear image to be processed, and the edge areas of the detection frame images of each moving object after being overlaid are smoothed to obtain a motion blurred image corresponding to the clear image to be processed, including: for each moving object in the clear image to be processed, the pixel value of each pixel in the detection frame image of the moving object in the clear image to be processed is updated to the pixel value of the pixel at the same position in the motion blurred image slice of the moving object, thereby overlaying the motion blurred image slice of the moving object on the detection frame image of the moving object in the clear image to be processed; smoothing the edge areas of the detection frame images of each moving object after being overlaid in the clear image to be processed by a Gaussian smoothing algorithm; determining the clear image to be processed after smoothing as the motion blurred image corresponding to the clear image to be processed, and storing the motion blurred image in a training sample file.

[0067] Therefore, the image slices of each moving object with motion blur effect can be overlaid on the detection frame image of each moving object in the clear image to be processed, so as to obtain a motion blurred image which is blurred due to the movement of each moving object and is simulated based on the clear image to be processed, and the edge area of ​​the covered detection frame image is smoothed by a Gaussian smoothing algorithm, so as to finally obtain a smoother and more natural motion blurred image which is blurred due to the movement of each moving object.

[0068] Optionally, for each pixel point in the detection frame image of the moving object in the clear image to be processed, the pixel value of the pixel point can be updated to the pixel value of the pixel point located at the same position as the pixel point in the motion blurred image slice of the moving object, thereby overlaying the motion blurred image slice of the moving object on the detection frame image of the moving object in the clear image to be processed.

[0069] Optionally, the edge region of the detection frame image may refer to an image region located at the boundary of the detection frame image in the clear image to be processed. The edge region of the detection frame image of each moving object covered in the clear image to be processed may be smoothed by a Gaussian smoothing algorithm, thereby making the edge region of the detection frame image of each moving object covered in the clear image to be processed more stable and natural.

[0070] Optionally, the training sample file may be a pre-set file for storing training samples of an image processing model. The image processing model is used to perform deblurring processing on a motion blurred image.

[0071] The technical solution of the embodiment of the present invention obtains a clear image to be processed, performs target detection on the clear image to be processed, and determines the detection frame image position information and category labels of each object in the clear image to be processed; then, according to the detection frame image position information and category labels of each object, determines the moving objects in each object; according to the detection frame image position information of each moving object, determines the detection frame image of each moving object, copies the detection frame image of each moving object, and obtains the image slice of each moving object; for each moving object, randomly generates a filter kernel corresponding to the moving object, and uses the filter kernel corresponding to the moving object to perform convolution processing on the image slice of the moving object, and obtains the motion blurred image slice of each moving object; finally, overlays the motion blurred image slices of each moving object on the detection frame image of each moving object in the clear image to be processed, and smoothes the edge area of ​​the covered detection frame image of each moving object to obtain the motion blurred image corresponding to the clear image to be processed, thereby solving the problem of image motion blur processing in the related art. The solution cannot perform different processing on each moving object in the clear image, cannot effectively simulate the motion blurred image caused by the movement of each moving object, and cannot provide high-quality training samples for the image processing model. The moving objects in the clear image can be determined based on the detection frame image position information and category labels of each object in the clear image. The image slices of each moving object can be convolved based on the randomly generated filter kernel corresponding to the moving object to obtain image slices of each moving object with motion blur effect. The image slices of each moving object with motion blur effect are overlaid on the detection frame image of each moving object in the clear image, and the edge area of ​​the detection frame image is smoothed to obtain a smooth and natural motion blurred image caused by the movement of each moving object. Different processing can be performed on each moving object in the clear image, and the motion blurred image caused by the movement of each moving object can be effectively simulated, so as to provide high-quality training samples for the image processing model.

[0072] The technical solution of the embodiment of the present invention can effectively simulate the image motion blur caused by the relative motion of the target object and the camera in the actual acquisition process, and effectively simulate the motion blurred image caused by multiple moving objects moving in different directions and speeds. The motion simulation effect is more realistic and has practical value.

[0073] The technical solution of the embodiment of the present invention can be well applied to various images taken by mobile phones, tablets or cameras, and can meet the performance requirements of the business system of financial institutions for motion blurred image augmentation. Based on the technical solution of the embodiment of the present invention, a large number of clear images can be processed, and a large number of images with motion blur effects can be synthesized within an acceptable time, providing data support for the training process of the image processing model.

[0074] Embodiment 2

[0075] Figure 2 The flowchart of a method for image motion blur processing provided by the second embodiment of the present invention. The embodiment of the present invention can be combined with each optional solution in one or more of the above embodiments. Figure 2 As shown, the method includes:

[0076] Step 201: obtain a clear image to be processed, input the clear image to be processed into a pre-trained target detection model, and obtain the detection frame image position information and category label of each object in the clear image to be processed output by the target detection model.

[0077] Step 202: Determine the moving objects among the objects according to the detection frame image position information and category labels of the objects.

[0078] Step 203: determine the detection frame image of each moving object according to the detection frame image position information of each moving object, copy the detection frame image of each moving object, and obtain the image slice of each moving object.

[0079] Step 204: for each moving object, randomly generate a filter kernel corresponding to the moving object, and use the filter kernel corresponding to the moving object to perform convolution processing on the image slices of the moving object to obtain motion blurred image slices of each moving object.

[0080] Step 205: For each moving object in the clear image to be processed, the pixel value of each pixel point in the detection frame image of the moving object in the clear image to be processed is updated to the pixel value of the pixel point at the same position in the motion blurred image slice of the moving object, thereby overlaying the motion blurred image slice of the moving object on the detection frame image of the moving object in the clear image to be processed.

[0081] Step 206: Smoothing the edge areas of the covered detection frame images of the moving objects in the clear image to be processed by using a Gaussian smoothing algorithm.

[0082] Step 207: Determine the clear image to be processed after the smoothing process as a motion blurred image corresponding to the clear image to be processed, and store the motion blurred image in a training sample file.

[0083] The technical solution of the embodiment of the present invention can quickly determine the detection frame image position information and category labels of each object in the clear image to be processed based on a pre-trained target detection model, so as to facilitate the determination of each moving object in the clear image according to the detection frame image position information and category labels of each object in the clear image. The image slices of each moving object with motion blur effect can be overlaid on the detection frame image of each moving object in the clear image to be processed to obtain a motion blurred image that is blurred due to the movement of each moving object simulated based on the clear image to be processed, and the edge area of ​​the covered detection frame image is smoothed by a Gaussian smoothing algorithm to finally obtain a more stable and natural motion blurred image that is blurred due to the movement of each moving object.

[0084] Embodiment 3

[0085] Figure 3 This is a schematic diagram of the structure of an image motion blur processing device provided by Embodiment 3 of the present invention. The device can be configured in the business system of a financial institution. Figure 3 As shown, the device includes: a target detection module 301, a moving object determination module 302, a slice extraction module 303, a slice processing module 304 and a slice covering module 305.

[0086] Among them, the target detection module 301 is used to obtain a clear image to be processed, perform target detection on the clear image to be processed, and determine the detection frame image position information and category label of each object in the clear image to be processed; the moving object determination module 302 is used to determine the moving object in each object according to the detection frame image position information and category label of each object; the slice extraction module 303 is used to determine the detection frame image of each moving object according to the detection frame image position information of each moving object, copy the detection frame image of each moving object, and obtain the image slice of each moving object; the slice processing module 304 is used to randomly generate a filter kernel corresponding to the moving object for each moving object, and use the filter kernel corresponding to the moving object to perform convolution processing on the image slice of the moving object to obtain the motion blurred image slice of each moving object; the slice covering module 305 is used to cover the motion blurred image slice of each moving object on the detection frame image of each moving object in the clear image to be processed, and smooth the edge area of ​​the detection frame image of each moving object after covering, so as to obtain the motion blurred image corresponding to the clear image to be processed.

[0087] The technical solution of the embodiment of the present invention obtains a clear image to be processed, performs target detection on the clear image to be processed, and determines the detection frame image position information and category labels of each object in the clear image to be processed; then, according to the detection frame image position information and category labels of each object, determines the moving objects in each object; according to the detection frame image position information of each moving object, determines the detection frame image of each moving object, copies the detection frame image of each moving object, and obtains the image slice of each moving object; for each moving object, randomly generates a filter kernel corresponding to the moving object, and uses the filter kernel corresponding to the moving object to perform convolution processing on the image slice of the moving object, and obtains the motion blurred image slice of each moving object; finally, overlays the motion blurred image slices of each moving object on the detection frame image of each moving object in the clear image to be processed, and smoothes the edge area of ​​the covered detection frame image of each moving object to obtain the motion blurred image corresponding to the clear image to be processed, thereby solving the problem of image motion blur processing in the related art. The solution cannot perform different processing on each moving object in the clear image, cannot effectively simulate the motion blurred image caused by the movement of each moving object, and cannot provide high-quality training samples for the image processing model. The moving objects in the clear image can be determined based on the detection frame image position information and category labels of each object in the clear image. The image slices of each moving object can be convolved based on the randomly generated filter kernel corresponding to the moving object to obtain image slices of each moving object with motion blur effect. The image slices of each moving object with motion blur effect are overlaid on the detection frame image of each moving object in the clear image, and the edge area of ​​the detection frame image is smoothed to obtain a smooth and natural motion blurred image caused by the movement of each moving object. Different processing can be performed on each moving object in the clear image, and the motion blurred image caused by the movement of each moving object can be effectively simulated, so as to provide high-quality training samples for the image processing model.

[0088] In an optional implementation of an embodiment of the present invention, optionally, when the target detection module 301 performs target detection on the clear image to be processed and determines the detection box image position information and category labels of each object in the clear image to be processed, it is specifically used to: input the clear image to be processed into a pre-trained target detection model, and obtain the detection box image position information and category labels of each object in the clear image to be processed output by the target detection model.

[0089] In an optional implementation of an embodiment of the present invention, optionally, the moving object determination module 302 is specifically used to: perform the following operations for each object: detect whether there is a category label of the object in the stationary object label list; if it is detected that the category label of the object exists in the stationary object category label set, determine that the object is not a moving object.

[0090] In an optional implementation of an embodiment of the present invention, optionally, the moving object determination module 302 is specifically used to: if it is detected that the category label of the object does not exist in the stationary object category label set, then calculate the product of the width and the height in the detection frame image position information of the object to obtain the area of ​​the detection frame image of the object; calculate the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed, and determine whether the ratio is greater than a target value; if the ratio is greater than the target value, determine that the object is not a moving object; if the ratio is greater than or equal to the target value, determine that the object is a moving object.

[0091] In an optional implementation of an embodiment of the present invention, optionally, the slice processing module 304 is specifically used to: perform the following operations for each moving object: randomly generate an angle; determine the unit matrix corresponding to the moving object; perform an affine transformation on the unit matrix according to the angle, rotate the unit matrix by the angle, obtain a filter kernel corresponding to the moving object, and normalize the filter kernel; use the filter kernel to perform convolution processing on the image slice of the moving object to obtain a motion blurred image slice of the moving object.

[0092] In an optional implementation of an embodiment of the present invention, optionally, the slice covering module 305 is specifically used to: for each moving object in the clear image to be processed, update the pixel value of each pixel point in the detection frame image of the moving object in the clear image to be processed to the pixel value of the pixel point at the same position in the motion blurred image slice of the moving object, thereby covering the motion blurred image slice of the moving object on the detection frame image of the moving object in the clear image to be processed; smooth the edge area of ​​the covered detection frame image of each moving object in the clear image to be processed by a Gaussian smoothing algorithm; determine the clear image to be processed after smoothing as a motion blurred image corresponding to the clear image to be processed, and store the motion blurred image in a training sample file.

[0093] The image motion blur processing device provided in the embodiment of the present invention can execute the image motion blur processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0094] Embodiment 4

[0095] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement the image motion blur processing method of an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0096] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0097] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0098] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as an image motion blur processing method.

[0099] In some embodiments, the image motion blur processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the image motion blur processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the image motion blur processing method in any other appropriate manner (e.g., by means of firmware).

[0100] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or electronic device.

[0102] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein may be implemented on a heterogeneous hardware accelerator having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the heterogeneous hardware accelerator. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0104] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data electronic device), or a computing system that includes middleware components (e.g., an application electronic device), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0105] The computing system may include a client and an electronic device. The client and the electronic device are generally remote from each other and usually interact through a communication network. The relationship between the client and the electronic device is generated by computer programs running on corresponding computers and having a client-electronic device relationship with each other. The electronic device may be a cloud electronic device, also known as a cloud computing electronic device or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0106] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0107] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for processing image motion blur, characterized in that: include: Acquire a clear image to be processed, perform target detection on the clear image to be processed, and determine detection frame image position information and category labels of each object in the clear image to be processed; Determine the moving objects among the objects according to the detection frame image position information and category labels of the objects; Determine the detection frame image of each moving object according to the detection frame image position information of each moving object, copy the detection frame image of each moving object, and obtain the image slice of each moving object; For each moving object, a filter kernel corresponding to the moving object is randomly generated, and the image slices of the moving object are convolved using the filter kernel corresponding to the moving object to obtain motion blurred image slices of each moving object; The motion blurred image slices of each moving object are overlaid on the detection frame image of each moving object in the clear image to be processed, and the edge area of ​​the detection frame image of each moving object after overlay is smoothed to obtain a motion blurred image corresponding to the clear image to be processed.

2. The image motion blur processing method according to claim 1, characterized in that: Performing target detection on the clear image to be processed to determine the detection frame image position information and category label of each object in the clear image to be processed includes: The clear image to be processed is input into a pre-trained target detection model to obtain the detection frame image position information and category label of each object in the clear image to be processed output by the target detection model.

3. The image motion blur processing method according to claim 1, characterized in that: According to the detection frame image position information and category labels of each object, the moving objects in each object are determined, including: For each object, perform the following operations: Detect whether there is a category label of the object in the stationary object label list; If it is detected that the category label of the object exists in the stationary object category label set, it is determined that the object is not a moving object.

4. The image motion blur processing method according to claim 3, characterized in that: Determining the moving objects among the objects according to the detection frame image position information and category labels of the objects, further comprising: If it is detected that the category label of the object does not exist in the stationary object category label set, the product of the width and the height in the detection frame image position information of the object is calculated to obtain the area of ​​the detection frame image of the object; Calculating the ratio of the area of ​​the detection frame image of the object to the area of ​​the clear image to be processed, and determining whether the ratio is greater than a target value; If the ratio is greater than the target value, it is determined that the object is not a moving object; If the ratio is greater than or equal to the target value, it is determined that the object is a moving object.

5. The image motion blur processing method according to claim 1, characterized in that: For each moving object, a filter kernel corresponding to the moving object is randomly generated, and the image slices of the moving object are convolved using the filter kernel corresponding to the moving object to obtain motion blurred image slices of each moving object, including: For each moving object, perform the following operations: Generate a random angle; Determine the identity matrix corresponding to the moving object; Performing an affine transformation on the unit matrix according to the angle, rotating the unit matrix by the angle to obtain a filter kernel corresponding to the moving object, and normalizing the filter kernel; The image slices of the moving object are convolved using the filter kernel to obtain motion blurred image slices of the moving object.

6. The image motion blur processing method according to claim 1, characterized in that: Overlaying the motion blurred image slices of each moving object on the detection frame image of each moving object in the clear image to be processed, and performing smoothing on the edge area of ​​the covered detection frame image of each moving object to obtain a motion blurred image corresponding to the clear image to be processed, including: For each moving object in the clear image to be processed, the pixel value of each pixel point in the detection frame image of the moving object in the clear image to be processed is updated to the pixel value of the pixel point at the same position in the motion blurred image slice of the moving object, so that the motion blurred image slice of the moving object is overlaid on the detection frame image of the moving object in the clear image to be processed; Smoothing the edge areas of the detection frame images of each moving object covered in the clear image to be processed by a Gaussian smoothing algorithm; The clear image to be processed after smoothing is determined as a motion blurred image corresponding to the clear image to be processed, and the motion blurred image is stored in a training sample file.

7. An image motion blur processing device, characterized in that: include: The target detection module is used to obtain a clear image to be processed, perform target detection on the clear image to be processed, and determine the detection frame image position information and category label of each object in the clear image to be processed; A moving object determination module, used to determine the moving object among the objects according to the detection frame image position information and category labels of the objects; A slice extraction module is used to determine the detection frame image of each moving object according to the detection frame image position information of each moving object, and to copy the detection frame image of each moving object to obtain the image slice of each moving object; A slice processing module is used to randomly generate a filter kernel corresponding to each moving object, and use the filter kernel corresponding to the moving object to perform convolution processing on the image slice of the moving object to obtain motion blurred image slices of each moving object; The slice overlay module is used to overlay the motion blurred image slices of each moving object onto the detection frame image of each moving object in the clear image to be processed, and to smooth the edge area of ​​the detection frame image of each moving object after overlay to obtain a motion blurred image corresponding to the clear image to be processed.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image motion blur processing method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image motion blur processing method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the image motion blur processing method according to any one of claims 1 to 6 is implemented.

Images