Distorted image generation method, electronic device, and storage medium

Abstract

The application provides a distortion image generation method, an electronic device and a storage medium. The method comprises: acquiring an un-distorted image; acquiring a plurality of first pixel coordinates and corresponding first pixel values of the un-distorted image; selecting one first pixel coordinate as a distortion center coordinate from the plurality of first pixel coordinates; calculating the distance between the distortion center coordinate and each first pixel coordinate according to the distortion center coordinate and the plurality of first pixel coordinates; acquiring at least one distortion coefficient; calculating a second pixel coordinate according to the at least one distortion coefficient, the plurality of first pixel coordinates and the distance between the distortion center coordinate and each first pixel coordinate; taking the first pixel value as the second pixel value corresponding to each second pixel coordinate; and generating a distortion image of the un-distorted image based on the plurality of second pixel coordinates and the second pixel value corresponding to each second pixel coordinate. The application can acquire distortion images with different distortion degrees.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a distortion image generation method, an electronic device and a storage medium. BACKGROUND

[0002] In daily life, people often use a trained deep learning model to realize image recognition, segmentation and other functions. However, the images captured by a camera are distorted, and the degree of distortion varies, so that the trained deep learning model cannot accurately recognize or segment the obtained distorted image, resulting in a decrease in the accuracy of the deep learning model. SUMMARY

[0003] In view of the above, it is necessary to provide a distortion image generation method, an electronic device and a storage medium to solve the problem of decreased precision of a deep learning model caused by a distorted image.

[0004] The present application provides a distortion image generation method, which comprises: obtaining an undistorted image; obtaining a plurality of first pixel coordinates of the undistorted image and a first pixel value corresponding to each first pixel coordinate; and selecting one first pixel coordinate as a distortion center coordinate from the plurality of first pixel coordinates; calculating the distance between the distortion center coordinate and each first pixel coordinate according to the distortion center coordinate and the plurality of first pixel coordinates; obtaining at least one distortion coefficient, and calculating the second pixel coordinate corresponding to each first pixel coordinate according to the at least one distortion coefficient, the plurality of first pixel coordinates and the distance between the distortion center coordinate and each first pixel coordinate; taking the first pixel value corresponding to each first pixel coordinate as the second pixel value corresponding to each second pixel coordinate; and generating a distorted image of the undistorted image based on the plurality of second pixel coordinates and the second pixel value corresponding to each second pixel coordinate.

[0005] In an optional embodiment, the method further comprises training a deep learning model using the undistorted image and the distorted image.

[0006] In an optional embodiment, the method further comprises determining a distortion type according to the plurality of second pixel coordinates; and generating a distorted image corresponding to the distortion type based on the distortion type, the second pixel coordinates and the second pixel value corresponding to the second pixel coordinates.

[0007] In an optional embodiment, the distortion type includes a radial distortion type and a tangential distortion type.

[0008] In one optional implementation, calculating the distance between the distortion center coordinates and each of the first pixel coordinates based on the distortion center coordinates and the plurality of first pixel coordinates includes: according to the formula Calculate the distance between the coordinates of the distortion center and the coordinates of each of the first pixels; where r d This represents the distance between the coordinates of the distortion center and the coordinates of each first pixel, (x d ,y d (x0, y0) represents the coordinates of each first pixel, and (x0, y0) represents the coordinates of the distortion center.

[0009] In one optional implementation, obtaining at least one distortion coefficient and calculating a plurality of second pixel coordinates based on the at least one distortion coefficient, a plurality of first pixel coordinates, and the distance between the distortion center coordinates and each first pixel coordinate includes: according to the formula The coordinates of the plurality of second pixels are calculated; where, (x u ,y u (x) represents the coordinates of the second pixel. d ,y d ) represents the coordinates of the first pixel, r d This represents the distance between the coordinates of the distortion center and the coordinates of each first pixel, where k is a positive integer, λ1, λ2…λ k The distortion coefficient is denoted as .

[0010] In an optional implementation, the method further includes: according to the formula The coordinates of the plurality of second pixels are calculated; where, (x u ,y u (x) represents the coordinates of the second pixel. d ,y d ) represents the coordinates of the first pixel, r d μ1 and μ2 represent the distance between the distortion center coordinates and the coordinates of each first pixel, respectively, and are the distortion coefficients.

[0011] In an optional implementation, the method further includes: augmenting the undistorted image by performing data augmentation processing on the undistorted image, wherein the data augmentation processing includes one or more of flipping, rotating, scaling, and shifting the undistorted image.

[0012] This application also provides an electronic device, which includes a processor and a memory. The processor is used to execute a computer program stored in the memory to implement the distorted image generation method.

[0013] The embodiment of the present application further provides a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor to implement the distortion image generation method.

[0014] The distortion image generation method, the electronic device and the storage medium provided in the embodiment of the present application can generate distortion images with different distortion degrees and different types, and train a deep learning model by using the distortion images with different distortion degrees and different types, so that the deep learning model can recognize various distortion images, thereby improving the precision and robustness of the deep learning model. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A flowchart of a distortion image generation method provided in the embodiment of the present application.

[0016] Figure 2 A structural schematic diagram of an electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to more clearly understand the above objectives, features and advantages of the present application, the present application will be described in detail below with reference to the drawings and specific embodiments. It should be noted that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0018] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application. The described embodiments are merely some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0019] Hereinafter, the terms "first" and "second" are only used for description purposes, and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first" and "second" can explicitly or implicitly include one or more of the features. In the description of some embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance or illustration. Any embodiment or design solution described as "exemplary" or "for example" in some embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design solutions. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0021] Referring to Figure 1 as shown, Figure 1 A flow chart of a method for generating a distorted image is provided in embodiments of the present application. The method is applied in an electronic device (for example, an electronic device 2 as shown) and the electronic device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive Internet Protocol Television (IPTV), a smart wearable device, etc. Figure 2 The electronic device is a device that can automatically perform numerical calculation and / or information processing according to pre-set or stored instructions. The hardware of the electronic device includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, etc.

[0022] The electronic device can also include a network device and / or a user device. The network device includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on Cloud Computing.

[0023] The method specifically includes the following steps:

[0024] The method specifically includes the following steps:

[0025] 101, the electronic device acquires an undistorted image.

[0026] In at least one embodiment of the present application, the undistorted image includes an undistorted image or an image after distortion correction. The image includes, but is not limited to, a road scene image, an industrial product image. For example, the undistorted image can include an undistorted image of the field of view in front of a vehicle, an undistorted image of a plastic product.

[0027] In at least one embodiment of the present application, the obtaining the non-distorted image comprises: taking an image of a road scene or an industrial product by a camera after distortion correction as the non-distorted image. In this embodiment, the distorted image can also be de-distorted to obtain the non-distorted image.

[0028] In at least one embodiment of the present application, the method further comprises: obtaining more non-identical non-distorted images by performing a data enhancement operation on the non-distorted image, the data enhancement operation comprising, but not limited to, one or more of the following processing methods: flipping the image, rotating the image, scaling the image, and cropping the image. The data enhancement operation can effectively expand the non-distorted image, thereby expanding the number of training samples.

[0029] 102, obtaining a plurality of first pixel coordinates of the non-distorted image and a first pixel value corresponding to each first pixel coordinate, and selecting one first pixel coordinate from the plurality of first pixel coordinates as a distortion center coordinate.

[0030] In at least one embodiment of the present application, the obtaining a plurality of first pixel coordinates of the non-distorted image comprises: obtaining a plurality of first pixel coordinates of the non-distorted image based on an OpenCV method. That is, all pixel coordinates in the non-distorted image are obtained.

[0031] In at least one embodiment of the present application, the obtaining a first pixel value corresponding to each first pixel coordinate comprises:

[0032] The first pixel value corresponding to each first pixel coordinate of the non-distorted image is obtained based on an OpenCV method. In this embodiment, the first pixel value is represented in the RGB manner. For example, the first pixel coordinate is (1750, 160), and the corresponding first pixel value is (113, 65, 79), that is, R: 113, G: 65, and B: 79.

[0033] In at least one embodiment of the present application, the selecting one first pixel coordinate from the plurality of first pixel coordinates as a distortion center coordinate comprises:

[0034] Any one of the first pixel coordinates is selected as the distortion center coordinate from all the first pixel coordinates. The distortion center is a reference center of the distortion amount at different fields of view. The farther the field of view is from the distortion center, the larger the corresponding distortion amount is. That is, the farther the first pixel coordinate is from the distortion center, the larger the distortion amount at the first pixel coordinate is.

[0035] It can be understood that all the first pixel coordinates can be the distortion center, and all the first pixel coordinates are selected as the distortion center once to generate more distorted images.

[0036] 103. Calculate the distance between the distortion center coordinate and each of the first pixel coordinates according to the distortion center coordinate and the first pixel coordinates.

[0037] In at least one embodiment of the present application, the calculation of the distance between the distortion center coordinate and each of the first pixel coordinates according to the distortion center coordinate and the first pixel coordinates comprises:

[0038] According to formula ①:

[0039]

[0040] Calculate the distance between the distortion center coordinate and each of the first pixel coordinates.

[0041] Wherein, r d represents the distance between the distortion center coordinate and each of the first pixel coordinates, (x d ,y d ) represents the first pixel coordinate, and (x0, y0) represents the distortion center coordinate.

[0042] Specifically, each of the first pixel coordinates and the distortion center coordinate can be substituted into the above formula ①, and the distance between the distortion center and each of the first pixel coordinates can be calculated through the above formula ①.

[0043] 104. Obtain at least one distortion coefficient, and calculate the second pixel coordinates corresponding to each of the first pixel coordinates according to the at least one distortion coefficient, the first pixel coordinates, and the distance between the distortion center coordinate and each of the first pixel coordinates.

[0044] In at least one embodiment of the present application, the distortion coefficient is used to change the distortion degree of the distorted image, and the distortion coefficient can be obtained from the corresponding camera.

[0045] In at least one embodiment of the present application, the calculation of the second pixel coordinates according to the at least one distortion coefficient, the first pixel coordinates, and the distance between the distortion center coordinate and each of the first pixel coordinates comprises:

[0046] According to formula ②:

[0047]

[0048] Calculate the second pixel coordinates.

[0049] Wherein, (x u ,y u ) represents the second pixel coordinate, (x d ,y d ) represents the first pixel coordinate, and r dr k is a distortion coefficient. In the embodiment, the greater the value of k, the greater the degree of distortion.

[0050] Specifically, each first pixel coordinate, the distance from the distortion center to the first pixel coordinate, the distortion coefficient and the value of k are substituted into the above formula 2, and the second pixel coordinate can be calculated by the above formula 2.

[0051] In at least one embodiment of the present application, the method for calculating the plurality of second pixel coordinates further comprises:

[0052] According to formula 3:

[0053]

[0054] The plurality of second pixel coordinates are calculated; wherein (x u ,y u ) represents the second pixel coordinate, (x d ,y f ) represents the first pixel coordinate, r d represents the distance from the distortion center to each first pixel coordinate, and μ1, μ2 is a distortion coefficient.

[0055] Specifically, each first pixel coordinate, the distance from the distortion center to the first pixel coordinate, and the distortion coefficient are substituted into the above formula 3, and the second pixel coordinate can be calculated by the above formula 3.

[0056] 105, the first pixel value corresponding to each first pixel coordinate is taken as the second pixel value corresponding to each second pixel coordinate.

[0057] In at least one embodiment of the present application, the first pixel value corresponding to each first pixel coordinate is taken as the second pixel value corresponding to each second pixel coordinate. For example, a first pixel coordinate is (1750, 160), and the corresponding first pixel value is (113, 65, 79). The second pixel coordinate calculated by the above formula 2 or formula 3 is (1230, 132), and the second pixel value of the second pixel coordinate is determined as (113, 65, 79).

[0058] Step 106, generating a distortion image of the non-distortion image based on the plurality of second pixel coordinates and the second pixel value corresponding to each second pixel coordinate.

[0059] In at least one embodiment of the present application, the distortion image of the non-distortion image is generated based on the plurality of second pixel coordinates and the second pixel value corresponding to each second pixel coordinate.

[0060] In at least one embodiment of the present application, the method further comprises: determining a distortion type according to the plurality of second pixel coordinates; and generating a distortion image corresponding to the distortion type based on the distortion type, the second pixel coordinates, and the plurality of second pixel values.

[0061] Specifically, the distortion type comprises a radial distortion type and a tangential distortion type.

[0062] In the present embodiment, the determination of the distortion type according to the plurality of second pixel coordinates comprises:

[0063] The second coordinates are calculated by using the above formula ②, and the distortion type is determined to be a radial distortion type according to the calculation of the second coordinates by using the formula ②. The second coordinates are calculated by using the above formula ③, and the distortion type is determined to be a tangential distortion type according to the calculation of the second coordinates by using the formula ③.

[0064] In at least one embodiment of the present application, the method further comprises: training a deep learning model using the non-distorted image and the distortion image.

[0065] Specifically, the distortion image comprises a distortion image of different distortion degrees and a distortion image of different distortion types generated by using the above method. The deep learning model can also be any one of AlexNet, VGGNet, GoogLeNet, ResNet, DenseNet, SSDNet, RCNN, YOLO series, FCN, SegNet, etc. The type of the deep learning model is not specifically limited in the present application.

[0066] Further, for example, the deep learning model adopts a ResNet model for classifying objects in an image. The robustness of the ResNet model can be improved, and the accuracy of object classification can be improved by inputting the distortion image into the ResNet model for training. Illustratively, the deep learning model adopts a RCNN model for detecting whether a target object is contained in an image. The robustness of the RCNN model can be improved, and the accuracy of object recognition can be improved by inputting the distortion image into the RCNN model for training. Further illustratively, the deep learning model adopts a FCN model for segmenting a target object in an image. The robustness of the FCN model can be improved, and the accuracy of segmentation can be improved by inputting the distortion image into the FCN model for training.

[0067] As can be seen from the above technical solutions, the present application generates images of different distortion degrees and distortion images of different distortion types, and trains a deep learning model using different types of distortion images, different distortion degree images, and the non-distorted image, so that the deep learning model can recognize different distortion images, thereby improving the accuracy and robustness of the deep learning model.

[0068] It is understandable that electronic devices acquire images through cameras and input these images into trained deep learning models to perform operations such as image recognition and segmentation. However, the acquired images may contain various distortions due to insufficient camera correction, preventing the deep learning model from accurately recognizing and segmenting the images, thus reducing the accuracy of the trained deep learning model. This application generates images with different degrees of distortion and distorted images of different types, and trains a deep learning model using undistorted images to obtain a trained deep learning model. This allows the trained deep learning model to recognize different distorted images without considering whether the camera's image correction is sufficient. In short, regardless of whether the image captured by the camera is distorted, the trained deep learning model can perform image recognition and segmentation operations, thereby reducing dependence on camera correction and improving the accuracy and robustness of the deep learning model.

[0069] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. For those skilled in the art, improvements can be made without departing from the inventive concept of this application, but these all fall within the scope of protection of this application.

[0070] like Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 2 includes a memory 201, at least one processor 202, a computer program 203 stored in the memory 201 and executable on the at least one processor 202, and at least one communication bus 204.

[0071] Those skilled in the art will understand that Figure 2 The schematic diagram shown is merely an example of the electronic device 2 and does not constitute a limitation on the electronic device 2. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device 2 may also include input / output devices, network access devices, etc.

[0072] The at least one processor 202 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The at least one processor 202 can be a microprocessor or the at least one processor 202 can also be any conventional processor, etc. The at least one processor 202 is a control center of the electronic device 2, and connects all parts of the electronic device 2 through various interfaces and lines.

[0073] The memory 201 can be used to store the computer program 203, and the at least one processor 202 realizes various functions of the electronic device 2 by running or executing the computer program 203 stored in the memory 201, and calling data stored in the memory 201. The memory 201 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data (such as audio data) created according to the use of the electronic device 2, etc. In addition, the memory 201 can include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device.

[0074] The modules / units integrated in the electronic device 2, if implemented in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be implemented by a computer program instructing related hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory and read-only memory (ROM).

[0075] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims to which they relate.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method of generating a distorted image, characterized by, The distortion image generation method comprises: obtaining an un-distorted image; obtaining a plurality of first pixel coordinates of the un-distorted image and a first pixel value corresponding to each first pixel coordinate; and selecting a first pixel coordinate as a distortion center coordinate from the plurality of first pixel coordinates, comprising: selecting any one pixel coordinate from all first pixel coordinates as a distortion center coordinate; the distortion center is a reference center of the size of the distortion variable at different fields of view, and the farther the field of view is from the distortion center, the larger the corresponding distortion variable is; calculating the distance between the distortion center coordinate and each first pixel coordinate according to the distortion center coordinate and the plurality of first pixel coordinates; obtaining at least one distortion coefficient, and calculating a second pixel coordinate corresponding to each first pixel coordinate according to the at least one distortion coefficient, the plurality of first pixel coordinates and the distance between the distortion center coordinate and each first pixel coordinate; taking the first pixel value corresponding to each first pixel coordinate as a second pixel value corresponding to each second pixel coordinate; generating a distortion image of the un-distorted image based on the plurality of second pixel coordinates and the second pixel value corresponding to each second pixel coordinate.

2. The distorted image generation method according to claim 1, characterized by, The method further comprises: training a deep learning model using the un-distorted image and the distortion image.

3. The distorted image generation method according to claim 1, characterized by, The method further comprises: determining a distortion type according to the plurality of second pixel coordinates; generating a distortion image corresponding to the distortion type based on the distortion type, the second pixel coordinates and the second pixel value corresponding to the second pixel coordinates.

4. The distorted image generation method according to claim 3, characterized by, The distortion type comprises a radial distortion type and a tangential distortion type.

5. The distorted image generation method according to claim 1, characterized by, The calculating the distance between the distortion center coordinate and each first pixel coordinate according to the distortion center coordinate and the plurality of first pixel coordinates comprises: According to the formula calculating a distance between the distortion center coordinate and each of the first pixel coordinates; wherein denotes the distance of the distortion center coordinate from each first pixel coordinate, denotes each first pixel coordinate, denotes the distortion center coordinate.

6. The distorted image generation method according to claim 1, characterized by, The obtaining at least one distortion coefficient, and calculating a plurality of second pixel coordinates according to the at least one distortion coefficient, the plurality of first pixel coordinates and the distance between the distortion center coordinate and each first pixel coordinate comprises: The plurality of second pixel coordinates are calculated according to the formula ​ wherein, denotes a second pixel coordinate, denotes a first pixel coordinate, denotes a distance of the distortion center coordinate to each first pixel coordinate, is a positive integer, , … is the distortion coefficient.

7. The distorted image generation method according to claim 5, characterized by, The method further comprises: The plurality of second pixel coordinates are calculated according to the formula ​ wherein, denotes a second pixel coordinate, denotes a first pixel coordinate, denotes the distance of the distortion center coordinate from each first pixel coordinate, , is a distortion coefficient.

8. The distorted image generation method of claim 1, wherein, The method further comprises: expanding the un-distorted image by performing data enhancement processing on the un-distorted image, wherein the data enhancement processing comprises one or more of flipping, rotating, scaling, and shifting processing on the un-distorted image.

9. An electronic device, comprising: The electronic device comprises a processor and a memory, and the processor is configured to execute a computer program stored in the memory to implement the distortion image generation method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, and the at least one instruction is executed by the processor to implement the distortion image generation method according to any one of claims 1 to 8.

Images