Fisheye image correction method, device, electronic device and storage medium

Abstract

The present invention discloses a fisheye image correction method, device, electronic device and storage medium. The method comprises: obtaining a fisheye image taken by a fisheye camera; processing the fisheye image to obtain the gradient amplitude of the fisheye image; determining the edge points of the fisheye image based on the gradient amplitude; wherein the edge points include reflector edge points and fisheye edge points; the edge points are used to characterize the pixel points whose gradient amplitude is greater than a preset threshold; performing circle fitting on the edge points to obtain edge point fitting results; determining the projection function of the fisheye image based on the edge point fitting results, and using the projection function to correct the fisheye image to obtain a target fisheye image. The present technical solution can directly correct the fisheye image without prior labeling, thus reducing the workload.

Description

Technical Field

[0001] The present invention relates to the technical field of fisheye correction, and in particular to a fisheye image correction method, device, electronic equipment and storage medium. Background Art

[0002] Fisheye cameras are panoramic cameras that can independently monitor a wide range of objects without blind spots. Due to the large field of view of fisheye cameras, the edge distortion of the captured images is large, which directly affects the imaging effect. In actual application scenarios, it is often necessary to correct the captured images.

[0003] Currently, in the commonly used fisheye image correction technology, it is necessary to know the angle and distance of the camera shooting in advance; or, it is necessary to know the incident angle of the camera in advance and require a pre-confirmed pixel point as a marker to correct the fisheye image.

[0004] In the fisheye processing method in the known technology, it is necessary to know the shooting information of the camera in advance, which leads to a large workload of correction. Summary of the invention

[0005] The present invention provides a fisheye image correction method, device, electronic equipment and storage medium, which can directly correct the fisheye image without prior marking, thereby reducing the workload.

[0006] According to one aspect of the present invention, a fisheye image correction method is provided, the method comprising:

[0007] Get the fisheye image taken by the fisheye camera;

[0008] Processing the fisheye image to obtain a gradient amplitude of the fisheye image;

[0009] Based on the gradient amplitude, edge points of the fisheye image are determined; wherein the edge points include reflector edge points and fisheye edge points; the edge points are used to characterize pixel points whose gradient amplitude is greater than a preset threshold;

[0010] Performing circle fitting on the edge points to obtain edge point fitting results;

[0011] The projection function of the fisheye image is determined according to the edge point fitting result, and the fisheye image is corrected using the projection function to obtain a target fisheye image.

[0012] According to another aspect of the present invention, a fisheye image correction device is provided, the device comprising:

[0013] A fisheye image acquisition module is used to acquire fisheye images taken by a fisheye camera;

[0014] A gradient amplitude obtaining module, used for processing the fisheye image to obtain the gradient amplitude of the fisheye image;

[0015] An edge point determination module, used to determine the edge points of the fisheye image based on the gradient amplitude; wherein the edge points include reflector edge points and fisheye edge points; the edge points are used to characterize the pixel points whose gradient amplitude is greater than a preset threshold;

[0016] An edge point fitting result obtaining module is used to perform circle fitting on the edge points to obtain edge point fitting results;

[0017] The target fisheye image obtaining module is used to determine the projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain the target fisheye image.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a fisheye image correction method described in any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a fisheye image correction method described in any embodiment of the present invention when executed.

[0023] The technical solution of the embodiment of the present invention is to obtain a fisheye image taken by a fisheye camera; process the fisheye image to obtain the gradient amplitude of the fisheye image; determine the edge points of the fisheye image based on the gradient amplitude; perform circle fitting on the edge points to obtain the edge point fitting result; determine the projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain the target fisheye image. This technical solution can directly correct the fisheye image without prior labeling, thereby reducing the workload.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a fisheye image correction method provided according to Embodiment 1 of the present invention;

[0027] Figure 2 is a schematic diagram of a fisheye camera provided in Example 1 of the present application;

[0028] Figure 3 is a schematic diagram of a reflector in a fisheye camera provided in Example 1 of the present application;

[0029] Figure 4 is a schematic diagram of the gradient direction and edge direction provided in Example 1 of the present application;

[0030] Figure 5 is a schematic diagram of the edge point fitting result provided in Example 1 of the present application;

[0031] Figure 6 is a schematic diagram of fisheye lens imaging provided in Example 1 of the present application;

[0032] Figure 7 is a schematic diagram of a fisheye lens imaging lens assembly provided in Example 1 of the present application;

[0033] Figure 8 is a schematic diagram of a target fisheye image provided in Example 1 of the present application;

[0034] Fig. 9 A flow chart of a fisheye image correction process provided by Embodiment 2 of the present invention;

[0035] Fig.10 is a schematic diagram of a process for determining edge points to be processed provided in Embodiment 2 of the present application;

[0036] Fig.11 is a schematic diagram of the convex hull of edge points to be processed provided in Example 2 of the present application;

[0037] Fig.12 is a schematic diagram of edge points provided in Example 2 of the present application;

[0038] Fig.13 is a structural schematic diagram of a fisheye image correction device provided according to Embodiment 3 of the present invention;

[0039] Fig.14The present invention is a schematic diagram of the structure of an electronic device for implementing a fisheye image correction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] Embodiment 1

[0043] Figure 1 1 is a flow chart of a fisheye image correction method provided according to the first embodiment of the present invention. This embodiment is applicable to the case of correcting fisheye images. The method can be executed by a fisheye image correction method device. The fisheye image correction device can be implemented in the form of hardware and / or software. The fisheye image correction device can be configured in an intelligent terminal for image processing. Figure 1 As shown, the method includes:

[0044] S110: Obtain a fisheye image captured by a fisheye camera.

[0045] The fisheye image may refer to an image of piano keys during class in an online piano education scenario, and may be captured by a fisheye camera in the online piano education scenario.

[0046] For example, Figure 2 is a schematic diagram of a fisheye camera provided in Example 1 of the present application, such as Figure 2 As shown, in the online piano education scenario, fisheye photography is achieved by connecting a normal pad camera to a fisheye conversion lens.

[0047] For example, Figure 3 Schematic diagram of a reflector in a fisheye camera provided in Example 1 of the present application. Figure 3 As shown, there is a reflector in front of the fisheye camera for photographing the piano keys during class. Fisheye correction can be achieved by flattening the bottom edge of the reflector.

[0048] S120: Process the fisheye image to obtain a gradient amplitude of the fisheye image.

[0049] In this embodiment, the fisheye image may be processed using image processing technology to obtain the gradient amplitude of the fisheye image.

[0050] In the technical solution, optionally, processing the fisheye image to obtain the gradient amplitude of the fisheye image includes:

[0051] Based on a predetermined calculation formula, according to the pixel difference between adjacent pixels in the fisheye image, determine the gradient amplitude and gradient direction of the fisheye image;

[0052] If there is a gradient magnitude in the same gradient direction, the gradient magnitude is used as the gradient magnitude of the fisheye image;

[0053] If there are at least two gradient magnitudes in the same gradient direction, the maximum gradient magnitude of the at least two gradient magnitudes is used as the gradient magnitude of the fisheye image.

[0054] In this embodiment, since the fisheye image is a discrete set of pixels, the first-order derivative can be approximately represented by the intensity difference between adjacent pixels in the fisheye image: Then the gradient amplitude of the fisheye image can be determined as The gradient direction is

[0055] For example, Figure 4 is a schematic diagram of the gradient direction and edge direction provided in the first embodiment of the present application, such as Figure 4 As shown in (a), under normal circumstances, the edge should be a thin line and should not be too thick. If it is too thick, the edge will lose its meaning. Therefore, if there are multiple relatively large gradient amplitudes near a gradient direction, the gradient amplitudes need to be processed, and the pixel points corresponding to the largest gradient amplitude are retained, and the other pixel points are discarded. Figure 4 As shown in (b), for a pixel point, the middle line is the gradient direction. If point A is a local maximum point in a gradient direction, then the gradient of point A should be greater than the gradients of points d1 and d2. Given the gradients of t1 and t2, the gradient strength of d1 can be calculated by linear interpolation. Assuming the gradient direction is w, then: A similar method can be used to find the value of d2. If the gradient of point A is larger than that of both d1 and d2, then point A is the pixel selected in this step, otherwise it is not.

[0056] By determining the gradient amplitude and gradient direction, the edge points of the fisheye image can be obtained, which is conducive to the correction of the fisheye image.

[0057] S130. Determine edge points of the fisheye image based on the gradient amplitude; wherein the edge points include reflector edge points and fisheye edge points; and the edge points are used to characterize pixel points whose gradient amplitude is greater than a preset threshold.

[0058] In this scheme, the edge points of an image usually refer to places where the intensity of image pixel points changes greatly. The gradient amplitude can be screened by setting a threshold to obtain the gradient amplitude that meets the threshold condition, and the pixel points corresponding to the gradient amplitude are used as the edge points of the fisheye image.

[0059] S140: Perform circle fitting on the edge points to obtain edge point fitting results.

[0060] The edge point fitting results include the reflector fitting circle and the fisheye fitting circle.

[0061] In this embodiment, a predetermined fitting calculation formula may be used to perform circle fitting on edge points to obtain an edge fitting result.

[0062] In the technical solution, optionally, performing circle fitting on the edge points to obtain edge point fitting results includes:

[0063] Calculate the distance between the edge point and the preset center of the circle to obtain the target distance;

[0064] Determine a loss function using the target distance and a preset radius;

[0065] The loss function is fitted to obtain an edge point fitting result.

[0066] Specifically, according to the circle formula (xA) 2 +(yB) 2 =R 2 We can get:

[0067] R 2 =x 2 -2Ax+A 2 +y 2 -2By+B 2 ;

[0068] Let: a = -2A, b = -2B, c = A 2 +B 2 -R2 This gives another form of the equation of the circle:

[0069] x 2 +y 2 +ax+by+c=0;

[0070] For any set of edge points (X i ,Y i ),i∈(1,2...n), the distance from the edge point to the center of the circle is:

[0071] d i 2 =(X i -A) 2 +(Y i -B) 2 ;

[0072] The difference between the distance from the edge point to the center of the circle and the square of the radius is:

[0073]

[0074] The fitting goal is to make the difference between the distance from the edge point to the center of the circle and the radius as small as possible, so a loss function is defined

[0075] Since Q(a,b,c) is the sum of squares, there must be a minimum value greater than or equal to 0, and no maximum value. The minimum point can be found by taking the partial derivative of Q(a,b,c) with respect to a,b,c:

[0076]

[0077]

[0078]

[0079] Rewritten as a system of equations:

[0080]

[0081] This is a system of three linear equations about a, b, and c. Solving this system of equations gives:

[0082]

[0083] Where: n is the number of points to be fitted, C, D, E, G, H are defined as follows

[0084]

[0085] After finding a, b, c, substitute a=-2A, b=-2B, c=A 2 +B 2-R 2 available:

[0086]

[0087] The center and radius of the edge point fitting result can be calculated.

[0088] For example, Figure 5 is a schematic diagram of the edge point fitting result provided in Example 1 of the present application, such as Figure 5 As shown in the figure, the smaller circle is the fisheye edge point fitting result, and the larger circle is the reflector fitting result. Since the reflector fitting result is larger, only part of the area is shown in the figure.

[0089] By fitting the edge points, the edge point fitting results can be obtained, and the fisheye image can be directly corrected without prior labeling, thus reducing the workload.

[0090] S150 . Determine a projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain a target fisheye image.

[0091] For example, Figure 6 is a schematic diagram of fisheye lens imaging provided in Example 1 of the present application, such as Figure 6 As shown, a fisheye lens is generally composed of a dozen different lenses. During the imaging process, the incident light undergoes different degrees of refraction and is projected onto an imaging plane of limited size, which makes the fisheye lens have a larger field of view than ordinary lenses.

[0092] For example, Figure 7 Schematic diagram of a fisheye lens imaging lens assembly provided in the first embodiment of the present application. Figure 7 As shown in Figure 2, when studying fisheye camera imaging, the lens group can be simplified into a spherical surface, O1-X c Y c Z c is the camera coordinate system, O2-xy is the imaging plane. There is a point P in the real world, with an incident angle of θ. If we follow the pinhole camera model of an ordinary camera, the incident light PO1 does not change its path after passing through the lens, and the three points P, O1, and p′ are collinear, and p′ is the image point of P; but for a fisheye camera, the incident light PO1 will be refracted after passing through the lens, so the image point of P is point p, and the polar coordinates are expressed as The actual lens does not exactly conform to the projection model for various reasons. The first five terms of the Taylor expansion about θ can be taken to approximate the actual projection function of the fisheye lens: r(θ)≈k0θ+k1θ 3 +k2θ 5 +k3θ 7 +k4θ 9 .

[0093] In this embodiment, after obtaining the actual projection function, the actual projection function can be fitted based on the edge point fitting result to obtain the projection function of the fisheye image, and the fisheye image can be corrected using the projection function to obtain the target fisheye image.

[0094] In the technical solution, optionally, determining the projection function of the fisheye image according to the edge point fitting result includes:

[0095] The edge point fitting result is substituted into a preset projection function formula to calculate the projection function of the fisheye image.

[0096] In this embodiment, since the upper edge of the reflector in the lens must be a straight line, and the center of the fisheye lens can be considered to be the center of the fitting circle of the fisheye edge point, the edge point fitting result is brought into the actual projection function, the actual projection function is solved, and the projection function is obtained. By fitting the point on the circle in the horizontal direction to be a straight line, the target fisheye image is obtained.

[0097] For example, Figure 8 is a schematic diagram of a target fisheye image provided in Example 1 of the present application. Figure 8 As shown, by correcting the fisheye image, a target fisheye image with better quality is obtained.

[0098] By determining the projection function, the fisheye image can be corrected directly based on the projection function without prior labeling, thus reducing the workload.

[0099] The technical solution of the embodiment of the present invention is to obtain a fisheye image taken by a fisheye camera; process the fisheye image to obtain the gradient amplitude of the fisheye image; determine the edge points of the fisheye image based on the gradient amplitude; perform circle fitting on the edge points to obtain the edge point fitting result; determine the projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain the target fisheye image. By executing this technical solution, the fisheye image can be directly corrected without prior labeling, which reduces the workload.

[0100] Embodiment 2

[0101] Fig. 9 This is a flow chart of a fisheye image correction process provided by the second embodiment of the present invention. The relationship between this embodiment and the above embodiment is a detailed description of edge point determination. Fig. 9 As shown, the method includes:

[0102] S910. Obtain a fisheye image captured by a fisheye camera.

[0103] S920: Process the fisheye image to obtain a gradient amplitude of the fisheye image.

[0104] S930: If the gradient amplitude satisfies a preset threshold condition, taking the pixel point corresponding to the gradient amplitude as the edge point to be processed of the fisheye image.

[0105] The preset threshold condition may refer to that the gradient amplitude is greater than the preset threshold, and the preset threshold may be set according to the requirements of the edge point to be processed. That is, if the gradient amplitude is greater than the preset threshold, the gradient amplitude satisfies the preset threshold condition; if the gradient amplitude is less than the preset threshold, the gradient amplitude does not satisfy the preset threshold condition.

[0106] In the technical solution, optionally, if the gradient amplitude satisfies a preset threshold condition, the pixel point corresponding to the gradient amplitude is used as the edge point to be processed of the fisheye image, including:

[0107] If the gradient amplitude is greater than a preset first threshold, taking the pixel point corresponding to the gradient amplitude as the edge point to be processed of the fisheye image;

[0108] If the gradient amplitude is less than a preset first threshold and greater than a preset second threshold, the adjacent pixel points of the pixel point corresponding to the gradient amplitude are determined, and it is judged whether the gradient amplitude of the adjacent pixel points is greater than the preset first threshold. If it is greater, the adjacent pixel points are used as edge points to be processed of the fisheye image.

[0109] The preset first threshold and the preset second threshold can be set according to the requirements for determining the edge points to be processed. The first threshold is greater than the second threshold. Preferably, the first threshold is 2 or 3 times the second threshold.

[0110] In this scheme, if the gradient amplitude is greater than the preset first threshold, it is retained; if the gradient amplitude is less than the preset second threshold, it is deleted; if the gradient amplitude is less than the preset first threshold and greater than the preset second threshold, the gradient amplitude is searched from the 8-neighborhood of the pixel point corresponding to the gradient amplitude. If there is a gradient amplitude greater than the first threshold, it is retained; if not, it is discarded.

[0111] By determining the edge points to be processed, the fisheye camera can be calibrated, reducing the workload.

[0112] For example, Fig.10 is a schematic diagram of the process of determining edge points to be processed provided in the second embodiment of the present application, such as Fig.10 As shown, a fisheye image is obtained; a gradient calculation is performed on the fisheye image to obtain a gradient amplitude; non-maximum suppression is performed on the gradient amplitude; and then the gradient amplitude is screened based on a preset threshold to obtain edge points to be processed.

[0113] S940: Determine edge points of the fisheye image according to angles between edge points to be processed.

[0114] In this scheme, a point can be selected from each edge point to be processed as the first edge point, and then the angles from other edge points to be processed to the first edge point are calculated respectively, so as to determine the second edge point, and then traverse is performed to calculate the angles from other edge points to be processed to the second edge point, and the remaining edge points are determined in turn.

[0115] In the technical solution, optionally, determining the edge points of the fisheye image according to the angles between the edge points to be processed includes:

[0116] If the edge point of the fisheye image is the first edge point, selecting the edge point to be processed with the smallest horizontal and vertical coordinates from the edge points to be processed as the edge point of the fisheye image;

[0117] If the edge point of the fisheye image is not the first edge point, the angles between other edge points to be processed and the previously determined edge point of the fisheye image are calculated, and the edge point to be processed with the smallest angle value is used as the edge point of the fisheye image.

[0118] In this scheme, in a real vector space V, for a given set X, the intersection S of all convex sets containing X is called the convex hull of X. The convex hull of X can be constructed by the convex combination of all points (X1, ...Xn) in X. The edge points of the fisheye image can be determined by finding the convex hull of the edge points to be processed.

[0119] For example, Fig.11 is a schematic diagram of the convex hull of edge points to be processed provided in the second embodiment of the present application, such as Fig.11 As shown, first select an edge point with the smallest horizontal coordinate to be processed. If there are multiple such edge points to be processed, select the edge point with the smallest vertical coordinate to be processed as the edge point of the fisheye image, that is, the first edge point. Then traverse the remaining edge points to be processed, and calculate the angles between the remaining edge points to be processed and the edge points in a clockwise direction. Select the point with the smallest angle and add it to a point set. If there are multiple edge points to be processed with the same angle, calculate the distance from the edge point to these edge points to be processed, select the point with the largest distance to add to the point set (select point 10 for points 9 and 10), and repeat the above steps until the point to be added to the convex hull is the first edge point and the loop ends. At this time, the point set obtained is the convex hull.

[0120] For example, Fig.12 is a schematic diagram of edge points provided in the second embodiment of the present application, such as Fig.12 As shown, the edge point of the reflector is in the upper half of the fisheye image, and the edge point of the fisheye is in the lower half of the image.

[0121] By determining the edge points, the projection function can be determined based on the edge points, thereby achieving the correction of the fisheye camera and reducing the workload.

[0122] S950: Perform circle fitting on the edge points to obtain edge point fitting results.

[0123] S960: Determine a projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain a target fisheye image.

[0124] The technical solution of the embodiment of the present invention is to obtain a fisheye image taken by a fisheye camera; process the fisheye image to obtain the gradient amplitude of the fisheye image; if the gradient amplitude meets the preset threshold condition, the pixel points corresponding to the gradient amplitude are used as the edge points to be processed of the fisheye image; and the edge points of the fisheye image are determined according to the angles between the edge points to be processed; circle fitting is performed on the edge points to obtain the edge point fitting results; the projection function of the fisheye image is determined according to the edge point fitting results, and the fisheye image is corrected using the projection function to obtain the target fisheye image. By executing this technical solution, the fisheye image can be directly corrected without prior labeling, which reduces the workload.

[0125] Embodiment 3

[0126] Fig.13 FIG. 1 is a schematic diagram of the structure of a fisheye image correction device provided in Embodiment 3 of the present invention. Fig.13 As shown, the device comprises:

[0127] The fisheye image acquisition module 1310 is used to acquire the fisheye image taken by the fisheye camera;

[0128] A gradient amplitude obtaining module 1320 is used to process the fisheye image to obtain the gradient amplitude of the fisheye image;

[0129] The edge point determination module 1330 is used to determine the edge points of the fisheye image based on the gradient amplitude; wherein the edge points include reflector edge points and fisheye edge points; the edge points are used to represent the pixel points whose gradient amplitude is greater than a preset threshold;

[0130] The edge point fitting result obtaining module 1340 is used to perform circle fitting on the edge points to obtain edge point fitting results;

[0131] The target fisheye image obtaining module 1350 is used to determine the projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain the target fisheye image.

[0132] In this technical solution, optionally, the edge point determination module 1330 includes:

[0133] a to-be-processed edge point determination unit, configured to, if the gradient amplitude satisfies a preset threshold condition, use a pixel point corresponding to the gradient amplitude as a to-be-processed edge point of the fisheye image;

[0134] The edge point determination unit is used to determine the edge points of the fisheye image according to the angles between the edge points to be processed.

[0135] In the technical solution, optionally, the to-be-processed edge point determination unit is specifically used to:

[0136] If the gradient amplitude is greater than a preset first threshold, taking the pixel point corresponding to the gradient amplitude as the edge point to be processed of the fisheye image;

[0137] If the gradient amplitude is less than a preset first threshold and greater than a preset second threshold, the adjacent pixel points of the pixel point corresponding to the gradient amplitude are determined, and it is judged whether the gradient amplitude of the adjacent pixel points is greater than the preset first threshold. If it is greater, the adjacent pixel points are used as edge points to be processed of the fisheye image.

[0138] In the technical solution, optionally, the edge point determination unit is specifically used to:

[0139] If the edge point of the fisheye image is the first edge point, selecting the edge point to be processed with the smallest horizontal and vertical coordinates from the edge points to be processed as the edge point of the fisheye image;

[0140] If the edge point of the fisheye image is not the first edge point, the angles between other edge points to be processed and the previously determined edge point of the fisheye image are calculated, and the edge point to be processed with the smallest angle value is used as the edge point of the fisheye image.

[0141] In this technical solution, optionally, the edge point fitting result obtaining module 1340 is specifically used for:

[0142] Calculate the distance between the edge point and the preset center of the circle to obtain the target distance;

[0143] Determine a loss function using the target distance and a preset radius;

[0144] The loss function is fitted to obtain an edge point fitting result.

[0145] In the present technical solution, optionally, the target fisheye image obtaining module 1350 is specifically used to: bring the edge point fitting result into a preset projection function formula to calculate the projection function of the fisheye image.

[0146] In this technical solution, optionally, the gradient amplitude obtaining module 1320 is specifically used for:

[0147] Based on a predetermined calculation formula, according to the pixel difference between adjacent pixels in the fisheye image, determine the gradient amplitude and gradient direction of the fisheye image;

[0148] If there is a gradient magnitude in the same gradient direction, the gradient magnitude is used as the gradient magnitude of the fisheye image;

[0149] If there are at least two gradient magnitudes in the same gradient direction, the maximum gradient magnitude of the at least two gradient magnitudes is used as the gradient magnitude of the fisheye image.

[0150] A fisheye image correction device provided by an embodiment of the present invention can execute a fisheye image correction method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0151] Embodiment 4

[0152] Fig.14 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0153] like Fig.14 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0154] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0155] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a fisheye image correction method.

[0156] In some embodiments, a fisheye image correction method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of a fisheye image correction method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a fisheye image correction method in any other appropriate manner (e.g., by means of firmware).

[0157] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0159] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0160] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0161] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0162] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0164] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A fisheye image correction method, characterized in that: include: Get the fisheye image taken by the fisheye camera; Processing the fisheye image to obtain a gradient amplitude of the fisheye image; Based on the gradient amplitude, edge points of the fisheye image are determined; wherein the edge points include reflector edge points and fisheye edge points; the edge points are used to characterize pixel points whose gradient amplitude is greater than a preset threshold; Performing circle fitting on the edge points to obtain edge point fitting results; Determine a projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain a target fisheye image; The step of determining the edge points of the fisheye image based on the gradient magnitude includes: If the gradient amplitude satisfies a preset threshold condition, the pixel point corresponding to the gradient amplitude is used as the edge point to be processed of the fisheye image; Determining edge points of the fisheye image according to angles between edge points to be processed; If the gradient amplitude satisfies a preset threshold condition, taking the pixel point corresponding to the gradient amplitude as the edge point to be processed of the fisheye image, comprising: If the gradient amplitude is greater than a preset first threshold, taking the pixel point corresponding to the gradient amplitude as the edge point to be processed of the fisheye image; If the gradient amplitude is less than a preset first threshold and greater than a preset second threshold, the adjacent pixel points of the pixel point corresponding to the gradient amplitude are determined, and it is judged whether the gradient amplitude of the adjacent pixel points is greater than the preset first threshold. If it is greater, the adjacent pixel points are used as edge points to be processed of the fisheye image; wherein the preset first threshold and the preset second threshold are set according to the requirements for determining the edge points to be processed, and the preset first threshold is greater than the preset second threshold; The processing of the fisheye image to obtain the gradient amplitude of the fisheye image includes: Based on a predetermined calculation formula, according to the pixel difference between adjacent pixels in the fisheye image, determine the gradient amplitude and gradient direction of the fisheye image; If there is a gradient magnitude in the same gradient direction, the gradient magnitude is used as the gradient magnitude of the fisheye image; If there are at least two gradient amplitudes in the same gradient direction, the maximum gradient amplitude of the at least two gradient amplitudes is used as the gradient amplitude of the fisheye image; The step of determining the edge points of the fisheye image according to the angles between the edge points to be processed comprises: If the edge point of the fisheye image is the first edge point, selecting the edge point to be processed with the smallest horizontal and vertical coordinates from the edge points to be processed as the edge point of the fisheye image; If the edge point of the fisheye image is not the first edge point, then calculating the angle between other edge points to be processed and the previously determined edge point of the fisheye image, and taking the edge point to be processed with the smallest angle value as the edge point of the fisheye image; The method further includes: if there are multiple edge points to be processed with the same angle, calculating the distance from the edge point to the edge point to be processed, and selecting the point with the largest distance to add to the point set to enter a loop judgment until the first edge point in the point set is selected.

2. The method according to claim 1, characterized in that: Performing circle fitting on the edge points to obtain edge point fitting results includes: Calculate the distance between the edge point and the preset center of the circle to obtain the target distance; Determine a loss function using the target distance and a preset radius; The loss function is fitted to obtain an edge point fitting result.

3. The method according to claim 1, characterized in that: Determining a projection function of the fisheye image according to the edge point fitting result includes: The edge point fitting result is substituted into a preset projection function formula to calculate the projection function of the fisheye image.

4. A fisheye image correction device, characterized in that: include: A fisheye image acquisition module is used to acquire fisheye images taken by a fisheye camera; A gradient amplitude obtaining module, used for processing the fisheye image to obtain the gradient amplitude of the fisheye image; An edge point determination module, used to determine the edge points of the fisheye image based on the gradient amplitude; wherein the edge points include reflector edge points and fisheye edge points; the edge points are used to characterize the pixel points whose gradient amplitude is greater than a preset threshold; An edge point fitting result obtaining module is used to perform circle fitting on the edge points to obtain edge point fitting results; A target fisheye image obtaining module is used to determine the projection function of the fisheye image according to the edge point fitting result, and use the projection function to correct the fisheye image to obtain a target fisheye image; The edge point determination module comprises: a to-be-processed edge point determination unit, configured to, if the gradient amplitude satisfies a preset threshold condition, use a pixel point corresponding to the gradient amplitude as a to-be-processed edge point of the fisheye image; An edge point determination unit, used for determining edge points of the fisheye image according to angles between edge points to be processed; The unit for determining edge points to be processed is specifically configured to, if the gradient amplitude is greater than a preset first threshold, use the pixel corresponding to the gradient amplitude as the edge point to be processed of the fisheye image; if the gradient amplitude is less than the preset first threshold and greater than a preset second threshold, determine the adjacent pixel points of the pixel corresponding to the gradient amplitude, and judge whether the gradient amplitude of the adjacent pixel points is greater than the preset first threshold; if so, use the adjacent pixel points as the edge point to be processed of the fisheye image; wherein the preset first threshold and the preset second threshold are set according to the requirement for determining the edge points to be processed, and the preset first threshold is greater than the preset second threshold; The gradient amplitude obtaining module is specifically used to: determine the gradient amplitude and gradient direction of the fisheye image based on a predetermined calculation formula and according to the pixel difference between adjacent pixels in the fisheye image; If there is a gradient magnitude in the same gradient direction, the gradient magnitude is used as the gradient magnitude of the fisheye image; If there are at least two gradient amplitudes in the same gradient direction, the maximum gradient amplitude of the at least two gradient amplitudes is used as the gradient amplitude of the fisheye image; The edge point determination unit is specifically configured to: if the edge point of the fisheye image is the first edge point, select the edge point to be processed with the smallest horizontal and vertical coordinates from the edge points to be processed as the edge point of the fisheye image; If the edge point of the fisheye image is not the first edge point, then calculating the angle between other edge points to be processed and the previously determined edge point of the fisheye image, and taking the edge point to be processed with the smallest angle value as the edge point of the fisheye image; The edge point determination unit is also used to If there are multiple edge points to be processed with the same angle, the distance from the edge point to the edge point to be processed is calculated, and the point with the largest distance is selected to be added to the point set to enter a loop judgment until the first edge point in the point set is selected.

5. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a fisheye image correction method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a fisheye image correction method according to any one of claims 1 to 3 when executed.

Images