Distortion correction method, device, equipment and storage medium for vehicle-mounted fisheye camera

Abstract

The embodiment of the present invention discloses a method, device, equipment and storage medium for distortion correction of a vehicle-mounted fisheye camera. The method includes: obtaining first position information of key points in a calibration image; controlling a vehicle-mounted fisheye camera to collect the calibration image to obtain a distorted image, and extracting second position information of the key points in the distorted image; fitting the first position information and the second position information of the key points using a cubic B-spline curve fitting algorithm to obtain a correction relationship; obtaining the image to be corrected collected by the vehicle-mounted fisheye camera; and correcting the image to be corrected according to the correction relationship. The method for distortion correction of a vehicle-mounted fisheye camera provided by the embodiment of the present invention uses a cubic B-spline curve fitting algorithm to fit pixel points in a calibration image and pixel points in a distorted image to obtain a correction relationship, thereby performing distortion correction on the image collected by the fisheye camera according to the correction relationship, which can improve the accuracy of distortion correction.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of autonomous driving technology, and in particular to a method, device, equipment and storage medium for vehicle-mounted fisheye camera distortion correction. Background Art

[0002] The notable features of the fisheye lens on autonomous driving vehicles are short focal length and large field of view. Its advantages are large viewing angle and small blind spot. The market range can contain the content of multiple lenses. The image contains a large amount of information and is rich. It is widely used in autonomous driving and other systems. However, the ultra-wide field of view of the fisheye lens also brings obvious disadvantages. The captured image has serious distortion, resulting in distortion of real objects. If the special effect of ultra-wide angle is not required, but the information of ultra-wide viewing angle of the image is to be used, then these distorted images need to be expanded into real perspective projection images. Therefore, in order to make full use of the information in the fisheye image, it is very important in the vehicle-mounted intelligent system to correct this distortion and restore the real object.

[0003] The existing fisheye distortion correction algorithm defines the fisheye lens imaging model as a polynomial of the incident angle with respect to the distance between the imaging point and the optical axis. At the same time, collinearity constraints, parallel constraints, and orthogonality constraints are introduced, and the parameters of the model are solved by minimizing the eigenvalues. Finally, the obtained correction model is used to correct the fisheye image. The image corrected by this method has a realistic correction effect on the content on the ground in the small viewing area in the middle of the image, but the three-dimensional objects above the ground are severely stretched and distorted, and the edge area with a large field of view is blurred and deformed. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, equipment and storage medium for correcting distortion of a vehicle-mounted fisheye camera, which can improve the accuracy of distortion correction.

[0005] In a first aspect, an embodiment of the present invention provides a method for correcting distortion of a vehicle-mounted fisheye camera, comprising:

[0006] Acquire first position information of a key point in a calibration image; the key point includes a plurality of key points;

[0007] Controlling the on-vehicle fisheye camera to collect the calibration image to obtain a distorted image, and extracting second position information of the key point in the distorted image;

[0008] A cubic B-spline curve fitting algorithm is used to fit the first position information and the second position information of the key point to obtain a correction relationship; the correction relationship represents the relationship between the pixel points in the calibration image and the pixel points in the distorted image;

[0009] Obtaining the image to be corrected captured by the vehicle-mounted fisheye camera;

[0010] The image to be corrected is corrected according to the correction relationship.

[0011] Furthermore, the calibration image includes two mutually perpendicular straight lines with the center of the calibration image as the intersection point and a plurality of concentric circles with the center of the image as the center; the key point is the intersection point of the plurality of concentric circles and the two mutually perpendicular straight lines.

[0012] Furthermore, extracting second position information of the key point in the distorted image includes:

[0013] The distorted image is subjected to de-noising, gray-scaling and binarization processing in sequence;

[0014] Extract straight lines in the processed image;

[0015] Key points are extracted from the straight line to obtain second position information of the key points.

[0016] Further, a cubic B-spline curve fitting algorithm is used to fit the first position information and the second position information of the key point to obtain a correction relationship, including:

[0017] Determine a first distance according to the first position information; the first distance is the distance between the key point in the calibration image and the image center;

[0018] Determine a second distance according to the second position information; the second distance is the distance between the key point in the distorted image and the image center;

[0019] The first distance and the second distance are fitted using a cubic B-spline curve fitting algorithm to obtain a correction relationship.

[0020] Further, a cubic B-spline curve fitting algorithm is used to fit the first distance and the second distance to obtain a correction relationship, including:

[0021] The first distance and the second distance of each key point constitute a data pair to obtain a plurality of initial data pairs;

[0022] Based on the multiple initial data pairs, a catch-up method is used to calculate to obtain multiple control data pairs; the control data pairs include the initial data pairs;

[0023] Obtaining basis functions in the cubic B-spline curve fitting algorithm;

[0024] randomly extracting four data pairs from the plurality of control data pairs;

[0025] generating a correction relationship according to the four data pairs and the basis functions;

[0026] Determine whether the end condition is met; if not, return to execute the operation of randomly extracting four data pairs from the multiple control data pairs until the end condition is met.

[0027] Furthermore, the end condition includes that the number of cycles reaches a set number or the correction relationship satisfies a set condition.

[0028] Further, correcting the image to be corrected according to the correction relationship includes:

[0029] Obtaining an initial rectified image, and traversing each pixel in the initial rectified image;

[0030] Determine the third distance between the traversed pixel point and the image center;

[0031] determining a fourth distance according to the third distance and the correction relationship;

[0032] Determining, according to the fourth distance, position information of a pixel point in the image to be corrected corresponding to the traversed pixel point;

[0033] The grayscale value of the pixel point traversed in the initial corrected image is replaced by the grayscale value of the pixel point at the position information in the image to be corrected, and the next pixel point in the initial corrected image is traversed continuously until the corrected image is obtained.

[0034] In a second aspect, an embodiment of the present invention further provides a vehicle-mounted fisheye camera distortion correction device, comprising:

[0035] A first position information acquisition module is used to acquire first position information of a key point in a calibration image; the key point includes a plurality of key points;

[0036] A second position information acquisition module, used to control the vehicle-mounted fisheye camera to collect the calibration image, obtain a distorted image, and extract the second position information of the key point in the distorted image;

[0037] A correction relationship acquisition module, used to fit the first position information and the second position information of the key point using a cubic B-spline curve fitting algorithm to obtain a correction relationship; the correction relationship represents the relationship between the pixel points in the calibration image and the pixel points in the distorted image;

[0038] A module for acquiring an image to be corrected, used for acquiring the image to be corrected captured by the vehicle-mounted fisheye camera;

[0039] A correction module is used to correct the image to be corrected according to the correction relationship.

[0040] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for correcting the distortion of a vehicle-mounted fisheye camera as described in an embodiment of the present invention is implemented.

[0041] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle-mounted fisheye camera distortion correction method as described in an embodiment of the present invention.

[0042] The embodiment of the present invention discloses a method, device, equipment and storage medium for distortion correction of a vehicle-mounted fisheye camera. Obtain first position information of key points in a calibration image; control the vehicle-mounted fisheye camera to collect the calibration image to obtain a distorted image, and extract second position information of key points in the distorted image; use a cubic B-spline curve fitting algorithm to fit the first position information and the second position information of the key points to obtain a correction relationship; obtain the image to be corrected collected by the vehicle-mounted fisheye camera; correct the image to be corrected according to the correction relationship. The method for distortion correction of a vehicle-mounted fisheye camera provided by the embodiment of the present invention uses a cubic B-spline curve fitting algorithm to fit the pixel points in the calibration image and the pixel points in the distorted image to obtain a correction relationship, thereby performing distortion correction on the image collected by the fisheye camera according to the correction relationship, which can improve the accuracy of distortion correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of a method for correcting distortion of a vehicle-mounted fisheye camera in Embodiment 1 of the present invention;

[0044] Figure 2 is an example diagram of a calibration image in the first embodiment of the present invention;

[0045] Figure 3 is a structural schematic diagram of a vehicle-mounted fisheye camera distortion correction device in Embodiment 2 of the present invention;

[0046] Figure 4 It is a structural diagram of a computer device in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0048] Embodiment 1

[0049] Figure 1This is a flow chart of a method for correcting distortion of a vehicle-mounted fisheye camera provided in Embodiment 1 of the present invention. This embodiment is applicable to the case where distortion correction is performed on an image captured by a vehicle-mounted fisheye camera. The method can be performed by a vehicle-mounted fisheye camera distortion correction device, such as Figure 1 As shown, the method specifically comprises the following steps:

[0050] Step 110: Obtain first position information of key points in the calibration image.

[0051] Among them, the key point may include multiple ones. The calibration image may be an artificially designed image. In this embodiment, the calibration image includes two mutually perpendicular straight lines with the center of the calibration image as the intersection point and multiple concentric circles with the center of the image as the center, and the spacing between adjacent concentric circles is equal; the key point is the intersection point of the multiple concentric circles and the two mutually perpendicular straight lines. Exemplary, Figure 2 is an example diagram of the calibration image in this embodiment.

[0052] The first position information may be understood as the position information of the key point in the calibration image, that is, the position information of the key point before distortion occurs.

[0053] Step 120 , controlling the vehicle-mounted fisheye camera to collect the calibration image, obtaining a distorted image, and extracting second position information of key points in the distorted image.

[0054] In this embodiment, due to the characteristics of the fisheye camera itself, when collecting images, the collected images will be distorted to a certain extent. Therefore, by using the on-board fisheye camera to collect the calibration image, a distorted image corresponding to the calibration image can be obtained.

[0055] Specifically, the process of extracting the second position information of the key point in the distorted image can be: performing denoising, grayscale and binarization processing on the distorted image in sequence; extracting a straight line in the processed image; extracting the key point of the straight line to obtain the second position information of the key point.

[0056] Among them, a bilateral filtering algorithm can be used to remove noise points from the distorted image. A Hough line detection algorithm can be used to extract straight lines in the image. A key point detection algorithm (Fast algorithm) can be used to extract key points on the straight line to obtain the second position information of the key points in the distorted image.

[0057] Step 130: Use a cubic B-spline curve fitting algorithm to fit the first position information and the second position information of the key point to obtain a correction relationship.

[0058] The correction relationship represents the relationship between the pixel points in the calibration image and the pixel points in the distorted image, and specifically represents the relationship between the distance between the pixel points in the calibration image and the image center and the distance between the pixel points in the distorted image and the image center. In this embodiment, the first distance between the key point in the calibration image and the image center and the second distance between the key point in the distorted image and the image center can be fitted.

[0059] Specifically, a cubic B-spline curve fitting algorithm is used to fit the first position information and the second position information of the key point, and a method for obtaining the correction relationship can be: determining a first distance based on the first position information; determining a second distance based on the second position information; and fitting the first distance and the second distance using a cubic B-spline curve fitting algorithm to obtain the correction relationship.

[0060] Among them, the first distance is the distance between the key point in the calibration image and the center of the image. The second distance is the distance between the key point in the distorted image and the center of the image. In this embodiment, the first distance and the second distance are fitted by a cubic B-spline curve fitting algorithm, and the correction relationship can be obtained by: forming a data pair with the first distance and the second distance of each key point to obtain multiple initial data pairs; using the catch-up method to calculate based on the multiple initial data pairs to obtain multiple control data pairs; obtaining the basis function in the cubic B-spline curve fitting algorithm; randomly extracting four data pairs from the multiple control data pairs; generating a correction relationship based on the four data pairs and the basis function; judging whether the end condition is met; if not, returning to execute the operation of randomly extracting four data pairs from the multiple control data pairs until the end condition is met.

[0061] Among them, the control data pair includes the initial data pair. The end condition can be that the number of cycles reaches the set number of times or the correction relationship meets the set conditions. Exemplarily, assuming that the initial data pair consisting of the first distance and the second distance is (x_in, y_in), the initial data pair is calculated using the catch-up method to obtain n control data pairs (x_control, y_control). Then, four basis functions b0, b1, b2, and b3 are calculated according to the four basis function formulas. Each time, 4 data pairs are randomly selected from the n control data pairs and substituted into the following formula to obtain the correction relationship: p i (t) = p i b0+p i+1 b1+p i+2 b2+p i+3 b3; If the end condition is not met, continue to randomly select 4 data pairs and substitute them into the above formula to obtain the correction relationship p i+1 (t), compare p i+1 (t) and p i (t), retain the better correction relationship until the cycle end condition is met and the optimal correction relationship is obtained.

[0062] Step 140: Obtain an image to be corrected captured by a vehicle-mounted fisheye camera.

[0063] The image to be corrected is an image obtained by collecting the surrounding information of the vehicle by the on-board fisheye camera.

[0064] Step 150, correcting the image to be corrected according to the correction relationship.

[0065] Specifically, the method of correcting the image to be corrected according to the correction relationship can be: obtaining the initial corrected image and traversing each pixel in the initial corrected image; determining the third distance between the traversed pixel and the center of the image; determining the fourth distance according to the third distance and the correction relationship; determining the position information of the pixel corresponding to the traversed pixel in the image to be corrected according to the fourth distance; replacing the grayscale value of the traversed pixel in the initial corrected image with the grayscale value of the pixel at the position information in the image to be corrected, and continuing to traverse the next pixel in the initial corrected image until the corrected image is obtained.

[0066] The initial corrected image may be an image without any content, such as a pure white or pure black image. In this embodiment, the correction process may be to fill the grayscale values ​​of the pixels in the image to be corrected into the corrected image according to the corrected positions.

[0067] The technical solution of this embodiment is to obtain the first position information of the key point in the calibration image; control the vehicle-mounted fisheye camera to collect the calibration image to obtain the distorted image, and extract the second position information of the key point in the distorted image; use the cubic B-spline curve fitting algorithm to fit the first position information and the second position information of the key point to obtain the correction relationship; obtain the image to be corrected collected by the vehicle-mounted fisheye camera; and correct the image to be corrected according to the correction relationship. The vehicle-mounted fisheye camera distortion correction method provided by the embodiment of the present invention uses the cubic B-spline curve fitting algorithm to fit the pixel points in the calibration image and the pixel points in the distorted image to obtain the correction relationship, thereby performing distortion correction on the image collected by the fisheye camera according to the correction relationship, which can improve the accuracy of distortion correction.

[0068] Embodiment 2

[0069] Figure 3 Schematic diagram of the structure of a vehicle-mounted fisheye camera distortion correction device provided by the second embodiment of the present invention. Figure 3 As shown, the device comprises:

[0070] A first position information acquisition module 210 is used to acquire first position information of a key point in a calibration image; the key point includes multiple key points;

[0071] The second position information acquisition module 220 is used to control the vehicle-mounted fisheye camera to collect the calibration image, obtain the distorted image, and extract the second position information of the key point in the distorted image;

[0072] The correction relationship acquisition module 230 is used to fit the first position information and the second position information of the key point using a cubic B-spline curve fitting algorithm to obtain a correction relationship; the correction relationship represents the relationship between the pixel points in the calibration image and the pixel points in the distorted image;

[0073] The image to be corrected acquisition module 240 is used to acquire the image to be corrected captured by the vehicle-mounted fisheye camera;

[0074] The correction module 250 is used to correct the image to be corrected according to the correction relationship.

[0075] Optionally, the calibration image includes two mutually perpendicular straight lines with the center of the calibration image as the intersection point and multiple concentric circles with the center of the image as the center, and the spacing between adjacent concentric circles is equal; the key point is the intersection point of the multiple concentric circles and the two mutually perpendicular straight lines.

[0076] Optionally, the second location information obtaining module 220 is further used to:

[0077] The distorted image is de-noised, grayed and binarized in turn;

[0078] Extract straight lines in the processed image;

[0079] The key points of the straight line are extracted to obtain the second position information of the key points.

[0080] Optionally, the correction relationship acquisition module 230 is further used to:

[0081] Determine a first distance according to the first position information; the first distance is the distance between the key point in the calibration image and the center of the image;

[0082] Determine a second distance according to the second position information; the second distance is the distance between the key point in the distorted image and the center of the image;

[0083] The first distance and the second distance are fitted using a cubic B-spline curve fitting algorithm to obtain a correction relationship.

[0084] Optionally, the correction relationship acquisition module 230 is further used to:

[0085] The first distance and the second distance of each key point constitute a data pair to obtain a plurality of initial data pairs;

[0086] Based on the multiple initial data pairs, a chase method is used to calculate and obtain multiple control data pairs; the control data pairs include the initial data pairs;

[0087] Get the basis functions in the cubic B-spline curve fitting algorithm;

[0088] Four data pairs are randomly selected from multiple control data pairs;

[0089] Generate a correction relationship based on four data pairs and basis functions;

[0090] Determine whether the end condition is met; if not, return to execute the operation of randomly extracting four data pairs from multiple control data pairs until the end condition is met.

[0091] Optionally, the end condition includes the number of cycles reaching a set number or the correction relationship meeting a set condition.

[0092] Optionally, the correction module 250 is further configured to:

[0093] Get the initial rectified image and traverse each pixel in the initial rectified image;

[0094] Determine the third distance between the traversed pixel point and the image center;

[0095] determining a fourth distance based on the third distance and the correction relationship;

[0096] Determine, according to the fourth distance, position information of a pixel point in the image to be corrected corresponding to the traversed pixel point;

[0097] The grayscale value of the pixel point traversed in the initial corrected image is replaced by the grayscale value of the pixel point at the position information in the image to be corrected, and the next pixel point in the initial corrected image is traversed continuously until the corrected image is obtained.

[0098] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not described in detail in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0099] Embodiment 3

[0100] Figure 4 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. Figure 4 A block diagram of a computer device 312 suitable for use in implementing embodiments of the present invention is shown. Figure 4 The computer device 312 shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. The device 312 is a typical computing device for the distortion correction function of a vehicle-mounted fisheye camera.

[0101] like Figure 4As shown, the computer device 312 is in the form of a general-purpose computing device. The components of the computer device 312 may include, but are not limited to: one or more processors 316, a storage device 328, and a bus 318 connecting different system components (including the storage device 328 and the processor 316).

[0102] Bus 318 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0103] The computer device 312 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 312, including volatile and non-volatile media, removable and non-removable media.

[0104] The storage device 328 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 330 and / or cache memory 332. The computer device 312 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 334 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a read-only optical disk (Compact Disc-Read Only Memory, CD-ROM), a digital video disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 318 via one or more data medium interfaces. The storage device 328 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0105] A program 336 having a set (at least one) of program modules 326 may be stored, for example, in a storage device 328, such program modules 326 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 326 generally perform the functions and / or methods of the embodiments described herein.

[0106] The computer device 312 may also communicate with one or more external devices 314 (e.g., keyboard, pointing device, camera, display 324, etc.), may also communicate with one or more devices that enable a user to interact with the computer device 312, and / or communicate with any device that enables the computer device 312 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 322. In addition, the computer device 312 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter 320. As shown, the network adapter 320 communicates with other modules of the computer device 312 through a bus 318. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the computer device 312, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.

[0107] The processor 316 executes various functional applications and data processing by running the programs stored in the storage device 328, such as implementing the vehicle-mounted fisheye camera distortion correction method provided in the above embodiment of the present invention.

[0108] Embodiment 4

[0109] The embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processing device, a method for switching the simulated flight scene in the embodiment of the present invention is implemented. The above-mentioned computer-readable medium of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0110] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0111] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0112] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the first position information of the key point in the calibration image; the key point includes multiple points; controls the vehicle-mounted fisheye camera to collect the calibration image, obtains the distorted image, and extracts the second position information of the key point in the distorted image; uses the cubic B-spline curve fitting algorithm to fit the first position information and the second position information of the key point to obtain a correction relationship; the correction relationship represents the relationship between the pixel points in the calibration image and the pixel points in the distorted image; obtains the image to be corrected collected by the vehicle-mounted fisheye camera; and corrects the image to be corrected according to the correction relationship.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0116] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0117] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for correcting distortion of a vehicle-mounted fisheye camera, characterized in that: include: Acquire first position information of a key point in a calibration image; the key point includes a plurality of key points; Controlling the on-vehicle fisheye camera to collect the calibration image to obtain a distorted image, and extracting second position information of the key point in the distorted image; Using a cubic B-spline curve fitting algorithm to fit the first position information and the second position information of the key point to obtain a correction relationship; The correction relationship represents the relationship between the pixel points in the calibration image and the pixel points in the distorted image; Obtaining the image to be corrected captured by the vehicle-mounted fisheye camera; Correcting the image to be corrected according to the correction relationship; The first position information and the second position information of the key point are fitted using a cubic B-spline curve fitting algorithm to obtain a correction relationship, including: Determine a first distance according to the first position information; the first distance is the distance between the key point in the calibration image and the image center; Determine a second distance according to the second position information; the second distance is the distance between the key point in the distorted image and the image center; Using a cubic B-spline curve fitting algorithm to fit the first distance and the second distance to obtain a correction relationship; The first distance and the second distance are fitted using a cubic B-spline curve fitting algorithm to obtain a correction relationship, including: The first distance and the second distance of each key point constitute a data pair to obtain a plurality of initial data pairs; Based on the multiple initial data pairs, a catch-up method is used to calculate to obtain multiple control data pairs; the control data pairs include the initial data pairs; Obtaining basis functions in the cubic B-spline curve fitting algorithm; randomly extracting four data pairs from the plurality of control data pairs; generating a correction relationship according to the four data pairs and the basis functions; Determine whether the end condition is met; if not, return to execute the operation of randomly extracting four data pairs from the multiple control data pairs until the end condition is met.

2. The method according to claim 1, characterized in that: The calibration image includes two mutually perpendicular straight lines with the center of the calibration image as the intersection point and a plurality of concentric circles with the center of the image as the center, and the spacing between adjacent concentric circles is equal; the key point is the intersection point of the plurality of concentric circles and the two mutually perpendicular straight lines.

3. The method according to claim 1, characterized in that: Extracting second position information of the key point in the distorted image includes: The distorted image is subjected to de-noising, gray-scaling and binarization processing in sequence; Extract straight lines in the processed image; Key points are extracted from the straight line to obtain second position information of the key points.

4. The method according to claim 1, characterized in that The end condition includes that the number of cycles reaches a set number or the correction relationship satisfies a set condition.

5. The method according to claim 1, characterized in that Correcting the image to be corrected according to the correction relationship includes: Obtaining an initial rectified image, and traversing each pixel in the initial rectified image; Determine the third distance between the traversed pixel point and the image center; determining a fourth distance according to the third distance and the correction relationship; Determining, according to the fourth distance, position information of a pixel point in the image to be corrected corresponding to the traversed pixel point; The grayscale value of the pixel point traversed in the initial corrected image is replaced by the grayscale value of the pixel point at the position information in the image to be corrected, and the next pixel point in the initial corrected image is traversed continuously until the corrected image is obtained.

6. A vehicle-mounted fisheye camera distortion correction device, characterized in that: include: A first position information acquisition module, used to acquire first position information of key points in the calibration image; The key points include multiple; A second position information acquisition module, used to control the vehicle-mounted fisheye camera to collect the calibration image, obtain a distorted image, and extract the second position information of the key point in the distorted image; A correction relationship acquisition module, used to fit the first position information and the second position information of the key point using a cubic B-spline curve fitting algorithm to obtain a correction relationship; The correction relationship represents the relationship between the pixel points in the calibration image and the pixel points in the distorted image; A module for acquiring an image to be corrected, used for acquiring the image to be corrected captured by the vehicle-mounted fisheye camera; A correction module, used for correcting the image to be corrected according to the correction relationship; Wherein, the correction relationship acquisition module is also used for: Determine a first distance according to the first position information; the first distance is the distance between the key point in the calibration image and the center of the image; Determine a second distance according to the second position information; the second distance is the distance between the key point in the distorted image and the center of the image; The first distance and the second distance are fitted using a cubic B-spline curve fitting algorithm to obtain a correction relationship; Wherein, the correction relationship acquisition module is also used for: The first distance and the second distance of each key point constitute a data pair to obtain a plurality of initial data pairs; Based on the multiple initial data pairs, a chase method is used to calculate and obtain multiple control data pairs; the control data pairs include the initial data pairs; Get the basis functions in the cubic B-spline curve fitting algorithm; Four data pairs are randomly selected from multiple control data pairs; Generate a correction relationship based on four data pairs and basis functions; Determine whether the end condition is met; if not, return to execute the operation of randomly extracting four data pairs from multiple control data pairs until the end condition is met.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the vehicle-mounted fisheye camera distortion correction method as described in any one of claims 1-5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the vehicle-mounted fisheye camera distortion correction method as described in any one of claims 1 to 5 is implemented.

Images