Image processing method, device and storage medium

Abstract

The present invention discloses an image processing method, device and storage medium, which relate to the field of image processing technology. The method comprises: performing distortion correction on the distorted image captured by a fisheye camera to obtain a corrected image with a consistent magnification factor at all locations; correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles to obtain a corrected image; wherein the observation magnification factor of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver. The image processing method, device and storage medium disclosed in the present invention can correct the distorted image captured by the fisheye camera under the premise that the computing power of the vehicle-mounted electronic mirror system is limited.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image processing method, device and storage medium. Background Art

[0002] To ensure driving safety, according to the ISO (Digital Camera Sensitivity Quantification) standard, the image of objects displayed in the central area of ​​the car's electronic rearview mirror must show minimal distortion, and the display area must extract a sufficiently large field of view to allow the driver to accurately judge the driving environment and provide conditions for driving prediction.

[0003] Currently, most automotive electronic rearview mirrors on the market use fisheye cameras, which cannot achieve the display effects specified by ISO standards. To achieve a better viewing experience, the most common method is to use deep learning networks such as Inception or VGGNet (Visual Geometry Group Networks) to correct the distorted images captured by the automotive electronic rearview mirror. Although this transformation method can improve the accuracy of distorted image correction, the high computational complexity of deep learning algorithms makes it unsuitable for automotive electronic rearview mirror systems with limited computing power.

[0004] Therefore, how to provide an effective solution to correct the distorted image obtained by the fisheye camera under the premise of limited computing power of the vehicle-mounted electronic mirror system has become an urgent problem to be solved in the existing technology. Summary of the Invention

[0005] The purpose of the present invention is to provide an image processing method, device and storage medium to solve the above-mentioned problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides an image processing method for correcting a distorted image captured by a fisheye camera on a car, comprising:

[0008] Perform distortion correction on the distorted image captured by the fisheye camera to obtain a corrected image with consistent magnification coefficients at all locations;

[0009] Correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles to obtain a corrected image;

[0010] Among them, the observation magnification coefficient of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver.

[0011] In one possible design, the pixel coordinates of each pixel point in the corrected image are corrected based on the magnification factor of the glass mirror for observing objects at different angles, including:

[0012] Correcting the pixel coordinates of each pixel in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles, the polar longitude corresponding to the projection point of each pixel in the corrected image on the plane coordinate system, and the angle between the polar longitude corresponding to each pixel in the corrected image and the x-axis in the plane coordinate system;

[0013] The plane coordinate system is established with the direction parallel to the vehicle body as the y-axis, the direction perpendicular to the vehicle body as the x-axis, and the point where the fisheye camera is projected on the horizontal plane as the origin.

[0014] In one possible design, the pixel coordinates of each pixel point in the corrected image are corrected based on the observation magnification factor of the glass viewfinder for objects at different angles, the polar longitude corresponding to the projection point of each pixel point in the corrected image on the plane coordinate system, and the angle between the polar longitude corresponding to each pixel point in the corrected image and the x-axis in the plane coordinate system, including:

[0015] Correct the pixel coordinates of each pixel point in the corrected image according to the following formula;

[0016]

[0017] Among them, m 00 、m 01 、m 10 、m 11 、m 20 and m 21 are all parameters in the intrinsic parameter matrix of the fisheye camera, k is a constant, r represents the polar longitude corresponding to the projection point of any pixel point in the image on the plane coordinate system established with the y-axis parallel to the vehicle body, the x-axis perpendicular to the vehicle body, and the point where the fisheye camera is projected on the horizontal plane as the origin, θ represents the angle between the polar longitude corresponding to any pixel point and the x-axis, and M (θ) It represents the observation magnification factor of the glass mirror for an object with a polar angle of θ in a plane coordinate system.

[0018] In one possible design, the observation magnification factor is M (θ) =-0.6(π / 2+β0-θ) 2 +0.001(π / 2+β0-θ)+0.1, where β0 represents the angle between the driver's projected sight line and the y-axis, and the driver's projected sight line is the line formed by the driver's eye point projected onto the plane coordinate system and the origin.

[0019] In one possible design, before correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass lens for objects at different angles, the method further includes:

[0020] A plane coordinate system is established with the direction parallel to the vehicle body as the y-axis, the direction perpendicular to the vehicle body as the x-axis, and the point where the fisheye camera is projected on the horizontal plane as the origin.

[0021] In one possible design, the distortion correction of the distorted image captured by the fisheye camera includes:

[0022] The distortion images captured by the fisheye camera are corrected using Zhang Zhengyou's plane calibration method and polynomial model transformation algorithm.

[0023] In a second aspect, the present invention provides an image processing device for correcting a distorted image captured by a fisheye camera on a car, comprising:

[0024] A distortion correction unit is used to correct the distortion of the distorted image captured by the fisheye camera to obtain a corrected image with a consistent magnification factor at all locations;

[0025] A coordinate correction unit, configured to correct the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles, to obtain a corrected image;

[0026] Among them, the observation magnification coefficient of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver.

[0027] In a third aspect, the present invention provides an image processing device comprising a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the image processing method as described in the first aspect above.

[0028] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are run on a computer, the image processing method described in the first aspect is executed.

[0029] In a fifth aspect, the present invention provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the image processing method as described in the first aspect.

[0030] Beneficial effects:

[0031] The present invention creatively provides an image processing solution, namely, performing distortion correction on a distorted image captured by a fisheye camera to obtain a corrected image with a consistent magnification coefficient at all locations; and correcting the pixel coordinates of each pixel in the corrected image based on the magnification coefficient of the glass mirror for objects at different angles to obtain a corrected image; wherein the magnification coefficient of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver. Since the distorted image captured by the fisheye camera is first subjected to distortion correction to ensure that the magnification coefficient is consistent at all locations in the image, thereby making the magnification coefficient at all locations in the image close to the magnification coefficient of the glass mirror, and then correcting the pixel coordinates of each pixel in the corrected image based on the magnification coefficient of the glass mirror for objects at different angles, the corrected image is closer to the image directly observed by the naked eye, thereby achieving a better visual effect and greatly improving driving safety. Furthermore, the process does not require complex calculations and can correct the distorted image captured by the fisheye camera under the premise of limited computing power of the on-board electronic mirror system, facilitating practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0034] Figure 3 A schematic structural diagram of another image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0036] In order to solve the problem in the prior art that the computing power of the vehicle-mounted electronic mirror system is limited and cannot be used to correct the distorted image obtained by the fisheye camera, the embodiments of the present application provide an image processing method, device and storage medium. The image processing method, device and storage medium can correct the distorted image obtained by the fisheye camera under the premise that the computing power of the vehicle-mounted electronic mirror system is limited.

[0037] The image processing method provided in the embodiment of the present application can be applied to a vehicle-mounted electronic mirror system. The image processing method provided in the embodiment of the present application will be described in detail below. It can be understood that the execution entity does not constitute a limitation on the embodiment of the present application.

[0038] like Figure 1 , which is a flowchart of the image processing method provided in the first aspect of the embodiment of the present application. The image processing method may include, but is not limited to, the following steps S101-S102.

[0039] Step S101 : performing distortion correction on the distorted image captured by the fisheye camera to obtain a corrected image with a consistent magnification factor at all locations.

[0040] In the embodiment of the present application, when correcting the distorted image captured by the fisheye camera, the distortion correction can be performed on the distorted image captured by the fisheye camera first to obtain a corrected image with a consistent magnification factor at all locations.

[0041] When performing distortion correction on distorted images captured by a fisheye camera, the Zhang Zhengyou plane calibration method and the polynomial model transformation algorithm can be used to correct the distortion of the distorted images captured by the fisheye camera, thereby obtaining a corrected image with a consistent magnification factor at all locations. In this case, the corrected image is close to the image observed through a glass viewfinder. The Zhang Zhengyou plane calibration method and the polynomial model transformation algorithm are both prior arts, and their principles will not be described in detail in the embodiments of this application.

[0042] It is understandable that in some other embodiments, the distortion correction of the distorted image can also be performed through other existing methods, which will not be given one by one in the embodiments of the present application.

[0043] In addition, it should be noted that the method provided in the embodiment of the present application can also be applied to other types of cameras mounted on vehicles and similar to fisheye cameras, which have image distortion caused by the protrusion of the lens toward the front of the lens.

[0044] Step S102: Based on the observation magnification factor of the glass mirror for objects at different angles, the pixel coordinates of each pixel point in the corrected image are corrected to obtain a corrected image.

[0045] Using Zhang Zhengyou's plane calibration method and a polynomial model transformation algorithm, the distorted image captured by the fisheye camera is corrected for distortion. The resulting corrected image approximates the image observed through a glass lens, which differs somewhat from the image observed with the naked eye. Based on this, after correcting the distortion of the distorted image, the embodiments of the present application can also calibrate the pixel coordinates of each pixel in the corrected image based on the magnification factor of the glass lens for observing objects at different angles, thereby obtaining a corrected image.

[0046] Among them, the observation magnification coefficient of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver.

[0047] Specifically, when correcting the pixel coordinates of each pixel in the corrected image, the pixel coordinates of each pixel in the corrected image can be corrected based on the magnification factor of the glass lens for observing objects at different angles, the polar longitude corresponding to the projection point of each pixel in the corrected image on the plane coordinate system, and the angle between the polar longitude corresponding to each pixel in the corrected image and the x-axis of the plane coordinate system. The plane coordinate system is established with the y-axis parallel to the vehicle body, the x-axis perpendicular to the vehicle body, and the point where the fisheye camera is projected on the horizontal plane as its origin.

[0048] More specifically, the pixel coordinates of each pixel in the rectified image can be corrected according to the following formula (1):

[0049]

[0050] Among them, m 00 、m 01 、m 10 、m 11 、m 20 and m 21 are all parameters in the intrinsic parameter matrix of the fisheye camera, k is a constant, r represents the polar longitude corresponding to the projection point of any pixel point in the image on the plane coordinate system established with the y-axis parallel to the vehicle body, the x-axis perpendicular to the vehicle body, and the point where the fisheye camera is projected on the horizontal plane as the origin, θ represents the angle between the polar longitude corresponding to any pixel point and the x-axis, and M (θ) It represents the observation magnification factor of the glass mirror for an object with a polar angle of θ in a plane coordinate system.

[0051] In the embodiment of the present application, the intrinsic parameter matrix of the fisheye camera can be expressed as The internal parameter matrix is ​​the known parameters of the fisheye camera. The observation magnification factor can be expressed as M (θ) =-0.6(π / 2+β0-θ) 2 +0.001(π / 2+β0-θ)+0.1, where β0 represents the angle between the driver's projected sight line and the y-axis, and the driver's projected sight line is the line formed by the driver's eye point projected onto the plane coordinate system and the origin.

[0052] In one or more embodiments, before correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass viewfinder for objects at different angles, the method further includes: establishing a plane coordinate system with the direction parallel to the vehicle body as the y-axis, the direction perpendicular to the vehicle body as the x-axis, and the point projected by the fisheye camera on the horizontal plane as the origin.

[0053] The following will derive and explain the calibration formula for correcting the pixel coordinates of each pixel point in the corrected image in the embodiment of the present application.

[0054] In the embodiment of the present application, a correction formula for image correction can be determined by studying the relevant influencing factors of the glass sight glass and the magnification factor.

[0055] Taking the driver's side wide-angle exterior mirror as an example, assuming that O is the position of the glass mirror, and the projection of the glass mirror on the ground is the same as the projection position of the fisheye camera on the ground, A is the position of the object, and B is the driver's eye point. According to the ISO standard, the magnification factor of the glass mirror is mainly related to the distance d from the object to the glass mirror (the length of OA, hereinafter referred to as the object distance d) and the angle β between the line OB from the eye point to the glass mirror (hereinafter referred to as the driver's field of view angle).

[0056] In the ISO standard, the amplification factor is calculated as follows:

[0057] (1) When the object distance d is infinite and the driver's field of view angle β = 0 (i.e., directly in front of the glass mirror), the corresponding magnification coefficient calculation formula is shown in the following formula (2).

[0058]

[0059] Among them, r mirror Indicates the radius of curvature of the glass mirror, a mirror Indicates the distance from the eye point to the glass lens.

[0060] (2) When the object distance is infinite and the driver's field of view angle β is changed, the corresponding magnification factor is shown in the following formula (3).

[0061]

[0062] Among them, a=a mirror , r=r mirror , Δa represents the change in the distance from the eye point to the glass mirror, Δβ represents the change in the driver's field of view angle β, β1 represents the value of the driver's field of view angle β before the change, β2 represents the value of the driver's field of view angle β after the change, and β2>β1.

[0063] When the driver's field of view angle β=0, the corresponding magnification coefficient M(β=0, d) when the object distance d is changed is shown in formula (4).

[0064]

[0065] Combining formulas (2), (3), and (4), we can know that the amplification factor formula for changing β and d is shown in formula (5).

[0066]

[0067] Formula (5) shows the relationship between the glass mirror's magnification factor and the object distance d and the driver's field of view angle β. In actual driving, drivers typically focus on observing the environment within 30 meters of the vehicle in the main rearview mirror, with a field of view angle typically ranging from 55° to 75°. Table 1 below shows the change rate and ratio of the magnification factor M when the driver's side field of view angle β changes while the object distance d remains constant, and when the field of view angle β changes while the object distance d remains constant, calculated using Formula (5) in a test.

[0068] Table 1

[0069] β is constant, d varies d is constant, β varies (d constant, β varying) / (β constant, d varying) Main rearview mirror 10.71％ 43.97％ 4.11 Wide-angle exterior mirrors 20.43％ 128.25％ 6.28

[0070] As can be seen from Table 1, taking the driver's side wide-angle exterior mirror as an example, the effect of the change in the driver's field of view angle β on the magnification factor is 6.28 times the effect of the change in the object distance d on the magnification factor. Therefore, the effect of the change in the driver's field of view angle β on the magnification factor can be the main research object.

[0071] In the embodiment of the present application, the least square method can be used to fit multiple curves at different object distances, and the magnification coefficient can be expressed as M(β)=-0.6β 2 +0.001β+0.1.

[0072] The analysis of the glass mirror magnification factor is based on a transformation function obtained from a top-down view. This function is not directly applicable to the image magnification factor transformation of electronic rearview mirrors (fisheye cameras). A projection transformation is required to convert the relationship between the magnification factor M and the driver's field of view angle β from the top-down view into the relationship between pixels on the displayed image. Analysis of the transformation function shows that image magnification factor transformation effectively scales the image in the polar coordinates of the top-down view, maintaining the polar angle constant and applying the magnification factor to the polar direction.

[0073] Therefore, in this embodiment of the present application, a polar coordinate system can be constructed on the top view of the car, with the origin being the perpendicular point O between the glass mirror and the ground, the polar diameter being r, the polar angle being θ, the y-axis being parallel to the car body, and the x-axis being perpendicular to the car body. Assuming the angle between the line connecting the driver's eye point and the origin O and the y-axis is β0, the relationship between the magnification factor M and the polar angle θ can be calculated as:

[0074] M (θ) =-0.6(π / 2+β0-θ) 2+0.001(π / 2+β0-θ)+0.1 (6)

[0075] The change of the magnification coefficient in polar coordinates can be seen as the change in the relative position of the object and the origin, that is, the length of the polar longitude r is transformed. The polar longitude after the change is r′, and the following formula (7) is obtained.

[0076] r′=kM (θ) r (7)

[0077] Where k is a constant, let θ = 0, kM (θ) =1, combined with formula (7), we can know that the relationship between r and r′ is shown in formula (8).

[0078]

[0079] The coordinates of the center point of an object in polar coordinates are expressed in the following relation (9).

[0080]

[0081] Convert the polar coordinate system to the Cartesian coordinate system, and assume that the transformed coordinates of the point on the image are (x′, y′), then we have the following formula (10).

[0082]

[0083] However, the image relationship obtained from the top view cannot be directly used for the image captured by the camera. It needs to undergo a projection transformation to convert the relationship of the top view into the relationship of pixels on the image captured by the electronic rearview mirror.

[0084] According to the image processing of the fisheye camera, the following formula (11) can be obtained.

[0085]

[0086] Where, represents pixel coordinates, Represents the coordinates of a point in the world coordinate system, is the intrinsic parameter matrix of the fisheye camera.

[0087] Assume that the pixel coordinates of the changed image point are (u′, v′), then the pixel coordinates of the changed point can be expressed as:

[0088] That is, the correction formula for correcting the pixel coordinates of each pixel in the corrected image.

[0089] In summary, the image processing method provided in the embodiment of the present application corrects the distortion of the distorted image captured by the fisheye camera to obtain a corrected image with consistent magnification factors at all locations; then, based on the observation magnification factor of the glass viewfinder for objects at different angles, the pixel coordinates of each pixel point in the corrected image are corrected to obtain a corrected image; wherein, the observation magnification factor of the glass viewfinder for objects at different angles represents the ratio between the size of the object observed by the driver through the glass viewfinder at different angles and the size of the object actually observed by the driver. Since the distortion correction is first performed on the distorted image captured by the fisheye camera to ensure that the magnification coefficient of each part of the image is consistent, the magnification coefficient of each part of the image is close to the magnification coefficient of the glass viewfinder. Then, based on the observation magnification coefficient of the glass viewfinder for objects at different angles, the pixel coordinates of each pixel point in the corrected image are corrected. In this way, the corrected image is closer to the image directly observed by the naked eye, thereby achieving a better viewing effect and greatly improving driving safety. In addition, no complex calculations are required in this process. The distorted image obtained by the fisheye camera can be corrected under the premise that the computing power of the on-board electronic viewfinder system is limited, which is convenient for practical application and promotion.

[0090] See also Figure 2 According to a second aspect of an embodiment of the present application, an image processing device is provided for correcting a distorted image captured by a fisheye camera on a car. The image processing device includes:

[0091] A distortion correction unit is used to correct the distortion of the distorted image captured by the fisheye camera to obtain a corrected image with a consistent magnification factor at all locations;

[0092] A coordinate correction unit, configured to correct the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles, to obtain a corrected image;

[0093] Among them, the observation magnification coefficient of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver.

[0094] The working process, working details and technical effects of the device provided in the second aspect of this embodiment can be found in the first aspect of the embodiment and will not be repeated here.

[0095] like Figure 3 As shown, the third aspect of an embodiment of the present application provides another image processing device, comprising a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the image processing method as described in the first aspect of the embodiment.

[0096] For example, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out memory (FIFO) and / or first-in-last-out memory (FILO), etc.; the processor may be but is not limited to a microprocessor of the STM32F105 series, an ARM (Advanced RISC Machines), an X86 or other architecture processor, or a processor with an integrated NPU (neural-network processing units); the transceiver may be, but is not limited to, a WiFi (Wireless Fidelity) wireless transceiver, a Bluetooth wireless transceiver, a General Packet Radio Service (GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc.

[0097] A fourth aspect of this embodiment provides a computer-readable storage medium storing instructions containing the image processing method described in the first aspect of the embodiment, that is, the computer-readable storage medium stores instructions that, when executed on a computer, execute the image processing method described in the first aspect. The computer-readable storage medium refers to a data storage medium and may include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive, and / or a memory stick. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device.

[0098] The fifth aspect of this embodiment provides a computer program product containing instructions, which, when executed on a computer, causes the computer to execute the image processing method as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0099] It should be understood that certain details are provided in the following description to facilitate a thorough understanding of the example embodiments. However, one of ordinary skill in the art will appreciate that the example embodiments can be practiced without these specific details. For example, a system may be shown in block diagrams to avoid obscuring the example with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the example embodiments.

[0100] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. An image processing method for correcting a distorted image captured by a fisheye camera on a car, characterized in that: include: Perform distortion correction on the distorted image captured by the fisheye camera to obtain a corrected image with consistent magnification coefficients at all locations; Correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles to obtain a corrected image; Among them, the observation magnification coefficient of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver.

2. The image processing method according to claim 1, wherein: The method of correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification coefficient of the glass mirror for objects at different angles includes: Correcting the pixel coordinates of each pixel in the corrected image based on the magnification factor of the glass mirror for observing objects at different angles, the polar diameter corresponding to the projection point of each pixel in the corrected image on the plane coordinate system, and the angle between the polar diameter corresponding to each pixel in the corrected image and the x-axis in the plane coordinate system; The plane coordinate system is established with the direction parallel to the vehicle body as the y-axis, the direction perpendicular to the vehicle body as the x-axis, and the point where the fisheye camera is projected on the horizontal plane as the origin.

3. The image processing method according to claim 2, wherein: The method corrects the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass viewfinder for objects at different angles, the polar diameter corresponding to the projection point of each pixel point in the corrected image on the plane coordinate system, and the angle between the polar diameter corresponding to each pixel point in the corrected image and the x-axis in the plane coordinate system, including: Correct the pixel coordinates of each pixel point in the corrected image according to the following formula; Among them, m 00 、m 01 、m 10 、m 11 、m 20 and m 21 are all parameters in the intrinsic parameter matrix of the fisheye camera, k is a constant, r represents the polar radius corresponding to the projection point of any pixel point in the image on the plane coordinate system established with the y-axis parallel to the vehicle body, the x-axis perpendicular to the vehicle body, and the point where the fisheye camera is projected on the horizontal plane as the origin, θ represents the angle between the polar radius corresponding to any pixel point and the x-axis, and M (θ) It represents the observation magnification factor of the glass mirror for an object with a polar angle of θ in a plane coordinate system.

4. The image processing method according to claim 3, wherein: The observation magnification factor is M (θ) =-0.6(π / 2+β0-θ) 2 +0.001(π / 2+β0-θ)+0.1, where β0 represents the angle between the driver's projected sight line and the y-axis, and the driver's projected sight line is the line formed by the driver's eye point projected onto the plane coordinate system and the origin.

5. The image processing method according to claim 3, wherein: Before correcting the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles, the method further includes: A plane coordinate system is established with the direction parallel to the vehicle body as the y-axis, the direction perpendicular to the vehicle body as the x-axis, and the point where the fisheye camera is projected on the horizontal plane as the origin.

6. The image processing method according to claim 1, wherein: The distortion correction of the distorted image captured by the fisheye camera includes: The distortion images captured by the fisheye camera are corrected using Zhang Zhengyou's plane calibration method and polynomial model transformation algorithm.

7. An image processing device for correcting distorted images captured by a fisheye camera on a car, characterized in that: include: A distortion correction unit is used to correct the distortion of the distorted image captured by the fisheye camera to obtain a corrected image with a consistent magnification factor at all locations; A coordinate correction unit, configured to correct the pixel coordinates of each pixel point in the corrected image based on the observation magnification factor of the glass mirror for objects at different angles, to obtain a corrected image; Among them, the observation magnification coefficient of the glass mirror for objects at different angles represents the ratio between the size of the object observed by the driver through the glass mirror at different angles and the size of the object actually observed by the driver.

8. An image processing device, characterized in that: The device comprises a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the image processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the image processing method according to any one of claims 1 to 6 is executed.

Images