Image processing method and image processing apparatus

Abstract

Disclosed in the present invention is an image processing method, comprising: acquiring a target image to be processed, calibration parameters of an image acquisition device and chromatic aberration correction parameters; on the basis of the calibration parameters, calculating an included angle θg between the connecting line of an object corresponding to each pixel in the target image and the optical center of a lens and the main optical axis; on the basis of the chromatic aberration correction parameters and the included angle θg, calculating a correction included angle θr of an R channel and a correction included angle θb of a B channel of each pixel; and performing reprojection of an R channel component and a B channel component of each pixel on the basis of θr and θb respectively.

Description

Image processing method and image processing device Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and an image processing device. Background Art

[0002] When consumer camera imaging systems use fisheye lenses to capture digital images, the image is focused on points on the focal plane at different distances from the principal optical axis. This causes color fringing perpendicular to the principal optical axis in the imaged object, a phenomenon known as lateral chromatic aberration. Lens manufacturers attempt to eliminate lateral chromatic aberration by stacking multiple lenses, but this is often impossible to completely eliminate in consumer camera lenses, and the remaining color fringing can vary in color, making the effect less than ideal.

[0003] Summary of the Invention

[0004] The main purpose of the present invention is to provide an image processing method, device and storage medium, aiming to solve the problems of lateral chromatic aberration of images and poor image processing quality and effect in the prior art.

[0005] To achieve the above object, the present invention provides an image processing method, which includes:

[0006] Obtaining the target image to be processed, calibration parameters of the image acquisition device, and chromatic aberration correction parameters;

[0007] Calculate the angle θ between the line connecting the object and the optical center of the lens and the main optical axis corresponding to each pixel in the target image according to the calibration parameters g ;

[0008] According to the chromatic aberration correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ;

[0009] The R channel component and B channel component of each pixel are respectively calculated according to the θ r and the θ b Reproject.

[0010] To achieve the above object, the present invention further provides an image processing device, comprising:

[0011] An acquisition module is used to obtain the target image to be processed, the calibration parameters of the image acquisition device, and the chromatic aberration correction parameters;

[0012] The first calculation module is used to calculate the angle θ between the line connecting the object and the lens optical center corresponding to each pixel in the target image and the main optical axis according to the calibration parameters. g ;

[0013] The second calculation module is used to calculate the chromatic aberration correction parameter and the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ;

[0014] The projection module is used to project the R channel component and the B channel component of each pixel according to the θ r and the θ b Reproject.

[0015] The image processing method, device, and computer-readable storage medium provided by the present invention have the following advantages over the prior art: obtaining a target image to be processed, calibration parameters of an image acquisition device, and chromatic aberration correction parameters; and calculating, based on the calibration parameters, the angle θ between the line connecting the object and the optical center of the lens and the principal optical axis corresponding to each pixel in the target image. g According to the chromatic aberration correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ; The R channel component and the B channel component of each pixel are respectively calculated according to the θ r and the θ b Through the above method, based on the calibration parameters and chromatic aberration correction parameters of the image acquisition device, combined with the characteristics and performance of the image acquisition device itself, the image processing to eliminate lateral chromatic aberration can be performed more accurately. g Calculation, and based on θ g Determine the correction angles of the R channel and the B channel, and perform image processing correction through data calculation. This can not only accurately improve the image correction processing effect, but also reduce the image processing cost without adding additional structures, and improve the image processing effect and image processing cost-effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG1 is a schematic diagram of the system structure of the hardware operating environment involved in the embodiment of the present invention;

[0017] FIG2 is a flow chart of an embodiment of an image processing method according to the present invention;

[0018] FIG3 is a diagram showing the principle of lateral chromatic aberration in an embodiment of the present invention;

[0019] FIG. 4 is an example diagram of a RAW image obtained in step S50 in an embodiment of the image processing method of the present invention.

[0020] FIG5 is a schematic diagram of obtaining corner points of a corresponding channel graph having an object boundary area. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] In the prior art, when an imaging system uses a fisheye lens to capture images, lateral chromatic aberration is prone to occur, which affects the image quality and effect.

[0024] In order to solve the above technical problems, the present invention provides a testing method. In this method, based on the calibration parameters and chromatic aberration correction parameters of the image acquisition device, the image processing to eliminate lateral chromatic aberration is combined with the characteristics and performance of the image acquisition device itself, which can more accurately eliminate lateral chromatic aberration. g Calculation, and based on θ g Determine the correction angles of the R channel and the B channel, and perform image processing correction through data calculation. This can not only accurately improve the image correction processing effect, but also reduce the image processing cost without adding additional structures, and improve the image processing effect and image processing cost-effectiveness.

[0025] As shown in FIG1 , FIG1 is a schematic diagram of the system structure of the hardware operating environment involved in the embodiment of the present invention.

[0026] The terminal of the embodiment of the present invention can be a terminal device with computing capabilities, or it can be a PC, or it can be a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3) player, an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) player, a portable computer, and other mobile terminal devices with display capabilities.

[0027] As shown in Figure 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0028] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the mobile terminal can also be configured with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0029] Those skilled in the art will understand that the terminal structure shown in FIG1 does not constitute a limitation on the terminal, and may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components.

[0030] As shown in FIG1 , the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an image processing program.

[0031] In the terminal shown in FIG1 , the network interface 1004 is primarily used to connect to a backend server and perform data communications with the backend server; the user interface 1003 is primarily used to connect to a client (user end) and perform data communications with the client; and the processor 1001 can be used to call an image processing program stored in the memory 1005 and perform the following operations:

[0032] Obtaining the target image to be processed, calibration parameters of the image acquisition device, and chromatic aberration correction parameters;

[0033] Calculate the angle θ between the line connecting the object and the optical center of the lens and the main optical axis corresponding to each pixel in the target image according to the calibration parameters g ;

[0034] According to the chromatic aberration correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ;

[0035] The R channel component and B channel component of each pixel are respectively calculated according to the θ r and the θ b Reproject.

[0036] Furthermore, the processor 1001 may call the image processing program stored in the memory 1005 and perform the following operations:

[0037] Read the pre-stored θ g Calculation formula: θ g =atan(sqrt((XC x )2+(YC y )2) / f);

[0038] According to the g The calculation formula and the image center coordinates C x ,C y , and the focal length f is calculated as θ g .

[0039] Furthermore, the processor 1001 may call the image processing program stored in the memory 1005 and perform the following operations:

[0040] The color difference correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b The steps include:

[0041] Read the pre-stored θ r Calculation formula and θ b Calculation formula: θ r =r1*θ g 7 +r2*θ g 5 +r3*θ g 3 +r4*θ g θ b=b1*θ g 7 +b2*θ g 5 +b3*θ g 3 +b4*θ g

[0042] According to θ r Calculation formula, θ g and r1, r2, r3, r4 to calculate θ r ; According to θ b Calculation formula, θ g and b1, b2, b3, b4 to calculate θ.

[0043] Furthermore, the processor 1001 may call the image processing program stored in the memory 1005 and perform the following operations:

[0044] Using an image acquisition device to capture a RAW image of a preset target object;

[0045] Extracting channel images of the three channels R, G, and B of the Bayer array based on the RAW image;

[0046] Match the feature points of the R channel image and the B channel image with the G channel image respectively;

[0047] According to the formula θ=atan(sqrt((XC x )2+(YC y )2) / f) Calculate the R channel map, B channel map, and G channel map respectively to obtain the θ of each feature point r ,θ b and θ g ;

[0048] Based on the calculated θ r ,θ b and θ g The color difference correction parameters of the R channel and the B channel are calculated using a preset least squares fitting formula, wherein the least squares fitting formula is: fi=θ g -(b1*θ bi 7 +b2*θ bi 5 +b3*θ bi 3 +b4*θ bi ) fi=θ g -(r1*θ ri 7 +r2*θ ri 5+r3*θ ri 3 +r4*θ ri ).

[0049] Furthermore, the processor 1001 may call the image processing program stored in the memory 1005 and perform the following operations:

[0050] Move the R channel image, B channel image and G channel image with a preset moving step length through the preset acquisition frame;

[0051] Calculating the grayscale change value of the image captured before and after the preset capture frame moves;

[0052] The feature points are determined according to the grayscale change values ​​moving in each direction, and the feature points of the R channel image, the B channel image, and the G channel image are matched.

[0053] Furthermore, the processor 1001 may call the image processing program stored in the memory 1005 and perform the following operations:

[0054] Count the number of times the preset acquisition frame moves;

[0055] The grayscale of the image in the preset acquisition frame is detected at a preset interval and the grayscale change value is calculated.

[0056] Furthermore, the processor 1001 may call the image processing program stored in the memory 1005 and perform the following operations:

[0057] Extracting two G channel images of the Bayer array based on the RAW image;

[0058] Calculate the average brightness of the two G channel images;

[0059] A final G channel map is determined based on the brightness average value.

[0060] Referring to Figure 2, Figure 2 is a flow chart of an embodiment of an image processing method of the present invention. The image processing method based on the present invention can be applied to application scenarios such as stitching or post-processing software of consumer-grade panoramic cameras or VR180 cameras. As shown in Figure 3, Figure 3 is a schematic diagram of the principle of lateral chromatic aberration in the implementation of the present invention. The cause of lateral chromatic aberration is that light of different wavelengths has different refractive indices. When passing through the lens together, they focus on points on the focal plane at different distances from the main optical axis, resulting in colored edges on the imaged object in the direction perpendicular to the main optical axis. In some embodiments, the image processing method includes:

[0061] Step S10: obtaining the target image to be processed, calibration parameters of the image acquisition device, and chromatic aberration correction parameters.

[0062] The target image refers to the image to be processed to eliminate lateral chromatic aberration. It can be an image captured by a camera in real time, an image saved locally after capture, or an image received through a network. The image acquisition device is the image acquisition device that captures the target image. The calibration parameters refer to the factory calibration parameters of the image acquisition device, including the coordinates of the image center point C x ,C y , and focal length f, etc. The chromatic aberration correction parameters are parameters determined based on the image processing method of the present invention, and are used to correct the image pixels based on the pixel G channel, the pixel R channel, and the pixel B channel. They can include four chromatic aberration correction parameters r1, r2, r3, r4 of the R channel, and four chromatic aberration correction parameters b1, b2, b3, b4 of the B channel, wherein the chromatic aberration correction parameters and the calibration parameters of the image acquisition device are predetermined parameters, which can be pre-stored in a preset position and read from the preset position when performing image processing. In some embodiments, the corresponding calibration parameters and the corresponding chromatic aberration correction parameters can be collected in advance according to the model of the image acquisition device, and the model of the image acquisition device can be associated with the corresponding parameters to generate an association table. Subsequently, when the image processing method of the present invention is used for image processing, the model of the image acquisition device is read, and the corresponding parameters are searched from the association table according to the read model. The specific method for determining the chromatic aberration correction parameters will be described in detail later.

[0063] Step S20, calculating the angle θ between the line connecting the object and the lens optical center corresponding to each pixel in the target image and the principal optical axis according to the calibration parameters g .

[0064] As mentioned above, in some embodiments, the calibration parameters of the image acquisition device include the image center point coordinates C x , C y , and focal length f. Step S20 specifically includes:

[0065] Step S21, read the pre-stored θ g Calculation formula: θ g =atan(sqrt((XC x )2+(YC y )2) / f);

[0066] where θ g is the angle between the line connecting the object and the lens' optical center and the principal optical axis for each pixel, which is also the angle between the pixel's G channel; X and Y are the pixel's coordinates. This calculation formula is pre-set and stored in a preset location, and is directly read from the formula's storage location during step S21.

[0067] Step S22, according to the θ g The calculation formula and the image center coordinates Cx ,C y , and the focal length f is calculated as θ g .

[0068] In obtaining θ g After the calculation formula, the coordinates (X, Y) of each pixel of the target image and the coordinates of the center point C x , C y Substitute the focal length f into the formula to calculate the θ corresponding to each pixel g By using the above method, the θ of each pixel can be quickly and accurately calculated. g .

[0069] Step S30, according to the color difference correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b .

[0070] The present invention is based on the pixel G channel, and corrects the pixel R channel and the pixel B channel, and aligns the pixel R channel and the pixel B channel with the pixel G channel. Therefore, according to the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b Specifically, as described above, the color difference correction parameters include four color difference correction parameters r1, r2, r3, and r4 of the R channel, and four color difference correction parameters b1, b2, b3, and b4 of the B channel. In some embodiments, step S30 includes:

[0071] Step S31, read the pre-stored θ r Calculation formula and θ b Calculation formula: θ r =r1*θ g 7 +r2*θ g 5 +r3*θ g 3 +r4*θ g θ b =b1*θ g 7 +b2*θ g 5 +b3*θ g 3 +b4*θ g

[0072] Step S32, according to θ r Calculation formula, θ g and r1, r2, r 3，r4 calculates θ r ; According to θ b Calculation formula, θ g and b1, b2, b3, b4 to calculate θ b .

[0073] Specific θ r Calculation formula and θ b The calculation formula is predetermined according to the imaging principle and experimental experience and is pre-stored in a preset position. When executing step S31, the calculation formulas of the two are directly read from the corresponding preset position, and the color difference correction parameters obtained, that is, the four color difference correction parameters r1, r2, r3, r4 of the R channel and the four color difference correction parameters b1, b2, b3, b4 of the B channel, and the θ of each pixel calculated in the above steps are calculated. g Substitute into the above formula to calculate the θ of each pixel r and θ b The above θ r and θ b The calculation method of θ for each pixel can be quickly and accurately calculated through the pre-stored formula r and θ b Perform calculations.

[0074] Step S40: The R channel component and the B channel component of each pixel are respectively calculated according to the θ r and the θ b Reproject.

[0075] After obtaining the θ of each pixel r and θ b Then, according to the θ of each pixel r and θ b , the R channel component and G channel component of each pixel are calculated according to θ r and θ b Reprojection is performed, that is, the R channel and B channel of the pixel are reprojected to the new angle respectively, so that the image can be corrected and the lateral chromatic aberration is eliminated.

[0076] In the above image processing method, the target image to be processed, the calibration parameters of the image acquisition device and the chromatic aberration correction parameters are obtained; the angle θ between the line connecting the object and the optical center of the lens corresponding to each pixel in the target image and the principal optical axis is calculated based on the calibration parameters. g According to the chromatic aberration correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ; The R channel component and the B channel component of each pixel are respectively calculated according to the θ r and the θ bThrough the above method, based on the calibration parameters and chromatic aberration correction parameters of the image acquisition device, combined with the characteristics and performance of the image acquisition device itself, the image processing to eliminate lateral chromatic aberration can be performed more accurately. g Calculation, and based on θ g Determine the correction angles of the R channel and the B channel, and perform image processing correction through data calculation. This can not only accurately improve the image correction processing effect, but also reduce the image processing cost without adding additional structures, and improve the image processing effect and image processing cost-effectiveness.

[0077] In some embodiments, the process before step S10 includes:

[0078] Step S50: Using an image acquisition device to capture a RAW image of a preset target object.

[0079] For each image acquisition device, such as a camera, each camera uses the same chromatic aberration correction parameters. Calibration of the chromatic aberration correction parameters is required for each new camera. This calibration must be completed before executing step S10 of the image processing method of the present invention. For details on the calibration method, refer to the contents of this embodiment.

[0080] Specifically, the preset target object is an object being photographed and a shooting environment, etc., designed for calibrating chromatic aberration correction parameters. In some embodiments, the preset target object is a hemispherical light box emitting uniform white light, with striped stickers affixed to various locations within the light box, evenly distributed in all directions. Specifically, the camera is first placed within a hemispherical light box emitting uniform white light, and striped stickers are affixed to various locations within the light box, evenly distributed in all directions. The camera then captures a RAW image, as shown in FIG4 , which is an example of a RAW image obtained in step S50 of an embodiment of the image processing method of the present invention.

[0081] Step S60 , extracting channel images of the three channels R, G, and B of the Bayer array based on the RAW image.

[0082] Specifically, in some embodiments, step S60 includes:

[0083] Step S61, extracting two G channel images of the Bayer array based on the RAW image;

[0084] Step S62, calculating the average brightness of the two G channel images;

[0085] Step S63: determining a final G channel image based on the brightness average value.

[0086] After obtaining the RAW image, we first extract the R, G, and B channel maps of the Bayer pattern. The extracted Bayer pattern consists of four channels: one R channel, two G channels, and one B channel. Since there is no positional deviation between the two G channels, to generate a single G channel map for feature matching, we average the brightness values ​​of the two G channels. The final G channel map is determined based on the average brightness of the two G channel maps, ultimately obtaining a single channel map for each of the R, G, and B channels.

[0087] Step S70 , performing feature point matching on the R channel image, the B channel image, and the G channel image respectively.

[0088] Specifically, in some embodiments, step S70 includes:

[0089] Step S71 : moving the R channel image, the B channel image, and the G channel image with a preset moving step length through a preset acquisition frame.

[0090] Specifically, feature point matching in this embodiment primarily detects feature points in one or more of the R, B, and G channel images, and then matches the feature points. The preset capture frame is an image capture frame of a set size, used to capture images by moving along a specific path and at a specific step size. Feature points are detected based on the grayscale change values ​​of the image captured by the preset capture frame.

[0091] Step S72: Calculate the grayscale change value of the image captured before and after the preset capture frame moves to find the corner points in the G channel image, the R channel image, and the B channel image.

[0092] First, we assume that the grayscale changes in the areas on both sides of the object boundary are large. Therefore, when the image acquisition frame moves from the outside of the object boundary (such as frame A1) to the inside of the object boundary (such as frame A2), or from the inside of the object boundary (such as frame A2) to the outside of the object boundary (such as frame A1), the average grayscale value of all pixels in the image acquisition frame will change dramatically. This is shown in Figure 5.

[0093] If the image acquisition frame moves inside the object's boundaries (near the A1 frame area) or outside the object's boundaries (near the A2 frame area), the average grayscale value of all pixels within the image acquisition frame remains essentially unchanged or changes very little. This movement can be done by setting the number of pixels in any direction, such as left, right, down, or up.

[0094] Similarly, when the image acquisition frame moves on the boundary of the object, the average grayscale value of all pixels in the image acquisition frame after moving along the extension direction of the object boundary hardly changes (such as from frame A7 to frame A7'), and the average grayscale value of all pixels in the image acquisition frame after moving in the direction perpendicular to the object boundary changes greatly (such as from frame A8 to frame A8'). Therefore, the image acquisition frames A1, A2, A7 and A8 cannot achieve the goal that when the image acquisition frame moves in all directions, the average grayscale value of all pixels in the image acquisition frame after moving changes significantly relative to the average grayscale value of all pixels in the image acquisition frame before moving.

[0095] The image acquisition frames A3, A4, A5, and A6 can achieve that when the image acquisition frames move in various directions, the average grayscale value of all pixels in the image acquisition frames after movement changes significantly relative to the average grayscale value of all pixels in the image acquisition frames before movement. There will be no situation where the average grayscale value of all pixels in the image acquisition frames hardly changes when the image acquisition frames move along a certain direction (the extension direction of the object boundary). Therefore, we set the image acquisition frames A3, A4, A5, and A6 as the image acquisition frames with the largest change in moving grayscale value.

[0096] In this way, the image acquisition frame with the largest change in moving grayscale value in the G channel image can be found through the above method. Here, the center point of the image acquisition frame with the largest change in moving grayscale value can be set as the corner point of the corresponding channel image. In this way, all corner points in the R channel image, B channel image, and G channel image can be obtained.

[0097] Step S73: Match the corner points in the G channel image with the feature points in the B channel image and the R channel image.

[0098] Here, we can first obtain the matching feature points in the R channel image based on the feature points (corner points) detected in the G channel image, and then obtain the matching feature points in the B channel image based on the feature points (corner points) detected in the G channel image.

[0099] Specifically, a G comparison pixel range (such as a 5*5 pixel range) can be set around each corner point of the G channel image, and then the R comparison pixel range corresponding to the R channel image is obtained (the corner point position of the G channel image is in the corresponding range of the R channel image). The R comparison pixel range should be larger than the G comparison pixel range (such as a 10*10 pixel range). In this way, multiple R comparison pixel ranges can be obtained in the R comparison pixel range of the R channel image by translation, and the pixel grayscale of each R comparison pixel range is compared with the corresponding pixel grayscale of the G comparison pixel range one by one. The sum of the square differences of all pixel grayscales of each R comparison pixel range and the G comparison pixel range is obtained, and the R comparison pixel range with the smallest sum of the square differences is determined to match the corresponding G comparison pixel range; and the center point of the R comparison pixel range is set as the preliminary feature point that matches the corner point in the G channel image.

[0100] For example, if the R comparison pixel range includes pixels R1, R2, ..., and R25, and the G comparison pixel range includes pixels G1, G2, ..., and G25, then obtain the grayscale difference RG1 between pixels R1 and G1, the grayscale difference RG2 between pixels R2 and G2, ..., and the grayscale difference RG25 between pixels R25 and G23. Then, match the R comparison pixel range with the smallest sum of the squares of RG1, RG2, ..., and RG25 to the corresponding G comparison pixel range. The center point of the R comparison pixel range is set as the preliminary feature point that matches the corner point in the G channel image in the R channel image.

[0101] Then, the prepared feature point is compared with all corner points in the R channel image. When the distance between the feature point and a corner point in the R channel image is less than a set value (such as 3 pixels), the prepared feature point is considered to be the feature point that matches the corner point in the G channel image in the R channel image. Otherwise, the matching is considered wrong and the corresponding prepared feature point is discarded. Similarly, the feature point that matches the corner point in the G channel image in the B channel image is obtained.

[0102] Finally, the least squares method is used to fit the other unmatched corner points in the G channel image in the R channel image and the B channel image, and finally the feature points corresponding to all the corner points in the G channel image in the R channel image and the B channel image are obtained.

[0103] Step S80, according to the formula θ=atan(sqrt((XC x )2+(YC y )2) / f) Calculate the R channel map, B channel map, and G channel map respectively to obtain the θ of each feature point r ,θ b and θ g ;

[0104] Step S90, based on the calculated θ r ,θ b and θg The color difference correction parameters of the R channel and the B channel are calculated using a preset least squares fitting formula, wherein the least squares fitting formula is: fi=θ g -(b1*θ bi 7 +b2*θ bi 5 +b3*θ bi 3 +b4*θ bi ) f i =θ g -(r1*θ ri 7 +r2*θ ri 5 +r3*θ ri 3 +r4*θ ri )

[0105] Among them, fi is the angular deviation function between the G channel and the B channel or the R channel; Z is the minimum value of the angular color difference between the G channel and the B channel or the R channel; i is the identifier of the feature point.

[0106] After obtaining a preset number of matching feature points of the G channel image, the B channel image, and the R channel image, the R channel chromatic aberration correction parameters r1, r2, r3, r4 and the B channel chromatic aberration correction parameters b1, b2, b3, b4 that minimize the difference in angular chromatic aberration of all feature points are calculated based on the above-mentioned least squares fitting formula.

[0107] In the above image processing method, a RAW image of a preset target object is captured by an image acquisition device; channel images of the three channels R, G, and B of the Bayer array are extracted based on the RAW image; feature points of the R channel image and the B channel image are matched with the G channel image respectively; according to the formula θ=atan(sqrt((XC x )2+(YC y )2) / f) Calculate the R channel map, B channel map, and G channel map respectively to obtain the θ of each feature point r ,θ b and θ g ; Based on the calculated θ r ,θ b and θ g The color difference correction parameters of the R and B channels are calculated using the preset least squares fitting formula. In this way, accurate and reliable color difference correction parameters of the R and B channels can be obtained, thereby improving the image correction quality.

[0108] In addition, the present invention also provides an image processing device.

[0109] The image processing device of the present invention includes:

[0110] An acquisition module is used to obtain the target image to be processed, the calibration parameters of the image acquisition device, and the chromatic aberration correction parameters;

[0111] The first calculation module is used to calculate the angle θ between the line connecting the object and the lens optical center corresponding to each pixel in the target image and the main optical axis according to the calibration parameters. g ;

[0112] The second calculation module is used to calculate the chromatic aberration correction parameter and the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ;

[0113] The projection module is used to project the R channel component and the B channel component of each pixel according to the θ r and the θ b Reproject.

[0114] The specific implementation of the image processing device of the present invention can refer to the various embodiments of the image processing method of the present invention, and will not be repeated here.

[0115] In addition, an embodiment of the present invention further provides a computer-readable storage medium.

[0116] The computer-readable storage medium of the present invention stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method described above are implemented.

[0117] The method implemented when the image processing program running on the processor is executed can refer to the various embodiments of the image processing method of the present invention, and will not be described in detail here.

[0118] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0119] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0120] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0121] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image processing method, comprising: Obtaining the target image to be processed, calibration parameters of the image acquisition device, and chromatic aberration correction parameters; Calculate the angle θ between the line connecting the object and the optical center of the lens and the main optical axis corresponding to each pixel in the target image according to the calibration parameters g ; According to the chromatic aberration correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ; The R channel component and B channel component of each pixel are respectively calculated according to the θ r and the θ b Perform reprojection; The chromatic aberration correction parameters are obtained by the following steps: Using an image acquisition device to capture a RAW image of a preset target object; Extracting channel images of the three channels R, G, and B of the Bayer array based on the RAW image; Match the feature points of the R channel image and the B channel image with the G channel image respectively; According to the formula θ=atan(sqrt((XC x )2+(YC y )2) / f) Calculate the R channel map, B channel map, and G channel map respectively to obtain the θ of each feature point r ,θ b and θ g ; Based on the calculated θ r ,θ b and θ g The color difference correction parameters of the R channel and the B channel are calculated using a preset least squares fitting formula, wherein the least squares fitting formula is: fi=θ g -(b1*θ bi 7 +b2*θ bi 5 +b3*θ bi 3 +b4*θ bi ) fi=θ g -(r1*θ ri 7 +r2*θ ri 5 +r3*θ ri 3 +r4*θ ri ); The angle θ between the line connecting the object and the optical center of the lens and the main optical axis corresponding to each pixel in the target image is calculated according to the calibration parameters. g The steps include: Read the pre-stored θ g Calculation formula: θ g =here(sqrt((XC x )2+(YC y )2) / f); According to the g The calculation formula and the image center coordinates C x , C y , and the focal length f is calculated as θ g ; The calibration parameters of the image acquisition device include the coordinates of the image center point C x , C y , and focal length f; wherein the color difference correction parameter, the angle θ g Calculate the R channel of each pixel Correction angle θ r And the correction angle θ of channel B b The steps include: Read the pre-stored θ r Calculation formula and θ b Calculation formula: i r =r1*θ g 7 +r2*θ g 5 +r3*θ g 3 +r4*θ g ; i b =b1*θ g 7 +b2*θ g 5 +b3*θ g 3 +b4*θ g ; According to θ r Calculation formula, θ g and r1, r2, r3, r4 to calculate θ r ; According to θ b Calculation formula, θ g and b1, b2, b3, b4 to calculate θ b , wherein the color difference correction parameters include four color difference correction parameters r1, r2, r3, r4 of the R channel, and four color difference correction parameters b1, b2, b3, b4 of the B channel.

2. An image processing method, comprising: Obtaining the target image to be processed, calibration parameters of the image acquisition device, and chromatic aberration correction parameters; Calculate the angle θ between the line connecting the object and the optical center of the lens and the main optical axis corresponding to each pixel in the target image according to the calibration parameters g ; According to the chromatic aberration correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ; The R channel component and B channel component of each pixel are respectively calculated according to the θ r and the θ b Reproject.

3. The image processing method according to claim 2, wherein the calibration parameters of the image acquisition device include the coordinates C of the image center point x , C y , and focal length f; the angle θ between the line connecting the object and the optical center of the lens and the principal optical axis corresponding to each pixel in the target image is calculated according to the calibration parameters g The steps include: Read the pre-stored θ g Calculation formula: θ g =here(sqrt((XC x )2+(YC y )2) / f); According to the g The calculation formula and the image center coordinates C x , C y , and the focal length f is calculated as θ g .

4. The image processing method according to claim 2, wherein the color difference correction parameters include four color difference correction parameters r1, r2, r3, and r4 of the R channel, and four color difference correction parameters b1, b2, b3, and b4 of the B channel; The color difference correction parameter, the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b The steps include: Read the pre-stored θ r Calculation formula and θ b Calculation formula: i r =r1*θ g 7 +r2*θ g 5 +r3*θ g 3 +r4*θ g ; i b =b1*θ g 7 +b2*θ g 5 +b3*θ g 3 +b4*θ g ; According to θ r Calculation formula, θ g and r1, r2, r3, r4 to calculate θ r ; According to θ b Calculation formula, θ g as well as b1, b2, b3, b4 calculate θ b .

5. The image processing method according to claim 2, wherein before the step of obtaining the target image to be processed, the calibration parameters of the image acquisition device, and the chromatic aberration correction parameters, the method further comprises: Using an image acquisition device to capture a RAW image of a preset target object; Extracting channel images of the three channels R, G, and B of the Bayer array based on the RAW image; Match the feature points of the R channel image and the B channel image with the G channel image respectively; According to the formula θ=atan(sqrt((XC x )2+(YC y )2) / f) Calculate the R channel map, B channel map, and G channel map respectively to obtain the θ of each feature point r ,θ b and θ g ; Based on the calculated θ r ,θ b and θ g The color difference correction parameters of the R channel and the B channel are calculated using a preset least squares fitting formula, wherein the least squares fitting formula is: fi=θ g -(b1*θ bi 7 +b2*θ bi 5 +b3*θ bi 3 +b4*θ bi ) fi=θ g -(r1*θ ri 7 +r2*θ ri 5 +r3*θ ri 3 +r4*θ ri )。 6. The image processing method according to claim 5, wherein the step of performing feature point matching on the R channel image and the B channel image respectively with the G channel image comprises: Move the R channel image, B channel image and G channel image with a preset moving step length through the preset acquisition frame; Calculate the grayscale change value of the image collected before and after the preset acquisition frame moves to find the corner points in the G channel image, the R channel image, and the B channel image; Match the corner points in the G channel image with the feature points in the B channel image and the R channel image.

7. The image processing method according to claim 6, wherein the step of calculating the grayscale change value of the image captured before and after the preset capture frame is moved to find the corner points in the G channel image, the R channel image, and the B channel image comprises: The center point of the image acquisition frame with the largest change in moving grayscale value is taken as the corner point of the corresponding channel image.

8. The image processing method according to claim 5, wherein the step of extracting channel images of the three channels R, G, and B of the Bayer array based on the RAW image comprises: Extracting two G channel images of the Bayer array based on the RAW image; Calculate the average brightness of the two G channel images; A final G channel map is determined based on the brightness average value.

9. The image processing method according to claim 5, wherein the preset target object is: a hemispherical light box emitting uniform white light, with striped stickers affixed to various locations inside the light box, and the stickers are evenly distributed in all directions.

10. An image processing device, comprising: An acquisition module is used to obtain the target image to be processed, the calibration parameters of the image acquisition device, and the chromatic aberration correction parameters; The first calculation module is used to calculate the angle θ between the line connecting the object and the lens optical center corresponding to each pixel in the target image and the main optical axis according to the calibration parameters. g ; The second calculation module is used to calculate the chromatic aberration correction parameter and the angle θ g Calculate the correction angle θ of the R channel of each pixel r And the correction angle θ of channel B b ; The projection module is used to project the R channel component and the B channel component of each pixel according to the θ r and the θ b Reproject.

Images