Aberration Index Determination Method, Device, Equipment and Storage Medium

Abstract

An embodiment of the present application discloses a method, device, equipment, and readable storage medium for determining a distortion index. The method includes: performing image capture on the inner surface of a test box using a fisheye camera to obtain a distortion test image; performing feature extraction processing on the distortion test image to obtain the color patch information features of different color patches on the distortion test image; according to the color patch information features, extracting pixel key points within different color patches, and sorting the pixel key points to obtain sorted key points corresponding to different color patches; processing the sorted key points based on a preset distortion model to obtain color patch distortion indexes corresponding to different color patches; and determining a target distortion index of the fisheye camera based on the color patch distortion indexes. The embodiment of the present application can improve the accuracy and efficiency of fisheye camera distortion prediction.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of camera distortion testing, and in particular to a method, device, equipment and storage medium for determining distortion index. Background Art

[0002] The vision technology in the intelligent Internet of Things industry has developed rapidly. Relying on wide-angle lenses and cameras, surround monitoring and vision devices can achieve the purpose of high information volume and low hardware investment.

[0003] Although wide-angle lenses can help expand the field of view, the problem of object distortion and deformation caused by radial distortion is also relatively obvious, which greatly reduces the adaptability of intelligent recognition algorithms (such as face recognition, tracking, license plate detection, etc.). This situation is particularly serious for fisheye lenses.

[0004] The shooting angle of fisheye lenses is mostly about 180°, or even more than 180°. Most current typical products are used for surround shooting and special video shooting. Considering the balance between wide angle and geometric distortion, it is necessary to evaluate the geometric distortion state of the lens and perform correction based on the distortion state.

[0005] The current distortion test method usually involves shooting geometric distortion images (taking a grid image as an example), making the image fill the complete view angle as much as possible, using image segmentation to automatically extract the basic content of the image (grid lines), then performing feature dimensionality reduction on the extracted targets (coordinates of grid line intersections), fitting a radial high-order curve based on the grid line coordinates, such as 3rd order, 5th order, etc., and then using the fitting curve coefficients as evaluation indicators.

[0006] However, the above distortion test method has insufficient accuracy in image content extraction, which causes calculation errors of intersection coordinates in the evaluation model, resulting in low distortion evaluation accuracy. And there are infinite points in the 180° field of view, which may cause the failure of high-order curve fitting and lead to the inability to evaluate the distortion degree. Summary of the Invention

[0007] The embodiments of the present application provide a method, device, equipment and storage medium for determining camera distortion, which can be used to improve the accuracy and efficiency of fisheye camera distortion testing. The technical solutions are as follows:

[0008] On the one hand, the embodiments of the present application provide a method for determining distortion index, the method includes:

[0009] An image of the inner surface of a test box is captured by a fisheye camera to obtain a distorted test image. The test box is a cube structure. An image acquisition hole is reserved on the first side of the test box. The inner surfaces of the other five sides of the test box except the first side are surfaces with two colors alternately and evenly arranged. The two colors include white and other colors except white. And the colors of the two opposite inner surfaces of the test box are the same. The RGB values printed in the dark areas of the five inner surfaces include three kinds, which are: (0, 0, 255), (255, 0, 0) and (0, 0, 0). The distorted test image is an image including the other colors on the five inner surfaces.

[0010] Feature extraction processing is performed on the distorted test image to obtain the color block information features of different color blocks on the distorted test image.

[0011] According to the color block information features, pixel key points in different color blocks are extracted, and the pixel key points are sorted to obtain sorted key points corresponding to different color blocks.

[0012] Based on a preset distortion model, the sorted key points are processed to obtain color block distortion indexes corresponding to different color blocks.

[0013] Based on the color block distortion indexes, the target distortion index of the fisheye camera is determined.

[0014] On the other hand, an embodiment of the present application provides a device for determining a distortion index. The device includes:

[0015] A distorted test image acquisition module, configured to capture an image of the inner surface of a test box by a fisheye camera to obtain a distorted test image. The test box is a cube structure. An image acquisition hole is reserved on the first side of the test box. The inner surfaces of the other five sides of the test box except the first side are surfaces with two colors alternately and evenly arranged. The two colors include white and other colors except white. And the colors of the two opposite inner surfaces of the test box are the same. The RGB values printed in the dark areas of the five inner surfaces include three kinds, which are: (0, 0, 255), (255, 0, 0) and (0, 0, 0). The distorted test image is an image including the other colors on the five inner surfaces.

[0016] A color block information feature acquisition module, configured to perform feature extraction processing on the distorted test image to obtain the color block information features of different color blocks on the distorted test image.

[0017] A sorted key point acquisition module, configured to extract pixel key points in different color blocks according to the color block information features, and sort the pixel key points to obtain sorted key points corresponding to different color blocks.

[0018] The color patch distortion index acquisition module is configured to process the sorted key points based on a preset distortion model to obtain the color patch distortion indexes corresponding to different color patches;

[0019] The target distortion index determination module is configured to determine the target distortion index of the fisheye camera based on the color patch distortion indexes.

[0020] In another aspect, an embodiment of the present application provides a computer device, which includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method described in the above aspect.

[0021] In yet another aspect, an embodiment of the present application provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method described in the above aspect.

[0022] In yet another aspect, an embodiment of the present application provides a computer program product, which is used to execute the method described in the above aspect when the computer program product is executed.

[0023] In the technical solution provided by the embodiment of the present application, by providing a special test box body, the problem of low evaluation accuracy caused by low feature extraction accuracy or even feature extraction failure at the infinite far point of the boundary can be eliminated, and the accuracy of distortion evaluation can be improved. Moreover, by adopting the solution provided by this embodiment, only one image needs to be captured to complete the distortion evaluation process, and the evaluation efficiency is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flowchart of the steps of a distortion index determination method provided by an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of a test box body provided by an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of an unfolded surface of a test box body provided by an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of the inner surface imaging of a test box body provided by an embodiment of the present application;

[0028] Figure 5 is a schematic diagram of a distortion test image provided by an embodiment of the present application;

[0029] Figure 6It is a schematic structural diagram of a distortion index determination device provided by an embodiment of the present application;

[0030] Figure 7 It is a structural block diagram of a computer device provided by an embodiment of the present application. Specific implementation manners

[0031] The following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0032] Referring to Figure 1 , a step flowchart of a distortion index determination method provided by an embodiment of the present application is shown. As Figure 1 shown, the distortion index determination method may include the following steps:

[0033] Step 101: Based on a fisheye camera, take an image of the inner surface of the test box to obtain a distortion test image.

[0034] The embodiments of the present application can be applied to a scenario of testing the geometric distortion degree of a fisheye camera.

[0035] This embodiment provides a dedicated test box, which can be combined with Figure 2 and 3 and will be described in detail as follows.

[0036] Referring to Figure 2 , a schematic diagram of a test box provided by an embodiment of the present application is shown. As Figure 2 shown, the test box is a cube structure, and the length, width, and height parameters of the test box can be based on the focal plane distance of the fisheye camera. For example, if the focal plane of the fisheye camera is 0.5 m, then the length, width, and height of the test box are 0.5 m, etc. An image acquisition hole is reserved on one side surface (i.e., the first side surface) of the test box, and the image acquisition hole can be used for the fisheye camera to acquire an image of the inner surface of the box.

[0037] Referring to Figure 3 , a schematic diagram of an unfolded surface of a test box provided by an embodiment of the present application is shown. As Figure 3 shown, the inner surfaces of the other five side surfaces of the test box except the first side surface are surfaces with two colors alternately and evenly arranged. The two colors include white and other colors except white, and the colors of the inner surfaces facing each other of the test box are the same. The RGB values of the dark areas printed on the five inner surfaces include three types, which are: (0, 0, 255), (255, 0, 0), and (0, 0, 0). The imaging image of each inner surface can be as Figure 4 shown.

[0038] The distortion test image is an image including the other colors on the five inner surfaces, as Figure 3As shown, the style of the inner surface can be a checkerboard style, a dot pattern style, a grid style, etc. Specifically, it can be determined according to business requirements, and this embodiment does not limit this. A camera fixing device is also provided on the first side to fix the fish-eye camera when the fish-eye camera takes pictures, an external power supply interface to provide electric energy, and a light source adjustment device is also provided to adjust the light source when the fish-eye camera collects images of the inner surface, so as to avoid overexposure of the captured images. The light source of the standard surface (i.e., the first side) needs to have a fixed color temperature, and the brightness is controlled by a hardware control circuit, supporting adjustable brightness. The typical color temperature is 6500K, and the brightness can be controlled at 200 - 2000 lux. The camera fixing device needs to be controllably retractable according to the size of the camera device. Specifically, the controllable retraction method can adopt conventional methods, and this embodiment does not limit this.

[0039] When using a fish-eye camera to take pictures of the inner surface of the test box, the fish-eye camera can be placed on the fixing device, the light source is turned on, the light source is adjusted until there is no obvious overexposure in the picture taken by the fish-eye camera, the position of the fish-eye camera is adjusted to ensure that each checkerboard (or grid or dot pattern, etc.) in the field of view can be seen clearly. At the same time, the visual axis direction needs to be adjusted to the center of the checkerboard (or grid or dot pattern, etc.) being photographed and the fish-eye camera is fixed to ensure that the fish-eye camera does not move during the shooting process. Then the shooting process is executed to obtain a distorted test image. After the shooting is completed, the power supply of the light source can be turned off and the fixation of the fish-eye camera is released.

[0040] The image taken by the fish-eye camera can be as Figure 5 shown. The captured image simultaneously includes the inner surfaces of other sides except the first side, so that a distorted test image including the inner surfaces of five sides can be formed.

[0041] After obtaining the distorted test image by taking pictures of the inner surface of the test box based on the fish-eye camera, step 102 is executed.

[0042] Step 102: Perform feature extraction processing on the distorted test image to obtain the color patch information features of different color patches on the distorted test image.

[0043] After obtaining the distorted test image, the features of different color patches in the distorted test image can be extracted to obtain the color patch information features of different color patches on the distorted test image. The process of obtaining the color patch information can be described in detail in combination with the following specific implementation manner.

[0044] In a specific implementation manner of the present application, the above step 102 may include:

[0045] Sub-step A1: Perform clustering processing on the image pixels of the distorted test image based on a preset clustering algorithm.

[0046] In this embodiment, after obtaining the distorted test image, the image pixels of the distorted test image can be clustered based on a preset clustering algorithm. Specifically, the clustering process can be as follows: clustering can be performed using the RGB color information of the distorted test image, and a typical K-means clustering algorithm can be used for clustering. First, the clustering target can be 4 categories, and the initial clustering centers are respectively: (255, 255, 255), (255, 0, 0), (0, 0, 0), (0, 255, 0).

[0047] After clustering the image pixels of the distorted test image based on the preset clustering algorithm, sub-step A2 is executed.

[0048] Sub-step A2: According to the clustering result, perform binarization processing on the distorted test image to generate a binary image.

[0049] After clustering the image pixels of the distorted test image based on the preset clustering algorithm, the distorted test image can be binarized according to the clustering result to obtain a binary image. Specifically, all pixel positions whose clustering centers are close to any one of other colors can be screened, and the original image (i.e., the distorted test image) is binarized. The processing method is as follows: all pixel positions classified into any one color are set to 1, and other pixel positions are set to 0.

[0050] After generating a binary image by binarizing the distorted test image according to the clustering result, sub-step A3 is executed.

[0051] Sub-step A3: Perform morphological opening processing on the binary image to obtain a processed image.

[0052] Morphological opening processing refers to performing erosion and then dilation on an image to make the contour of the image smooth, and it can also break narrow connections on the image and eliminate thin burrs, etc.

[0053] After obtaining the binary image, morphological opening processing can be performed on the binary image to obtain a processed image. Then, sub-step A4 is executed.

[0054] Sub-step A4: Obtain the convex hull in the processed image, and determine the color block information features of different color blocks according to the contour of the convex hull.

[0055] Convex Hull is a concept in computational geometry (graphics). Given a set of points in a two-dimensional plane, the convex hull is a convex polygon formed by connecting the outermost points, which contains all the points in the click.

[0056] After obtaining the processed image, each convex hull after processing can be calculated, and the color patch information features of different color patches can be determined according to the contour of the convex hull, that is, the contour of the convex hull is the color patch information feature of different color patches.

[0057] After performing feature extraction processing on the distorted test image to obtain the color patch information features of different color patches on the distorted test image, step 103 is executed.

[0058] Step 103: According to the color patch information features, extract the pixel key points within different color patches, and sort the pixel key points to obtain the sorted key points corresponding to different color patches.

[0059] After obtaining the color patch information features of different color patches on the distorted test image, the pixel key points within different color patches can be extracted according to the color patch information features of different color patches, and the pixel key points can be sorted in the row and column manner to obtain the sorted key points corresponding to different color patches. Specifically, it can be described in detail in combination with the following specific implementation manners.

[0060] In another specific implementation manner of the present application, the above step 103 may include:

[0061] Sub-step B1: Calculate the color patch center point of the color patch with the same color according to the color patch information features of the color patch with the same color.

[0062] In this embodiment, after obtaining the color patch information features of different color patches on the distorted test image, for the information features of the same color patch, the color patch center point of each color patch can be calculated. The specific calculation method can refer to the following formula (1):

[0063]

[0064] In the above formula (1), is the color patch center point, i ∈ [1, 100], representing the color patch label.

[0065] After calculating the color patch center point of the color patch with the same color according to the color patch information features of the color patch with the same color, sub-step B2 is executed.

[0066] Sub-step B2: Obtain the key points on the color patch with the same color.

[0067] After extracting the color patch information features of different color patches, the key points on the color patch with the same color can be obtained.

[0068] After obtaining the key points on the color patch with the same color, sub-step B3 is executed.

[0069] Sub-step B3: Determine the rotation key points among the key points based on the Euclidean distance between the key points and the center points of the color patches.

[0070] After obtaining the center points of the color patches and the key points of the same color, the Euclidean distance between the key points and the center points of the color patches can be calculated, and the rotation key points among the key points can be determined according to the Euclidean distance. Specifically, 1 key point (x t , y t ) on the color patch surface that is farthest from the center point of the color patch can be selected as the rotation key point.

[0071] After determining the rotation key points among the key points based on the Euclidean distance between the key points and the center points of the color patches, execute sub-step B4.

[0072] Sub-step B4: Perform rotation correction processing on the key points on all color patch surfaces based on the rotation key points.

[0073] After obtaining the rotation key points, rotation correction can be performed on the key points on all color patch surfaces based on the rotation key points. Specifically, the key points on each color patch surface can be rotationally corrected in combination with the rotation key points on that color patch surface.

[0074] Perform correction on the key points on all surfaces with θ as the rotation angle, as shown in the following formula (2):

[0075]

[0076] After performing rotation correction processing on the key points on all color patch surfaces based on the rotation key points, execute sub-step B5.

[0077] Sub-step B5: Obtain the correction center point and the correction key points according to the correction result.

[0078] After performing rotation correction processing on the key points on all color patch surfaces based on the rotation key points, the correction center point and the correction key points on the color patch surface can be obtained according to the correction result.

[0079] After obtaining the correction center point and the correction key points, execute sub-step B6.

[0080] Sub-step B6: Based on the correction center point, obtain the key point position information of the correction key points in the row direction and the column direction.

[0081] Sub-step B7: Determine the sorting key points based on the key point position information.

[0082] After obtaining the calibration center point, based on the calibration center point, the key point position information of the calibration key points in the row direction and column direction can be obtained, and the sorted key points can be determined based on the key point position information. Specifically, based on the calibrated central coordinates (x c ′, y c ′), in the central row direction, the calibrated key point set closest to the y direction is filtered out. Then, the key point closest to the origin is searched for in the positive y-axis direction and labeled as (0, 1). Then, the calibrated point closest to the key point (0, 1) is searched for in the same direction and labeled as (0, 2), and so on until the search in the positive direction is completed; in the same way, in the negative y-axis direction, key points are searched for and their coordinates are defined as (0, -1), (0, -2),... Similarly, the key point positions in the x-axis direction can also be confirmed in this way; the four closest points are found;

[0083] Through the above calculations, four quadrants of the coordinate system can be formed. For each quadrant, first, the key point position closest to the origin in the 45-degree direction is found and defined. Then, taking the found key point as a reference point, in the same search method as above, the key points in the x and y directions of the reference point are found and defined. Then, the point positions in different reference directions in the first quadrant are found in turn; similarly, the key points in the second, third, and fourth quadrants can be found and defined; after this step of processing, the calibrated coordinates of all key points on the vertical plane and the position information of each key point in the row and column directions can be obtained. Based on this position information, all calibrated key points can be sorted to obtain the sorted key points.

[0084] After extracting the pixel key points in different color blocks according to the color block information characteristics and sorting the pixel key points to obtain the sorted key points corresponding to different color blocks, step 104 is executed.

[0085] Step 104: Process the sorted key points based on a preset distortion model to obtain the color block distortion indexes corresponding to different color blocks.

[0086] The color block distortion index refers to the distortion index on the color blocks of different colors. This color block distortion index can be used to indicate the distortion degree of the color block surface. The larger the color block distortion index, the higher the distortion degree of the color block surface.

[0087] After obtaining the sorted key points corresponding to different color blocks, the sorted key points can be processed based on a preset distortion model to obtain the color block distortion indexes corresponding to different color blocks. Specifically, the methods for obtaining the color block distortion indexes in the visual axis direction, horizontal axis direction, and vertical axis direction can be combined with the following three methods.

[0088] 1. For the color block distortion index in the direction perpendicular to the visual axis

[0089] The sorting key points and calibration key points in the direction perpendicular to the optical axis can be processed based on the SMTA TV distortion model to obtain the color block distortion index of the color blocks in the direction perpendicular to the optical axis. Specifically, for the point information in the direction perpendicular to the optical axis, according to the SMIA TV distortion model and based on the calibrated point set and the row and column definition relationships, the basic distortion rate is calculated, that is, the color block distortion index in the direction perpendicular to the optical axis.

[0090] 2. For the color block distortion index in the horizontal axis direction

[0091] The sorting key points and calibration key points in the direction of the side of the horizontal axis can be processed based on the radial distortion model to obtain the color block distortion index of the color blocks in the direction of the side of the horizontal axis. Specifically, for the point information in the direction of the side of the horizontal axis, according to the radial basis distortion model and based on the calibrated point set and the row and column definition relationships, the basic distortion rate is calculated. The center point still takes (x c , y c ) as the standard (that is, based on the calibrated center point).

[0092] 3. For the color block distortion index in the vertical axis direction

[0093] The sorting key points and calibration key points in the direction of the side of the vertical axis can be obtained, and the sorting key points and calibration key points in the direction of the side of the vertical axis are processed by horizontal and vertical axis conversion. Then, based on the radial distortion model, the sorted key points and calibrated key points after the conversion process are processed to obtain the color block distortion index of the color blocks in the vertical axis direction. Specifically, for the point information in the up and down direction of the vertical axis, considering the model solving process, the calibrated point set and the row and column definition relationships are permuted in the horizontal and vertical axes, that is, (x r ′, y r ′) = (y′, x′), and then the basic distortion rate is calculated through the radial basis distortion model. The center point takes (y c , x c ) as the standard.

[0094] After obtaining the color block distortion indices in the three directions, step 105 is executed.

[0095] Step 105: Based on the color block distortion index, determine the target distortion index of the fish-eye camera.

[0096] After obtaining the color patch distortion indices in three directions, the target distortion index of the fish-eye camera can be determined based on the color patch distortion indices. Specifically, the maximum index value among the three color patch distortion indices can be obtained and used as the target distortion index, or the sum value of the distortion indices of the three color patch distortion indices can be obtained and used as the target distortion index, etc. Specifically, the method for obtaining the target distortion index can be determined according to the service requirements, and this embodiment does not limit it.

[0097] The distortion index determination method provided by the embodiments of the present application obtains a distortion test image by capturing an image of the inner surface of a test box using a fish-eye camera. The test box is a square box, and an image acquisition hole is reserved on the first side of the test box. The inner surfaces of the other five sides of the test box except the first side are surfaces uniformly arranged with two alternating colors. The two colors include white and other colors except white, and the colors of the two opposite inner surfaces of the test box are the same. The RGB values printed in the dark areas of the five inner surfaces include three types, namely: (0, 0, 255), (255, 0, 0), and (0, 0, 0). The distortion test image is an image including the other colors on the five inner surfaces. Feature extraction processing is performed on the distortion test image to obtain the color patch information features of different color patches on the distortion test image. According to the color patch information features, pixel key points within different color patches are extracted, and the pixel key points are sorted to obtain the sorted key points corresponding to different color patches. Based on a preset distortion model, the sorted key points are processed to obtain the color patch distortion indices corresponding to different color patches. Based on the color patch distortion indices, the target distortion index of the fish-eye camera is determined. By providing a special test box, the embodiments of the present application can eliminate the problem of low feature extraction accuracy at the infinite far point of the boundary or even feature extraction failure, resulting in low evaluation accuracy, and can improve the accuracy of distortion evaluation. Moreover, adopting the solution provided by this embodiment, only one image needs to be captured to complete the distortion evaluation process, and the evaluation efficiency is relatively high.

[0098] The following is an embodiment of the apparatus of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the embodiment of the apparatus of the present application, please refer to the method embodiment of the present application.

[0099] Refer to Figure 6 , which shows a schematic structural diagram of a distortion index determination device provided by an embodiment of the present application. As Figure 6 shown, the distortion index determination device 600 may include the following modules:

[0100] The distortion test image acquisition module 610 is configured to capture an image of the inner surface of the test box using a fisheye camera to obtain a distortion test image. The test box is a cube structure. An image acquisition hole is reserved on the first side of the test box. The inner surfaces of the other five sides of the test box except the first side are surfaces with two colors alternately and evenly arranged. The two colors include white and other colors except white. And the colors of the two opposite inner surfaces of the test box are the same. The RGB values printed in the dark areas of the five inner surfaces include three kinds, which are: (0, 0, 255), (255, 0, 0), and (0, 0, 0). The distortion test image is an image including the other colors on the five inner surfaces.

[0101] The color block information feature acquisition module 620 is configured to perform feature extraction processing on the distortion test image to obtain the color block information features of different color blocks on the distortion test image.

[0102] The sorting key point acquisition module 630 is configured to extract pixel key points in different color blocks according to the color block information features and sort the pixel key points to obtain the sorting key points corresponding to different color blocks.

[0103] The color block distortion index acquisition module 640 is configured to process the sorting key points based on a preset distortion model to obtain the color block distortion indexes corresponding to different color blocks.

[0104] The target distortion index determination module 650 is configured to determine the target distortion index of the fisheye camera based on the color block distortion indexes.

[0105] Optionally, the color block information feature acquisition module includes:

[0106] The pixel clustering processing unit is configured to perform clustering processing on the image pixels of the distortion test image based on a preset clustering algorithm.

[0107] The binary image generation unit is configured to perform binary processing on the distortion test image according to the clustering result to generate a binary image.

[0108] The processed image acquisition unit is configured to perform morphological opening processing on the binary image to obtain a processed image.

[0109] The color block information feature determination unit is configured to obtain the convex hull in the processed image and determine the color block information features of different color blocks according to the contour of the convex hull.

[0110] Optionally, the sorting key point acquisition module includes:

[0111] The color block center point calculation unit is used to calculate the color block center point of the color blocks of the same color according to the color block information characteristics of the color blocks of the same color;

[0112] The key point acquisition unit is used to acquire the key points on the color blocks of the same color;

[0113] The rotation key point determination unit is used to determine the rotation key points among the key points based on the Euclidean distance between the key points and the color block center point;

[0114] The rotation correction processing unit is used to perform rotation correction processing on the key points on all color block surfaces based on the rotation key points;

[0115] The correction point acquisition unit is used to acquire the correction center point and correction key points according to the correction result;

[0116] The key point position acquisition unit is used to acquire the key point position information of the correction key points in the row direction and column direction based on the correction center point;

[0117] The sorting key point determination unit is used to determine the sorting key points based on the key point position information;

[0118] Optionally, the color block distortion index acquisition module includes:

[0119] The visual axis distortion index acquisition unit is used to process the sorted key points and correction key points in the direction perpendicular to the visual axis based on the SMTA TV distortion model to obtain the color block distortion index of the color blocks in the direction perpendicular to the visual axis.

[0120] Optionally, the color block distortion index acquisition module includes:

[0121] The horizontal axis distortion index acquisition unit is used to process the sorted key points and correction key points in the horizontal axis side direction based on the radial distortion model to obtain the color block distortion index of the color blocks in the horizontal axis side direction.

[0122] Optionally, the color block distortion index acquisition module includes:

[0123] The correction key point acquisition unit is used to acquire the sorted key points and correction key points in the vertical axis direction;

[0124] The key point correction processing unit is used to perform horizontal and vertical axis conversion processing on the sorted key points and correction key points in the vertical axis direction;

[0125] The vertical axis distortion index acquisition unit is used to process the sorted key points and correction key points after conversion processing based on the radial distortion model to obtain the color block distortion index of the color blocks in the vertical axis direction.

[0126] Optionally, the target distortion index determination module includes:

[0127] A first target distortion index acquisition unit, configured to acquire the sum value of the color patch distortion indices, and use the sum value as the target distortion index.

[0128] Optionally, the target distortion index determination module includes:

[0129] A second target distortion index acquisition unit, configured to acquire the color patch distortion index with the largest index value among the color patch distortion indices, and use the color patch distortion index with the largest index value as the target distortion index.

[0130] The distortion index determination device provided by the embodiment of the present application obtains a distortion test image by performing image shooting on the inner surface of a test box body based on a fish-eye camera. The test box body is a square box body, and an image acquisition hole is reserved on the first side of the test box body. The inner surfaces of the other five sides of the test box body except the first side are surfaces uniformly arranged with two colors alternating. The two colors include white and other colors except white, and the colors of the two inner surfaces facing each other of the test box body are the same. The RGB values printed in the dark areas of the five inner surfaces include three types, which are: (0, 0, 255), (255, 0, 0), and (0, 0, 0). The distortion test image is an image including the other colors on the five inner surfaces. Feature extraction processing is performed on the distortion test image to obtain the color patch information features of different color patches on the distortion test image. According to the color patch information features, pixel key points in different color patches are extracted, and the pixel key points are sorted to obtain the sorted key points corresponding to different color patches. Based on a preset distortion model, the sorted key points are processed to obtain the color patch distortion indices corresponding to different color patches. Based on the color patch distortion indices, the target distortion index of the fish-eye camera is determined. The embodiment of the present application provides a special test box body to eliminate the problem of low evaluation accuracy caused by low feature extraction accuracy or even feature extraction failure at the infinite far point of the boundary, and can improve the accuracy of distortion evaluation. Moreover, by adopting the solution provided by this embodiment, only one image needs to be taken to complete the distortion evaluation process, and the evaluation efficiency is relatively high.

[0131] It should be noted that, when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the method embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be elaborated here.

[0132] Please refer to Figure 7, which shows the structural block diagram of a computer device provided by an embodiment of the present application. This computer device can be used to implement the training method of the face key point localization model provided in the above embodiment. The computer device can be a PC or a server, or other devices with data processing and storage capabilities. Specifically:

[0133] The computer device 700 includes a central processing unit (CPU) 701, a system memory 704 including a random access memory (RAM) 702 and a read-only memory (ROM) 703, and a system bus 705 connecting the system memory 704 and the central processing unit 701. The computer device 700 also includes a basic input / output system (I / O system) 706 for facilitating information transmission between various components within the computer, and a mass storage device 707 for storing an operating system 713, application programs 714, and other program modules 715.

[0134] The basic input / output system 706 includes a display 708 for displaying information and input devices 709 such as a mouse and a keyboard for user input. Among them, both the display 708 and the input devices 709 are connected to the central processing unit 701 through an input / output controller 710 connected to the system bus 705. The basic input / output system 706 may also include an input / output controller 710 for receiving and processing inputs from multiple other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 710 also provides output to a display screen, a printer, or other types of output devices.

[0135] The mass storage device 707 is connected to the central processing unit 701 through a mass storage controller (not shown) connected to the system bus 705. The mass storage device 707 and its associated computer-readable medium provide non-volatile storage for the computer device 700. That is to say, the mass storage device 707 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0136] Without loss of generality, the computer-readable medium may include a computer storage medium and a communication medium. The computer storage medium includes volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. The computer storage medium includes RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, magnetic tape cartridges, tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will know that the computer storage medium is not limited to the above several types. The above system memory 704 and mass storage device 707 can be collectively referred to as memory.

[0137] According to various embodiments of the present application, the computer device 700 can also run on a remote computer on the network connected through a network such as the Internet. That is, the computer device 700 can be connected to the network 712 through the network interface unit 711 connected to the system bus 705. Or rather, the network interface unit 711 can also be used to connect to other types of networks or remote computer systems (not shown).

[0138] The memory further includes one or more programs, the one or more programs are stored in the memory, and are configured to be executed by one or more processors. The above one or more programs include instructions for executing the training method of the above face key point localization model.

[0139] In an exemplary embodiment, a computer device is further provided. The computer device includes a processor and a memory, and at least one instruction, at least one segment of program, code set or instruction set is stored in the memory. The at least one instruction, at least one segment of program, code set or instruction set is configured to be executed by one or more processors to implement the above distortion index determination method.

[0140] In an exemplary embodiment, a computer-readable storage medium is further provided. At least one instruction, at least one segment of program, code set or instruction set is stored in the storage medium, and when being executed by a processor of a computer device, the at least one instruction, the at least one segment of program, the code set or the instruction set implement the above distortion index determination method.

[0141] Optionally, the above computer-readable storage medium can be ROM, RAM, CD-ROM, magnetic tape, floppy disk and optical data storage devices, etc.

[0142] In an exemplary embodiment, a computer program product is further provided. When the computer program product is executed, it is used to implement the above training method of the face key point localization model.

[0143] It should be understood that the "plurality" mentioned herein refers to two or more. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0144] The above are only exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

Claims

1. A method for determining a distortion index, characterized in that, The method includes: Performing image capture on the inner surface of the test box using a fish-eye camera to obtain a distorted test image; the test box is a cube structure, an image acquisition hole is reserved on the first side of the test box, the inner surfaces of the other five sides of the test box except the first side are surfaces uniformly arranged with two alternating colors, the two colors include white and other colors except white, and the colors of the two opposite inner surfaces of the test box are the same. The RGB values printed in the dark areas of the five inner surfaces include three kinds, which are: (0, 0, 255), (255, 0, 0), and (0, 0, 0). The distorted test image is an image including the other colors on the five inner surfaces; Performing feature extraction processing on the distorted test image to obtain the color patch information features of different color patches on the distorted test image; According to the color patch information features, extracting pixel key points in different color patches and sorting the pixel key points to obtain sorted key points corresponding to different color patches; Processing the sorted key points based on a preset distortion model to obtain color patch distortion indices corresponding to different color patches; Based on the color patch distortion indices, determining the target distortion index of the fish-eye camera.

2. The method according to claim 1, characterized in that, The performing feature extraction processing on the distorted test image to obtain the color patch information features of different color patches on the distorted test image includes: Performing clustering processing on the image pixels of the distorted test image based on a preset clustering algorithm; According to the clustering result, performing binaryzation processing on the distorted test image to generate a binary image; Performing morphological opening processing on the binary image to obtain a processed image; Obtaining the convex hull in the processed image and determining the color patch information features of different color patches according to the contour of the convex hull.

3. The method according to claim 2, wherein The extracting pixel key points in different color patches according to the color patch information features and sorting the pixel key points to obtain sorted key points corresponding to different color patches includes: Calculating the color patch center points of the color patches with the same color according to the color patch information features of the color patches with the same color; Obtaining the key points on the color patches with the same color; Based on the Euclidean distance between the key points and the color patch center points, determining the rotation key points among the key points; Performing rotation correction processing on the key points on all color patch surfaces based on the rotation key points; According to the correction result, obtaining the correction center point and correction key points; Based on the correction center point, obtaining the key point position information of the correction key points in the row direction and column direction; Based on the key point position information, determining the sorted key points.

4. The method according to claim 3, wherein The processing the sorted key points based on a preset distortion model to obtain color patch distortion indices corresponding to different color patches includes: Processing the sorted key points and correction key points in the direction perpendicular to the optical axis based on the SMTA TV distortion model to obtain the color patch distortion indices of the color patches in the direction perpendicular to the optical axis.

5. The method according to claim 3, characterized in that, The processing the sorted key points based on a preset distortion model to obtain color patch distortion indices corresponding to different color patches includes: Process the sorted key points and corrected key points in the horizontal side direction based on the radial distortion model to obtain the color block distortion index of the color blocks in the horizontal side direction.

6. The method according to claim 3, characterized in that, The process of processing the sorted key points based on the preset distortion model to obtain the color block distortion index corresponding to different color blocks includes: Obtain the sorted key points and corrected key points in the vertical axis direction; Perform horizontal-vertical axis conversion processing on the sorted key points and corrected key points in the vertical axis direction; Process the sorted key points and corrected key points after the conversion processing based on the radial distortion model to obtain the color block distortion index of the color blocks in the vertical axis direction.

7. The method according to claim 1, wherein The process of determining the target distortion index of the fish-eye camera based on the color block distortion index includes: Obtain the sum value of the color block distortion index and use the sum value as the target distortion index.

8. The method according to claim 1, wherein The process of determining the target distortion index of the fish-eye camera based on the color block distortion index includes: Obtain the color block distortion index with the largest index value among the color block distortion indexes, and use the color block distortion index with the largest index value as the target distortion index.

9. A distortion index determination device, characterized in that, The device includes: A distortion test image acquisition module, configured to capture an image of the inner surface of a test box using a fish-eye camera to obtain a distortion test image; the test box is a cube structure, an image acquisition hole is reserved on the first side of the test box, the inner surfaces of the other five sides of the test box except the first side are surfaces uniformly arranged with two alternating colors, the two colors include white and other colors except white, and the colors of the two opposite inner surfaces of the test box are the same, and the RGB values printed in the dark areas of the five inner surfaces include three, namely: (0, 0, 255), (255, 0, 0), and (0, 0, 0), and the distortion test image is an image including the other colors on the five inner surfaces; A color block information feature acquisition module, configured to perform feature extraction processing on the distortion test image to obtain the color block information features of different color blocks on the distortion test image; A sorted key point acquisition module, configured to extract pixel key points in different color blocks according to the color block information features and sort the pixel key points to obtain the sorted key points corresponding to different color blocks; A color block distortion index acquisition module, configured to process the sorted key points based on a preset distortion model to obtain the color block distortion index corresponding to different color blocks; A target distortion index determination module, configured to determine the target distortion index of the fish-eye camera based on the color block distortion index.

10. A computer device, characterized in that, The computer device includes a processor and a memory, and at least one instruction, at least one program, a code set, or an instruction set is stored in the memory, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the distortion index determination method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, At least one instruction, at least one program, a code set, or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the distortion index determination method according to any one of claims 1 to 8.

Images