Camera internal reference calibration method and device and electronic equipment

Abstract

The invention discloses a camera internal reference calibration method and device and electronic equipment, and the method comprises the steps: collecting an image in a current scene, carrying out the recognition of a distorted boundary pixel point of a linear structure in the collected image, and obtaining the boundary position information of the linear structure; sampling boundaries of the linear structure in the horizontal direction and the vertical direction according to the boundary position information, fitting sampling points to obtain a mean value position of the boundaries, and taking the mean value position as distortion centers in the horizontal direction and the vertical direction; constructing a fitness function based on the error sum of squares, and calculating to obtain a global optimal solution of the distortion coefficient according to the fitness function; and extracting parallel line identification information in the current scene, and obtaining a focal length closed solution based on the relative position relationship of the parallel lines. According to the technical scheme, the calibration process is simplified, the internal reference calibration of the camera is completed only depending on the linear features in the road scene, the camera can conduct self-adaptive calibration in the dynamic scene, and the calibration precision and stability are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a camera intrinsic parameter calibration method, device and electronic equipment. Background Art

[0002] With the rapid development of intelligent driving technology, the precise perception and decision-making capabilities of autonomous driving systems have become core requirements for achieving safe and stable driving. Stereo vision systems, as an important visual sensing method, can acquire distance information from a scene through two cameras, providing autonomous vehicles with a three-dimensional understanding of the vehicle's surroundings, enabling them to understand the distances of the environment. This is particularly critical in complex traffic environments. In scenarios such as highways and urban roads, accurate environmental modeling and analysis using line features (such as lane lines, road edges, and traffic signs) is crucial for autonomous driving positioning, path planning, and obstacle avoidance.

[0003] In the related art, camera intrinsic calibration mainly relies on physical calibration plates. However, this type of calibration has certain limitations, mainly reflected in the complex operation process, high environmental requirements, and relatively high maintenance costs.

[0004] Therefore, providing a camera intrinsic parameter calibration solution that is simple, easy to implement, and does not rely on a physical calibration board is an urgent problem to be solved. Summary of the Invention

[0005] The main purpose of the present invention is to disclose a camera intrinsic parameter calibration method, device and electronic equipment, so as to at least solve the problems of camera intrinsic parameter calibration schemes in related technologies that rely on physical calibration plates, have complex operating procedures, high environmental requirements, and relatively high maintenance costs.

[0006] According to one aspect of the present invention, a camera intrinsic parameter calibration method is provided.

[0007] The camera intrinsic parameter calibration method provided by the present invention includes: capturing an image in a current scene, identifying boundary pixel points of a distorted linear structure in the captured image, and obtaining boundary position information of the linear structure; sampling the boundary of the linear structure in the horizontal and vertical directions based on the boundary position information, and fitting the sampled points to obtain the mean position of the boundary as the distortion center in the horizontal and vertical directions; constructing a fitness function based on the sum of squared errors, and calculating and obtaining a global optimal solution for the distortion coefficient based on the fitness function; extracting parallel line identification information in the current scene, and obtaining a focal length closed solution based on the relative position relationship of the parallel lines.

[0008] Furthermore, the boundary pixel points of the distorted linear structure in the collected image are identified, and the boundary position information of the linear structure is obtained, including: using the semantic segmentation structure of the Mobile-Seed network to identify the area where the distorted linear structure exists in the collected image; using the boundary detection structure of the Mobile-Seed network to extract the boundary pixel points of the distorted linear structure in the collected image, and obtaining the boundary position information of the linear structure.

[0009] Furthermore, based on the boundary position information, the boundaries of the linear structure in the horizontal and vertical directions are sampled, and the sampling points are fitted to obtain the mean position of the boundaries as the distortion center in the horizontal and vertical directions, including: sampling the boundaries of the linear structure in the horizontal and vertical directions, obtaining multiple discrete sampling points as input data for Gaussian curve fitting; and obtaining the mean position of the curve after the linear structure is distorted by performing Gaussian curve fitting on the sampling points, as the distortion center in the horizontal and vertical directions.

[0010] Furthermore, constructing a fitness function based on the sum of squared errors, and calculating and obtaining a global optimal solution of the distortion coefficient according to the fitness function includes:

[0011] The fitness function Fitness is constructed by the following formula:

[0012]

[0013] in, is the number of image points after sampling distortion correction, is the coordinate of the dedistorted point after the i-th distortion correction, is the straight line obtained by least square fitting the coordinates of the dedistorted points, is the slope of the fitted straight line, is the intercept of the fitted straight line;

[0014] When the value of the fitness function Fitness is minimum, the radial distortion coefficients c1, c2 and the tangential distortion coefficients d1, d2 are taken as the global optimal solution of the distortion coefficients.

[0015] Furthermore, when the value of the fitness function Fitness is minimum, the radial distortion coefficients c1, c2 and the tangential distortion coefficients d1, d2 are taken as the global optimal solution of the distortion coefficients, including:

[0016] S1: Sampling the boundaries of distorted linear structures in the image to obtain sampling points, and applying random initial radial distortion coefficients c1, c2 and tangential distortion coefficients d1, d2 to perform distortion correction;

[0017] S2: For the distortion-corrected sampling points, the straight line obtained by least square fitting the sampling point coordinates ;

[0018] S3: Substitute the coordinates of the n sampling points after distortion correction, the values ​​of m and b into the fitness function formula to calculate the value of the fitness function;

[0019] S4: Loop through S1 to S3, dynamically adjusting c1, c2, d1, and d2 each time. Through continuous iterative search, when the value of the fitness function Fitness is minimized, the corresponding radial distortion coefficients c1 and c2 and tangential distortion coefficients d1 and d2 are taken as the global optimal solution for the distortion coefficients.

[0020] Furthermore, parallel line identification information is extracted in the current scene, and a focal length closed solution is obtained based on the relative position relationship of the parallel lines, including: extracting one or more groups of parallel lines in the current scene according to specific identifications; for each group of parallel lines, obtaining a unique solution of the projective depth corresponding to the multiple endpoints of the group of parallel lines according to the image point coordinates corresponding to the multiple endpoints of the group of parallel lines; obtaining a focal length closed solution corresponding to the group of parallel lines according to the unique solution of the projective depth, the distortion center, and the image point coordinates; and determining a final focal length closed solution by combining the calculation results of the focal length closed solutions corresponding to the one or more groups of parallel lines.

[0021] Furthermore, for each group of parallel lines, according to the image point coordinates corresponding to the multiple endpoints of the group of parallel lines, obtaining a unique solution for the projective depth corresponding to the multiple endpoints of the group of parallel lines includes: constructing a binocular stereo vision system, setting the origin of the world coordinate system to be located at the optical center of one of the binocular eyes; substituting the coordinates of the multiple endpoints of the group of parallel lines in the world coordinate system and the image point coordinates corresponding to the multiple endpoints into the projective equation; finding the correlation between the projective depth and the image point coordinates according to the projective equation to obtain a constraint equation; setting the projective depth to be positive, and all valid three-dimensional points in the image to be located in front of the camera, and when the image point coordinates are known, obtaining a unique solution for the projective depth corresponding to the multiple endpoints by solving the null space of the matrix composed of the image point coordinates.

[0022] Furthermore, obtaining a closed-form focal length solution corresponding to the group of parallel lines according to the unique projective depth solution, the distortion center position, and the image point coordinates includes:

[0023] The focal length closed-form solution f is obtained by the following formula:

[0024]

[0025] in, , and They are vectors and No. elements, , is the distortion center, , are the projective depths of the two endpoints of a line segment in the set of parallel lines, are the projected depths of the two endpoints of another line segment of the set of parallel lines that is of equal length to the line segment, are the image point coordinates corresponding to the two endpoints of the line segment, are the image point coordinates corresponding to the two endpoints of the other line segment.

[0026] According to another aspect of the present invention, a camera intrinsic parameter calibration device is provided.

[0027] The camera intrinsic parameter calibration device according to the present invention includes: an identification module for capturing images in a current scene, identifying boundary pixels of distorted linear structures in the captured images, and obtaining boundary position information of the linear structures; a distortion center acquisition module for sampling the boundaries of the linear structures in the horizontal and vertical directions based on the boundary position information, and fitting the sampled points to obtain mean positions of the boundaries as the distortion center positions in the horizontal and vertical directions; a distortion coefficient acquisition module for constructing a fitness function based on the sum of squared errors, and calculating a global optimal solution for the distortion coefficients based on the fitness function; and a focal length acquisition module for extracting parallel line identification information in the current scene and obtaining a closed-form focal length solution based on the relative positional relationship of the parallel lines.

[0028] According to yet another aspect of the present invention, an electronic device is provided.

[0029] An electronic device according to the present invention includes: a processor and a memory, wherein the memory is used to store executable instructions of the processor, wherein the processor is configured to perform any of the above-mentioned camera intrinsic parameter calibration methods by executing the executable instructions.

[0030] The present invention proposes a camera intrinsic parameter calibration method that does not require a physical calibration plate. By capturing images in the current scene, the linear structures in the current scene (e.g., road scenes) (e.g., road markings, building edges, etc.) are directly used for distortion calibration, effectively solving the dependence on calibration plates in traditional methods. The operation process is simple and easy to implement, the environmental requirements are not high, and the maintenance cost is relatively low. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0032] Figure 1 is a flowchart of a camera intrinsic parameter calibration method according to an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of center point calibration according to an example of the present invention;

[0034] Figure 3 is a schematic diagram of obtaining focal length through parallel lines according to an embodiment of the present invention;

[0035] Figure 4 is a flowchart of a camera intrinsic parameter calibration method according to a preferred embodiment of the present invention;

[0036] Figure 5 is a diagram showing the dedistortion effect according to an example of the present invention;

[0037] Figure 6 is a structural block diagram of a camera intrinsic parameter calibration device according to an embodiment of the present invention;

[0038] Figure 7 is a structural block diagram of an electronic device according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following describes the implementation of the present invention using specific embodiments. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. Obviously, the embodiments described are only a portion of the present invention, not all of it. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0040] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings.

[0041] According to an embodiment of the present invention, a camera intrinsic parameter calibration method is provided.

[0042] Figure 1 FIG. 1 is a flow chart of a camera intrinsic parameter calibration method according to an embodiment of the present invention. Figure 1 As shown, the camera intrinsic parameter calibration method includes:

[0043] Step S101: capturing an image in the current scene, identifying boundary pixel points of a distorted linear structure in the captured image, and obtaining boundary position information of the linear structure;

[0044] Step S102: sampling the boundaries of the linear structure in the horizontal and vertical directions according to the boundary position information, and fitting the sampling points to obtain the mean position of the boundary as the distortion center in the horizontal and vertical directions;

[0045] Step S103: constructing a fitness function based on the sum of squared errors, and calculating and obtaining a global optimal solution for the distortion coefficient according to the fitness function;

[0046] Step S104: extracting parallel line identification information in the current scene, and obtaining a focal length closed solution based on the relative position relationship of the parallel lines.

[0047] The camera intrinsic parameter calibration solution in related technologies that relies on physical calibration plates has problems such as complex operation procedures, high environmental requirements, and relatively high maintenance costs. Figure 1 The provided camera intrinsic calibration method captures images in the current scene and directly uses the linear structures in the current scene (e.g., road scenes, etc.) (such as road markings and building edges) for distortion correction, effectively solving the problem of relying on calibration plates in related technologies. Figure 1 The provided camera intrinsic parameter calibration method simplifies the calibration process and only relies on linear features in the road scene to complete the camera intrinsic parameter calibration, enabling the camera to perform adaptive calibration in dynamic scenes, effectively improving the calibration accuracy and stability.

[0048] It should be noted that the above-mentioned camera intrinsic parameters refer to the parameters that describe the geometric and optical characteristics of the camera itself, usually including focal length, principal point position (usually coincident with the distortion center), distortion coefficient, etc.

[0049] Distortion coefficients are used to describe and correct for image distortion caused by camera lenses. Common types of distortion include radial distortion and tangential distortion. Radial distortion is caused by the spherical shape of the lens, resulting in different degrees of magnification or reduction of points in the center and edges of the image. Tangential distortion is caused by imperfect alignment or installation between the lens and the image sensor, resulting in a slight skew in the image.

[0050] The center of distortion is the area of ​​the image where the distortion is strongest, typically located at the center of the image. Most camera lenses exhibit both radial and tangential distortion, and the center of distortion is where the distortion is most pronounced, typically coinciding with the camera's optical center (the principal point).

[0051] Among them, focal length refers to the distance from the optical center (focus) of the lens to the imaging plane. The focal length determines the camera's viewing angle and magnification capability.

[0052] In step S101, the boundary pixel points of the distorted linear structure in the collected image are identified, and obtaining the boundary position information of the linear structure can further include the following processing: using the semantic segmentation structure of the Mobile-Seed network to identify the area where the distorted linear structure exists in the collected image; using the boundary detection structure of the Mobile-Seed network to extract the boundary pixel points of the distorted linear structure in the collected image, and obtaining the boundary position information of the linear structure.

[0053] In a preferred embodiment, a Mobile-Seed network can first be used to accurately identify the boundaries of distorted linear structures (such as road markings and lampposts) in the original image, obtaining the boundary position information of these straight structures that appear curved due to lens distortion. The Mobile-Seed network achieves this goal through a dual-stream structure that combines semantic segmentation and boundary detection. The semantic stream is responsible for global semantic segmentation and identifies linear structure regions in the image, while the boundary stream focuses on accurately detecting the boundaries of these structures and outputting a boundary map containing pixel position information of the boundaries of linear structures in the image. By dynamically adjusting the feature fusion weights of the semantic stream and boundary stream, this method can adaptively optimize feature fusion in different image regions, thereby significantly improving the accuracy and robustness of boundary recognition in distorted images.

[0054] In the above-mentioned step S102, sampling the boundaries of the linear structure in the horizontal and vertical directions according to the boundary position information, and fitting the sampling points to obtain the mean position of the boundaries as the distortion center in the horizontal and vertical directions can further include the following: sampling the boundaries of the linear structure in the horizontal and vertical directions, obtaining multiple discrete sampling points as input data for Gaussian curve fitting; and obtaining the mean position of the curve after the linear structure is distorted by performing Gaussian curve fitting on the sampling points, as the distortion center in the horizontal and vertical directions.

[0055] According to the property that distortion satisfies central symmetry, before distortion correction, straight lines in the image will be affected by distortion and appear as curves. Figure 2 As shown, due to the distortion characteristics of the camera lens, the straight line without distortion and Appears as a curve in the image and , but these curves still maintain their central symmetry. Based on this property, the Mobile-Seed network accurately extracts the boundary position information of distorted objects (such as road markings and building edges). Sampling is performed on the identified boundaries (usually uniform sampling, but non-uniform sampling is also possible) to obtain a series of discrete points as input data for Gaussian curve fitting.

[0056] The mean position of the curve is obtained by fitting the sampling points of the linear structures with distortion in the horizontal and vertical directions of the image, which is used as the key parameter in the camera intrinsic calibration. . Camera intrinsic parameter matrix Expressed as:

[0057]

[0058] in, and The calculation formula is: , represents the focal length, and Respectively expressed in Direction and Pixel size in direction, This is the center of distortion.

[0059] As can be seen, this application uses curve recognition and Gaussian curve fitting techniques to fuse images to locate the center of distortion based on distorted linear structures (such as road markings and lampposts) in the original image captured in the current scene (for example, a road scene). By calculating the mean of multiple pairs of distortion curve fitting results, the horizontal and vertical locations of the distortion center are ultimately obtained, which serve as key parameters in intrinsic calibration.

[0060] In step S103, a fitness function is constructed based on the sum of squared errors, and obtaining a global optimal solution of the distortion coefficient by calculating the fitness function may further include the following processing:

[0061] First, the fitness function Fitness is constructed using the following formula:

[0062]

[0063] in, is the number of image points after sampling distortion correction, is the coordinate of the dedistorted point after the i-th distortion correction, is the straight line obtained by least square fitting the coordinates of the dedistorted points, is the slope of the fitted straight line, is the intercept of the fitted straight line;

[0064] Secondly, when the value of the fitness function Fitness is minimum, the radial distortion coefficients c1, c2 and the tangential distortion coefficients d1, d2 are taken as the global optimal solution of the distortion coefficients.

[0065] Wherein, when the value of the fitness function Fitness is minimum, taking the radial distortion coefficients c1, c2 and the tangential distortion coefficients d1, d2 as the global optimal solution of the distortion coefficients may further include the following processing:

[0066] S1: Sampling the boundaries of distorted linear structures in the image to obtain sampling points, and applying random initial radial distortion coefficients c1, c2 and tangential distortion coefficients d1, d2 to perform distortion correction;

[0067] S2: For the distortion-corrected sampling points, the straight line obtained by least square fitting the sampling point coordinates ;

[0068] S3: Substitute the coordinates of the n sampling points after distortion correction, the values ​​of m and b into the fitness function formula to calculate the value of the fitness function;

[0069] S4: Loop through S1 to S3, dynamically adjusting c1, c2, d1, and d2 each time. Through continuous iterative search, when the value of the fitness function Fitness is minimized, the corresponding radial distortion coefficients c1 and c2 and tangential distortion coefficients d1 and d2 are taken as the global optimal solution for the distortion coefficients.

[0070] The distortion coefficients mentioned above are used to compensate for image distortion caused by lens distortion, restoring the curve to its original straight line. The fitness function mentioned above is used in genetic algorithms to evaluate the performance of individuals. It measures the fitness of an individual based on its performance in a specific task. Specifically, the fitness function returns a numerical value indicating the quality of the individual (solution). The optimization objective of the fitness function is typically to maximize or minimize a certain metric.

[0071] In the preferred implementation process, the present application is based on the distortion center positioning result, the fitness function Fitness based on the sum of squared errors, and combined with the genetic algorithm for global optimization to solve the distortion coefficient, ensuring that the optimization result of the distortion coefficient reaches the global optimum. Among them, the above-mentioned genetic algorithm refers to an optimization algorithm that simulates the principles of natural selection and genetics. It solves optimization problems by simulating the process of biological evolution in nature. It is suitable for solving complex or high-dimensional optimization problems, especially problems that cannot be solved by traditional methods. First, according to the principle of geometric restoration of distortion coefficients, for known sampling points on the boundary of objects distorted by distortion in the image, a random initial radial distortion coefficient can be applied. and the tangential distortion coefficient Perform distortion correction. The correction formula of the distortion coefficient to the distorted image point is as follows:

[0072]

[0073] in, represents the coordinates of a point in the image after distortion correction, Indicates the coordinates of the point in the distorted image. Yes To the center of distortion The square of the distance.

[0074] It should be noted that as the distortion coefficients are closer to their true values, the restored sampling point fitting results will be closer to a straight line. Therefore, the straightness of the curve reflects the degree to which the distortion parameters are close to their true values. Based on this characteristic, the fitness function described above is designed to measure the quality of the distortion coefficients by evaluating the degree of fit between the coordinate points after distortion correction and the ideal straight line. To achieve this goal, the fitness function is defined as follows:

[0075]

[0076] in, is the number of image points after sampling distortion correction, is the coordinate of the dedistorted point after the i-th correction, is the straight line obtained by least square fitting the coordinates of the dedistorted points, is the slope of the fitted straight line, is the intercept of the fitted line.

[0077] In the optimization process of the genetic algorithm, the smaller the value of the fitness function, the more effectively the current distortion coefficient can correct the image distortion, making the fitting line closer to the ideal distortion-free line and the error smaller. Such distortion coefficients will obtain higher fitness and thus be retained and optimized first in the iteration. Through continuous iterative search, the algorithm eventually converges to the optimal solution of the distortion coefficient, thereby accurately estimating the radial distortion coefficient. and the tangential distortion coefficient , to achieve accurate correction of image distortion.

[0078] In step S104, parallel line identification information is extracted in the current scene, and based on the relative position relationship of the parallel lines, obtaining the focal length closed solution may further include the following processing: extracting one or more groups of parallel lines in the current scene according to specific identifications; for each group of parallel lines, obtaining a unique solution of the projective depth corresponding to the multiple endpoints of the group of parallel lines according to the image point coordinates corresponding to the multiple endpoints of the group of parallel lines; obtaining the focal length closed solution corresponding to the group of parallel lines according to the unique solution of the projective depth, the distortion center, and the image point coordinates; and determining the final focal length closed solution by combining the calculation results of the focal length closed solutions corresponding to the one or more groups of parallel lines.

[0079] The projective depth reflects the distance information of objects in the camera image and indicates the distance between each point in the scene and the camera.

[0080] Among them, for each group of parallel lines, according to the image point coordinates corresponding to the multiple endpoints of the group of parallel lines, obtaining the unique solution of the projective depth corresponding to the multiple endpoints of the group of parallel lines can further include the following processing: constructing a binocular stereo vision system, setting the origin of the world coordinate system to be located at the optical center of one of the binocular eyes; substituting the coordinates of the multiple endpoints of the group of parallel lines in the world coordinate system and the image point coordinates corresponding to the multiple endpoints into the projection equation; finding the correlation between the projective depth and the image point coordinates according to the projection equation to obtain the constraint equation; setting the projective depth to be positive, and all valid three-dimensional points in the image are located in front of the camera, and when the image point coordinates are known, obtaining the unique solution of the projective depth corresponding to the multiple endpoints by solving the null space of the matrix composed of the image point coordinates.

[0081] Among them, in the above-mentioned method of obtaining the focal length closed solution corresponding to the group of parallel lines based on the unique solution of the projective depth, the position of the distortion center, and the coordinates of the image point, the focal length closed solution f can be obtained by the following formula:

[0082]

[0083] in, , and They are vectors and No. elements, , is the distortion center, , are the projective depths of the two endpoints of a line segment in the set of parallel lines, are the projected depths of the two endpoints of another line segment of the set of parallel lines that is of equal length to the line segment, are the image point coordinates corresponding to the two endpoints of the line segment, are the image point coordinates corresponding to the two endpoints of the other line segment.

[0084] The following combination Figure 3 The above preferred embodiments are further described with reference to examples.

[0085] like Figure 3 As shown, the world coordinate system is defined as , the left camera coordinate system and the right camera coordinate system are defined as . Set the origin of the world coordinate system Located in the center of the left eye (Of course, you can also set the origin of the world coordinate system Located at the center of the right eye). In the current scene, one or more groups of parallel lines are extracted according to specific identification (such as zebra crossing, etc.), wherein the parallel lines extracted above are The endpoints of a line segment are , and their corresponding points in the image are The coordinates of the space point are defined as , the coordinates of the corresponding image points are The projection equation is expressed as:

[0086]

[0087] in, represents the camera intrinsic parameter matrix, is the scaling factor to ensure that the final image point coordinates can be correctly normalized, and are the rotation matrix and translation vector respectively.

[0088] In a set of parallel lines, extract the equal-length parallel segments. The four endpoints of this set of parallel segments are , are the coordinates of their corresponding image points. Substituting the endpoint coordinates into the projection equation, we have:

[0089]

[0090] in, The corresponding points The projective depth, The corresponding points The projective depth.

[0091] Since the coordinates of the point in the world coordinate system are unknown, the four equations above are simplified by subtracting each other and eliminating the common terms to find the direct connection between the projective depth and the image point coordinates. The constraint equation is as follows:

[0092]

[0093] in, ,

[0094] .

[0095] In order to resolve the scale ambiguity between variables, the projective depth is always set to be positive, and all valid 3D points in the image are located in front of the camera. It is known that by solving the null space of the linear equation system, the unique solution of the projective depth can be obtained. The specific calculation formula is:

[0096]

[0097] in, Represents a matrix consisting of image point coordinates, such as , Representation matrix The null space of The function is used to ensure that the returned value is an absolute value, thereby ensuring that the projective depth is positive and thus uniquely solving the projective depth.

[0098] According to the above projection equation, we can find the direct relationship between the projection depth and the image point coordinates, and obtain the constraint equation, which can be used to obtain the following two equations:

[0099]

[0100] For special calibration objects in the scene (for example, zebra crossings), their direction vectors meet the following conditions: , further, define: , are all 3×1 vectors. According to the orthogonality condition of the rotation matrix , we get the following equation:

[0101]

[0102] In order to simplify the computational complexity, the following conditions are assumed: According to the above equation, only the focal length is unknown, so the focal length is The closed form solution is:

[0103]

[0104] in, , and They are vectors and No. elements, , is the principal point position (i.e. the distortion center mentioned above).

[0105] In a preferred implementation, the present application provides a focal length calibration method based on multiple sets of parallel line segments to address the limited accuracy of focal length calibration using geometric constraints imposed by a single set of parallel line segments. This method calculates the focal lengths of multiple sets of parallel line segments and observes how the focal length values ​​change as the number of sets increases. When the focal length values ​​stabilize and their variation range is very small, for example, when the deviations of the individual closed-form focal length solutions from the mean do not exceed a preset threshold, the final closed-form focal length solution is determined. The intrinsic calibration results are then used to perform image dedistortion.

[0106] The following combination Figure 4 The above preferred embodiments are further described.

[0107] Figure 4 FIG. 1 is a flow chart of a camera intrinsic parameter calibration method according to a preferred embodiment of the present invention. Figure 4 As shown, the camera intrinsic parameter calibration method includes the following steps:

[0108] Step S401: Capture an image in the current scene, identify boundary pixels of distorted linear structures (curved curves) in the captured image, and obtain boundary position information of the linear structure. Specifically, a Mobile-Seed network may be used to extract boundary pixels of the curved curves in the image.

[0109] Step S402: Gaussian curve fitting is used to analyze uniformly sampled points on the boundary to accurately locate the center of distortion. Specifically, by fusion image curve recognition and Gaussian curve fitting, the center of distortion is located based on distorted linear structures (such as road markings and lampposts) in the original image. By calculating the mean of multiple pairs of distortion curve fitting results, the horizontal and vertical center positions are ultimately determined, serving as key parameters for intrinsic calibration.

[0110] Step S403: Based on the distortion center positioning result, a fitness function based on the sum of squared errors is designed, and a genetic algorithm is used for global optimization to solve the global optimal solution of the distortion coefficient, ensuring that the optimization result of the distortion coefficient reaches the global optimum. Step S404: Extract parallel line identification information from the current scene. Applying the properties of parallel lines, the relative positional relationships of parallel lines in space are analyzed to propose a focal length solution based on parallel line constraints, and calculate a closed-form solution for the focal length. This focal length solution utilizes projective geometry theory combined with the characteristics of parallel lines in the image to derive a closed-form solution for the focal length.

[0111] In a preferred implementation, a focal length calibration method based on multiple sets of parallel line segments was proposed to address the limited accuracy of focal length calibration using geometric constraints imposed by a single set of parallel line segments. This method calculates the focal lengths of multiple sets of parallel line segments and observes how the focal length values ​​change as the number of sets increases. The final intrinsic parameter calibration result is determined when the focal length values ​​stabilize and their variation range is very small.

[0112] Step S405: performing image dedistortion processing using the intrinsic calibration result.

[0113] Through the above embodiments, considering that the complexity of the scene has a crucial impact on the camera intrinsic calibration results, this embodiment combines line feature recognition technology and parallel line constraints to provide a solution that does not require additional physical calibration equipment. This method improves the flexibility and adaptability of calibration while achieving instant execution in the actual operating environment and simplifying the calibration process. In addition, the intrinsic calibration results are used to perform image dedistortion processing, and the dedistortion effect is shown in the figure below. Figure 5 As shown in the figure, by comparing the images before and after dedistortion, it can be seen that the image distortion after calibration parameter processing is effectively corrected and the image quality is significantly improved, which further verifies the effectiveness of the technical solution provided by the present invention in practical applications.

[0114] According to an embodiment of the present invention, a camera intrinsic parameter calibration device is provided.

[0115] Figure 6 FIG is a structural block diagram of a camera intrinsic parameter calibration device according to an embodiment of the present invention. Figure 6 As shown, the camera intrinsic parameter calibration device includes: an identification module 60, which is used to capture images in the current scene, identify boundary pixel points of distorted linear structures in the captured images, and obtain boundary position information of the linear structures; a distortion center acquisition module 62, which is used to sample the boundaries of the linear structures in the horizontal and vertical directions according to the boundary position information, and fit the sampling points to obtain the mean position of the boundaries as the distortion center position in the horizontal and vertical directions; a distortion coefficient acquisition module 64, which is used to construct a fitness function based on the sum of squared errors, and calculate the global optimal solution of the distortion coefficient according to the fitness function; a focal length acquisition module 66, which is used to extract parallel line identification information in the current scene, and obtain a focal length closed solution based on the relative position relationship of the parallel lines.

[0116] Figure 6The camera intrinsic calibration device shown here directly utilizes linear structures (such as road markings and building edges) in the captured scene for distortion correction, effectively eliminating the traditional reliance on calibration plates. This simplifies the calibration process, relying solely on linear features in the road scene to complete camera intrinsic calibration. This enables adaptive calibration in dynamic scenes, significantly improving calibration accuracy and stability.

[0117] It should be noted that the above-mentioned camera intrinsic parameter calibration device can refer to Figures 1 to 5 The corresponding descriptions and effects in the embodiments shown can be understood and will not be repeated here.

[0118] According to an embodiment of the present invention, an electronic device is further provided.

[0119] Figure 7 1 is a block diagram of an electronic device according to a preferred embodiment of the present invention. Figure 7 As shown, the electronic device according to the present invention includes: a memory 70 and a processor 72, the memory 70 is used to store computer-executable instructions of the processor 70; the processor 72 is configured to execute any one of the above-mentioned camera intrinsic parameter calibration methods by executing the above-mentioned executable instructions.

[0120] The processor 72 may be a central processing unit (CPU). The processor 72 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of these chips.

[0121] Memory 70, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the camera intrinsic parameter calibration method in the embodiments of the present invention. The processor executes the non-transitory software programs, instructions, and modules stored in the memory to perform various processor functions and data processing.

[0122] The memory 70 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 70 may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0123] The one or more modules are stored in the memory 70 and when executed by the processor 72, the following steps are performed: Figures 1 to 5 The camera intrinsic parameter calibration method in the illustrated embodiment.

[0124] For details of the above electronic equipment, please refer to Figures 1 to 5 The corresponding descriptions and effects in the embodiments shown can be understood and will not be repeated here.

[0125] In summary, the above-described embodiments of the present invention utilize linear structures (such as road markings and building edges) within the image captured in the current scene to calibrate camera intrinsic parameters, effectively addressing the reliance on calibration plates in related methods. A Mobile-Seed network is used to accurately extract the boundary pixels of curved curves in the image, Gaussian curve fitting is used to locate the center of distortion, and an improved genetic algorithm is employed to optimize the distortion coefficients. Furthermore, by introducing spatial parallel line constraints and optimizing the camera focal length, intrinsic parameter calibration is further improved. This method simplifies the calibration process, relying solely on linear features in the road scene to estimate camera intrinsic parameters. This enables adaptive calibration of the camera in dynamic scenes, effectively improving calibration accuracy and stability.

[0126] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A camera intrinsic parameter calibration method, characterized in that: include: Collecting an image in the current scene, identifying boundary pixel points of a distorted linear structure in the collected image, and obtaining boundary position information of the linear structure; Sampling the boundaries of the linear structure in the horizontal and vertical directions according to the boundary position information, and fitting the sampling points to obtain the mean position of the boundaries as the distortion center in the horizontal and vertical directions; Constructing a fitness function based on the sum of squared errors, and calculating a global optimal solution for the distortion coefficient according to the fitness function; Extract the parallel line identification information in the current scene and obtain the focal length closed solution based on the relative position relationship of the parallel lines.

2. The method according to claim 1, characterized in that Identifying boundary pixel points of a distorted linear structure in the collected image and obtaining boundary position information of the linear structure includes: Using a semantic segmentation structure of a Mobile-Seed network to identify regions of distorted linear structures in the collected image; The boundary detection structure of the Mobile-Seed network is used to extract boundary pixel points of the distorted linear structure in the collected image, and the boundary position information of the linear structure is obtained.

3. The method according to claim 1, characterized in that Sampling the boundaries of the linear structure in the horizontal and vertical directions according to the boundary position information, and fitting the sampling points to obtain the mean position of the boundary as the distortion center in the horizontal and vertical directions includes: Sampling the boundaries of the linear structure in the horizontal and vertical directions to obtain a plurality of discrete sampling points as input data for Gaussian curve fitting; The mean position of the linear structure distorted curve is obtained by performing Gaussian curve fitting on the sampling points, and is used as the distortion center in the horizontal direction and the vertical direction.

4. The method according to claim 1, wherein Constructing a fitness function based on the sum of squared errors, and calculating and obtaining a global optimal solution of the distortion coefficient according to the fitness function includes: The fitness function Fitness is constructed by the following formula: in, is the number of image points after sampling distortion correction, is the coordinate of the dedistorted point after the i-th distortion correction, is the straight line obtained by least square fitting the coordinates of the dedistorted points, is the slope of the fitted straight line, is the intercept of the fitted straight line; When the value of the fitness function Fitness is minimum, the radial distortion coefficients c1, c2 and the tangential distortion coefficients d1, d2 are taken as the global optimal solution of the distortion coefficients.

5. The method according to claim 4, characterized in that When the value of the fitness function Fitness is minimum, the radial distortion coefficients c1, c2 and the tangential distortion coefficients d1, d2 are taken as the global optimal solution of the distortion coefficients, including: S1: Sampling the boundaries of distorted linear structures in the image to obtain sampling points, and applying random initial radial distortion coefficients c1, c2 and tangential distortion coefficients d1, d2 to perform distortion correction; S2: For the distortion-corrected sampling points, the straight line obtained by least square fitting the sampling point coordinates ; S3: Substitute the coordinates of the n sampling points after distortion correction, the values ​​of m and b into the fitness function formula to calculate the value of the fitness function; S4: Loop through S1 to S3, dynamically adjusting c1, c2, d1, and d2 each time. Through continuous iterative search, when the value of the fitness function Fitness is minimized, the corresponding radial distortion coefficients c1 and c2 and tangential distortion coefficients d1 and d2 are taken as the global optimal solution for the distortion coefficients.

6. The method according to claim 1, wherein Extracting parallel line identification information in the current scene and obtaining a focal length closed-loop solution based on the relative position relationship of the parallel lines include: Extract one or more sets of parallel lines according to specific identifiers in the current scene; For each set of parallel lines, find the unique solution of the projective depth corresponding to the multiple endpoints of the set of parallel lines according to the coordinates of the image points corresponding to the multiple endpoints of the set of parallel lines; Obtaining a closed-form focal length solution corresponding to the group of parallel lines according to the unique solution of the projective depth, the distortion center, and the image point coordinates; The calculation results of the focal length closed solutions corresponding to the one or more groups of parallel lines are synthesized to determine a final focal length closed solution.

7. The method according to claim 6, characterized in that For each set of parallel lines, based on the coordinates of the image points corresponding to the endpoints of the set of parallel lines, the unique solution for obtaining the projective depth corresponding to the endpoints of the set of parallel lines includes: Construct a binocular stereo vision system and set the origin of the world coordinate system to be located at the optical center of one of the binocular eyes; Substituting the coordinates of the multiple endpoints of the set of parallel lines in the world coordinate system and the coordinates of the image points corresponding to the multiple endpoints into the projection equation; Finding the correlation between the projective depth and the image point coordinates according to the projective equation to obtain a constraint equation; Assuming that the projective depth is positive and all valid three-dimensional points in the image are located in front of the camera, when the image point coordinates are known, a unique solution for the projective depth corresponding to the multiple endpoints is obtained by solving the null space of the matrix formed by the image point coordinates.

8. The method according to claim 6, characterized in that Obtaining a closed-form focal length solution corresponding to the set of parallel lines according to the unique projective depth solution, the distortion center position, and the image point coordinates includes: The focal length closed-form solution f is obtained by the following formula: in, , and They are vectors and No. elements, , is the distortion center, , are the projective depths of the two endpoints of a line segment in the set of parallel lines, are the projected depths of the two endpoints of another line segment of the set of parallel lines that is of equal length to the line segment, are the image point coordinates corresponding to the two endpoints of the line segment, are the image point coordinates corresponding to the two endpoints of the other line segment.

9. A camera intrinsic parameter calibration device, characterized in that: include: A recognition module is used to collect images in the current scene, identify boundary pixels of distorted linear structures in the collected images, and obtain boundary position information of the linear structures; a distortion center acquisition module, configured to sample the boundaries of the linear structure in the horizontal and vertical directions according to the boundary position information, and fit the sampling points to obtain the mean position of the boundaries as the distortion center position in the horizontal and vertical directions; A distortion coefficient acquisition module is used to construct a fitness function based on the sum of squared errors and calculate the global optimal solution of the distortion coefficient according to the fitness function; The focal length acquisition module is used to extract the identification information of parallel lines in the current scene and obtain the focal length closed solution based on the relative position relationship of the parallel lines.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store executable instructions of the processor, wherein the processor is configured to perform the camera intrinsic parameter calibration method according to any one of claims 1 to 8 by executing the executable instructions.

Images