RGB-T dual-light camera system parameter self-calibration method based on shape context

Abstract

The invention discloses an RGB-T dual-light camera system parameter self-calibration method based on shape context, and relates to the technical field of camera parameter self-calibration, and the method comprises the steps: synchronously collecting a plurality of frames of RGB images and thermal imaging images of an object at different visual angles, respectively extracting the outlines of the images through employing a contour extraction algorithm, carrying out the sampling of the outlines, and carrying out the self-calibration of the parameters of the RGB-T dual-light camera system. Obtaining a contour point set and calculating a shape context histogram; comparing the shape context histograms to obtain a matching point set, and when the number of matching points in the set exceeds a set threshold value, obtaining a matching point essential matrix based on an RANSAC algorithm; and decomposing the essential matrix through SVD to obtain decomposed external parameters, optimizing the decomposed external parameters through a nonlinear optimization technology, and completing RGB-T dual-light camera system parameter self-calibration by using the optimized external parameters. According to the method, the calculation of the relative pose external parameters between the cameras can be realized without special processing, and the calibration of the RGB-T dual-light camera system is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of camera parameter self-calibration, in particular to a method for self-calibrating the parameters of an RGB-T dual-camera system based on shape context. Background Technique

[0002] An RGB-T dual-camera system usually includes an RGB camera and a thermal imaging (Thermal) camera. The two cameras can be integrated inside a set of hardware devices or appear in the form of two independent cameras. It simultaneously acquires RGB images and thermal imaging images in color spaces such as white-hot / iron-red to perform image and video analysis and processing on scenes and objects. This camera is widely used in mechanical equipment inspection and visualization, health care, industrial production, security, agricultural production, building inspection, military, etc. Taking the application in the mechanical field as an example, by using an RGB-T dual-camera system, three-dimensional reconstruction of mechanical equipment can be performed for applications such as 3D simulation and digital twin. At the same time, by using multiple RGB-T dual-camera systems to perform collaborative video acquisition of mechanical equipment and operators from different perspectives, spatio-temporal 4D reconstruction can be carried out to realize applications such as visualization of production dynamic scenes and virtual maintenance. To perform three-dimensional reconstruction or multi-view camera 4D reconstruction, the first step is to calibrate the RGB camera and the thermal imaging camera in the dual-camera system. The calibration content usually includes the internal parameters of each of the two cameras, as well as the external parameters of the relative position t and the rotation orientation R between the two cameras.

[0003] For an RGB-T dual-camera system, the internal parameters of each of the two cameras are usually relatively easy to obtain through their respective calibrations, and when the device leaves the factory, the internal parameters are often provided, and this parameter can usually also be obtained from data such as the EXIF tags of the captured pictures. However, the external parameters of the relative coordinate system between the RGB and thermal imaging cameras, that is, the relative angle and relative offset between the two cameras, need to be obtained through a special calibration process. And for the case where the RGB camera and the thermal imaging camera are not integrated, their relative pose, that is, the external parameters often change due to vibration, collision, etc. Therefore, it is necessary to frequently calibrate their external parameters.

[0004] Existing camera calibration methods usually use a specially made standard checkerboard, or through preprocessing methods such as heating the standard checkerboard, so that it can be significantly captured in both RGB cameras and thermal imaging cameras. Then, through standard camera calibration methods [such as: Zhang Zhengyou calibration method, etc.], the relative external parameters are obtained. This method requires preparing a checkerboard calibration object for the standard plane, and the process is also very troublesome. At the same time, for ordinary RGB binocular cameras, there are currently methods for automatic calibration using feature point matching such as SIFT, SURF, and ORB. However, for the RGB-T camera system, the thermal imaging image is formed according to the local temperature of the object, and it cannot extract a large number of effective feature points like the RGB image, and its feature properties are also quite different from those of the RGB image. Therefore, there is currently a lack of an automatic calibration algorithm for the RGB-T dual-camera system. Summary of the Invention

[0005] In view of the problems existing in the existing self-calibration method for the parameters of the RGB-T dual-camera system based on shape context, the present invention is proposed. Therefore, the problem to be solved by the present invention is how to provide a self-calibration method for the parameters of the RGB-T dual-camera system based on shape context.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a self-calibration method for the parameters of an RGB-T dual-camera system based on shape context, which includes synchronously collecting multiple frames of RGB images and thermal imaging images of an object at different perspectives, respectively extracting the contours of the images by using a contour extraction algorithm, sampling the contours, obtaining a set of contour points, and calculating the shape context histogram;

[0008] Comparing the shape context histograms to obtain a set of matching points, and when the number of matching points in the set exceeds a set threshold, obtaining the essential matrix of the matching points based on the RANSAC algorithm;

[0009] Obtaining the decomposed external parameters by SVD decomposition of the essential matrix, optimizing the decomposed external parameters by using a nonlinear optimization technique, and completing the self-calibration of the parameters of the RGB-T dual-camera system by using the optimized external parameters.

[0010] As a preferred solution of the self-calibration method for the parameters of the RGB-T dual-camera system based on shape context of the present invention, wherein: the contour extraction algorithm includes the Canny contour extraction method, the Blob block extraction and contour parsing method, and the method of using image foreground segmentation and then extracting the contour; extracting the contour C from the RGB image I A and extracting the contour C from the thermal imaging image I A B B .

[0011] As a preferred solution of the RGB-T binocular camera system parameter self-calibration method based on shape context according to the present invention, wherein: the calculation of the shape context histogram includes the following:

[0012] Use the Jitendra sampling method to sample the contour C A and the contour C B respectively to obtain the contour point sets and and calculate the shape context histogram sets {H A} and the set {H B};

[0013] For each sampling point set P = {p1, p2, p3,..., p n}, construct a polar logarithmic coordinate system with the P i point as the center. With P i as the reference point, establish N concentric circles at logarithmic distance intervals within the area with P i as the center and r as the radius, and divide the area into M equal parts along the circumferential direction to form area partitions;

[0014] Calculate the angles and distances of other points in the sampling point set relative to the P i point, and map the points to the polar logarithmic coordinate system area, and count the number of points in each area to form a histogram;

[0015] Normalize the statistically obtained histogram to obtain the shape context histogram sets {H A} and the set {H B}.

[0016] As a preferred solution of the RGB-T binocular camera system parameter self-calibration method based on shape context according to the present invention, wherein: the matching point set includes, for the element p A in the shape context histogram set {H i}, calculate the distance from the element q B in the shape context histogram set {H j}, and use the points with a distance less than the distance judgment threshold λ as the matching points of the element p A in the set {H i} and put them into the matching point set {p||q}. The distance calculation formula between elements in the set is:

[0017]

[0018] In the formula, C i,j is the distance between the element p i and the element q j is hi (k) is the shape context histogram of element p i h j (k) is the shape context histogram of element q j ;

[0019] Use the same process to calculate the matching points for other perspective images. When the number of the matching point set {p||q} exceeds the set threshold σ, continue with the extrinsic parameter calculation; otherwise, the calculation fails and the acquisition object needs to be searched again.

[0020] As a preferred solution of the RGB-T binocular camera system parameter self-calibration method based on shape context according to the present invention, wherein: the essential matrix of the matching points obtained based on the RANSAC algorithm includes the following:

[0021] Randomly select 8 feature point pairs from the matching point list. The feature point pair set is represented as {p i ||q i |i = 1,..., 8};

[0022] Transform the feature point pairs in the set to the camera space through the camera intrinsic matrices K1 and K2 to form matrix x1 and matrix x2, and solve the essential matrix E, that is, use the least squares method to make the essential matrix E satisfy the following constraint conditions, expressed as:

[0023]

[0024] Construct the camera intrinsic matrix as:

[0025]

[0026] In the formula, f x , f y are the focal lengths of the camera on the x-axis and y-axis respectively, c x , c y are the principal point coordinates of the image on the x-axis and y-axis respectively;

[0027] The point p is represented in homogeneous coordinates as: (x, y, 1), and its transformation result is:

[0028]

[0029] For each pair of normalized matching points (p i , q i ), assuming p i =(u i v i 1) T , q i =(u' i v' i 1) TExpand the essential matrix E into a 9-dimensional vector E = [e 11 e 12 e 13 e 21 e 22 e 23 e 31 e 32 e 33 T , then the equation is expressed as:

[0030] [u' i u i u' i v i u' i v' i u i v' i v i v' i u i v i 1]·e = 0

[0031] Combine the equations of all matching points into an m×9 matrix A, where m≥8, expressed as:

[0032]

[0033] A·e = 0

[0034] Solve it by the least squares method, that is, perform singular value decomposition SVD on the matrix, then the essential matrix E corresponds to the 8th row vector of V T ;

[0035] A = UΣV T

[0036] Calculate the remaining matching point pairs in the {p||q} set. If it satisfies the residual constraint, then mark it as an inlier, and record the total number of inliers N and the corresponding essential matrix E;

[0037] If the total number of inliers N calculated new > the existing total number of inliers N, then update the essential matrix E = E new , and update N = N new ;

[0038] If the number of iteration times or the total tolerance error quantity exceeds the threshold μ, stop the iteration.

[0039] ​As a preferred embodiment of the method for self-calibrating the parameters of the RGB-T dual-optical camera system based on shape context according to the present invention, wherein: the singular value decomposition of the matrix includes using the singular value decomposition of the essential matrix E, setting the third singular value to 0 to satisfy the rank-2 constraint, and obtaining the initial rotation parameters and translation parameters of the camera according to the singular value decomposition result.

[0040] As a preferred embodiment of the method for self-calibrating the parameters of the RGB-T dual-optical camera system based on shape context according to the present invention, wherein: the optimization of the decomposed external parameters through non-linear optimization techniques includes the following: using non-linear optimization techniques to make the points in the RGB image in the inlier set reprojected back to the points in the thermal image and their corresponding points have the minimum sum of distances, obtaining the final rotation parameter matrix R and translation parameter vector t, that is, obtaining the optimized external parameters of the camera and completing the self-calibration of the camera.

[0041] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and: when the processor executes the computer program, it implements the steps of the method for self-calibrating the parameters of the RGB-T dual-optical camera system based on shape context.

[0042] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by a processor, it implements the steps of the method for self-calibrating the parameters of the RGB-T dual-optical camera system based on shape context.

[0043] By extracting contour information from the RGB image and the thermal image and using the shape context histogram features of contour sampling to form a matching point set, the self-calibration of the RGB-T dual-optical camera system is realized, so that it is not necessary to use a standard planar calibration board, nor to perform special treatments such as heating, and the calculation of the relative pose external parameters between the two cameras can be realized, greatly facilitating the calibration process of the RGB-T dual-optical camera system. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0045] Figure 1 It is a flowchart of the method for self-calibrating the parameters of the RGB-T dual-optical camera system based on shape context. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] To make the above objects, features, and advantages of the present invention more understandable, the following will provide a detailed description of the specific embodiments of the present invention with reference to the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0048] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is separate or selectively mutually exclusive with other embodiments.

[0049] Refer to Figure 1 , which is the first embodiment of the present invention. This embodiment provides a method for self-calibrating the parameters of an RGB-T dual-light camera system based on shape context, including:

[0050] S1: Synchronously collect multiple frames of RGB images and thermal imaging images of an object at different perspectives, respectively extract the contours of the images by using a contour extraction algorithm, sample the contours, obtain a set of contour points, and calculate the shape context histogram;

[0051] Specifically, find an object with an irregular surface shape (non-spherical, with multiple curves or surfaces on the surface). In a relatively clean background environment, use a contour extraction algorithm to automatically extract the contours of the RGB image and the thermal imaging image. Since the baseline of the RGB-T dual-light camera is usually small, the parallax of its imaging images is not large. Therefore, the imaging shape differences are small. By sampling the foreground contours extracted, calculate the shape context histograms of the two cameras respectively. By comparing the shape context histograms of the points of the two cameras, find the set of matching points of the two cameras. When the number of matching points in the set exceeds a certain threshold, by using the eight-point algorithm based on RANSAC, the essential matrix E can be accurately obtained. Then, through SVD decomposition, the decomposed external parameters R and t can be obtained. Further, through nonlinear optimization techniques, the external parameters can be further optimized.

[0052] S1.1: Select any object as the acquisition body, which needs to have a large difference from the background in both the RGB image and the thermal image, and start the calibration process.

[0053] S1.2: Synchronously acquire multiple frames of RGB images and thermal images at the same moment from different perspectives.

[0054] S1.3: Extract the contour C A from the RGB image I A . The Canny contour extraction method, or the Blob block extraction and contour parsing method can be used, or the method of foreground segmentation of the image and then contour extraction can also be used (the specific contour extraction method is not the focus of the present invention).

[0055] S1.4: Extract the contour C B from the thermal image I B , using the same method as in step 3.

[0056] S1.5: Sample the contour C A to obtain a set of contour points and calculate its set of shape context histograms {H A}. The sampling method can be uniform sampling, random sampling, etc.

[0057] For shape context, the Jitendra sampling method is usually used. This method ensures the uniform distribution of sampling points by deleting pairs of points that are too close. The calculation of shape context usually includes the following steps:

[0058] For each sampling point P = {p1, p2, p3,..., p n}, construct a polar logarithmic coordinate system with point P i as the center. The polar logarithmic coordinate system divides the space into multiple regions. Specifically, with P i as the reference point, N concentric circles are established at logarithmic distance intervals within the region with P i as the center and r as the radius, and this region is equally divided into M parts along the circumferential direction to form region separation.

[0059] Calculate the angles n and distances d of other points in the set P = {p1, p2, p3,..., p i} relative to point P , and map these points to the regions of the polar logarithmic coordinate system, and count the number of points in each region to form the histogram h i (k);

[0060] h i (k) = #{q ≠ p i : (q - p i ) ∈ bin(k)}

[0061] where: k = {1, 2,..., K}, K = M × N

[0062] Normalize the statistically obtained histogram so that its sum is 1, i.e.:

[0063]

[0064] Finally, obtain a set of shape context histograms, denoted as:

[0065] {H A} = {H i | i = 1,..., n}

[0066] S1.6: Perform the same operation as in step 5 on the contour C B to obtain a set of contour points and a set of shape context histograms {H B}.

[0067] S2: Compare the shape context histograms to obtain a set of matching points. When the number of matching points in the set exceeds a set threshold, obtain the essential matrix of the matching points based on the RANSAC algorithm;

[0068] Specifically, S2.1: For an element p A in {H i}, find the closest element q B in {H j} whose distance is less than the threshold λ, and use it as its matching point. Put it into the set of matching points {p||q}. The distance is calculated using the following formula:

[0069]

[0070] where the threshold λ is generally a manually given threshold.

[0071] S2.2: Repeat the above process for other perspective images.

[0072] S2.3: When the number of the set of matching points {p||q} exceeds σ (σ is generally a manually given threshold, usually set as σ > 100), proceed to the next step of calculating the external parameters. Otherwise, the calculation fails and other objects need to be searched for.

[0073] S2.4: Use the RANSAC algorithm for camera external parameter calibration. The specific algorithm is as follows:

[0074] Randomly select 8 feature point pairs from the list of matching points, and the set is {p i ||q i | i = 1,..., 8},

[0075] The feature point pairs p i ||q i in the set are transformed into the camera space through the camera intrinsic matrices K1 and K2 to form matrix x1 and matrix x2. The essential matrix E is solved by the eight-point algorithm, that is, the least squares method is used to make the matrix E satisfy the following constraints:

[0076] The camera transformation method is as follows. Assume the camera intrinsic matrix is:

[0077]

[0078] In the formula, f x , f y are the focal lengths of the camera on the x-axis and y-axis respectively, c x , c y are the principal point coordinates of the image on the x-axis and y-axis respectively;

[0079] The point p is represented in homogeneous coordinates as: (x, y, 1), and its transformation result is:

[0080]

[0081] The method for solving the essential matrix by the eight-point algorithm is:

[0082] For each pair of normalized matching points (p i , q i ), assume p i =(u i v i 1) T , q i =(u' i v' i 1) T The essential matrix E is expanded into a 9-dimensional vector E = [e 11 e 12 e 13 e 21 e 22 e 23 e 31 e 32 e 33 ) T , then the equation can be expressed as:

[0083] [u' i u i u' i v i u' i v' i u i v' i v i v' i ui v i 1]·e = 0,

[0084] Combine the equations of all matching points into a matrix A, where m ≥ 8:

[0085]

[0086] Then calculate the equation: A·e = 0,

[0087] S3: Obtain the decomposed external parameters by performing SVD on the essential matrix, optimize the decomposed external parameters through nonlinear optimization techniques, and use the optimized external parameters to complete the self-calibration of the RGB-T binocular camera system parameters.

[0088] Specifically, solve it by the least squares method, that is, perform singular value decomposition (SVD) on the matrix: A = UΣV T Then the essential matrix E corresponds to the 8th row vector of V T of.

[0089] S3.1: Calculate the remaining matching point pairs in the {p||q} set. If the residual constraint is satisfied (i.e., the residual < η, a manually given threshold), then record it as an inlier, and record the total number N of its inliers and the corresponding essential matrix E.

[0090] S3.2: Go back to S3.1 again. If the total number N of inliers calculated new > the existing total number N of inliers, then update the essential matrix E = E new , and update N = N new .

[0091] S3.3: If the number of iterations or the total tolerance error count exceeds the threshold μ (a manually given threshold), then stop the iteration.

[0092] S3.4: Perform SVD on the essential matrix E and set the third singular value to 0 to make it satisfy the rank-2 constraint. According to the singular value decomposition result, the initial rotation parameter R and translation parameter t of the camera can be obtained. The specific method is as follows:

[0093]

[0094] The rotation parameter matrix R and translation parameter vector t can be calculated by the following formula:

[0095] R = UWV T , or R = UW T V T , t = ±U 第三列

[0096] ​Calculate the reprojection error and select the solution with the minimum error as the final extrinsic parameters.

[0097] S3.5: Use non - linear optimization techniques (such as the Levenberg - Marquardt algorithm) to reproject the points in the RGB image in the inlier set back to the points in the thermal image and minimize the sum of the distances to their corresponding points to obtain the final rotation parameter matrix R and translation parameter vector t, that is, obtain the extrinsic parameters of the camera and complete the self - calibration of the camera. The specific method is as follows: Because:

[0098]

[0099] The optimization goal is to minimize the sum of the squares of all residuals, and the expression is:

[0100]

[0101] By calculating the Jacobian matrix and performing iterative updates.

[0102] This embodiment also provides a computer device applicable to the case of the self - calibration method of the RGB - T dual - camera system parameters based on shape context, including: a memory and a processor; the memory is used to store computer - executable instructions, and the processor is used to execute the computer - executable instructions to implement all or part of the steps of the method described in the above - mentioned embodiment of the present invention.

[0103] This embodiment also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, it executes the method in any optional implementation manner of the above - mentioned embodiment. Among them, the storage medium can be implemented by any type of volatile or non - volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read - only memory (EEPROM for short), erasable programmable read - only memory (EPROM for short), programmable read - only memory (PROM for short), read - only memory (ROM for short), magnetic memory, flash memory, a magnetic disk or an optical disc.

[0104] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. For technical details not described in detail in this embodiment, reference can be made to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0105] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A self-calibration method for RGB-T binocular camera system parameters based on shape context, characterized in that: including Synchronously collect multiple frames of RGB images and thermal imaging images of an object from different perspectives, respectively extract the contours of the images by using a contour extraction algorithm, sample the contours, obtain a set of contour points, and calculate the shape context histogram Compare the shape context histograms to obtain a set of matching points. When the number of matching points in the set exceeds a set threshold, obtain the essential matrix of the matching points based on the RANSAC algorithm Obtain the decomposed external parameters by SVD decomposing the essential matrix, optimize the decomposed external parameters through a non-linear optimization technique, and complete the self-calibration of the RGB-T dual-camera system parameters by using the optimized external parameters 2. The self-calibration method for RGB-T binocular camera system parameters based on shape context according to claim 1, characterized in that: The contour extraction algorithms include the Canny contour extraction method, the Blob block extraction and contour parsing method, and the method of using image foreground segmentation and then extracting the contour; extracting the contour C A from the RGB image I A , and extracting the contour C B from the thermal imaging image I B .

3. The method for self-calibrating the parameters of the RGB-T binocular camera system based on shape context according to claim 2, wherein: The calculating the shape context histogram includes the following contents Sample the contour C and the contour C respectively using the Jitendra sampling method A and the contour C B to obtain the contour point sets and and calculate the shape context histogram sets {H A} and the set {H B}; For each set of sampling points \(P = \{p_1, p_2, p_3, \ldots, p\}\) n}, a polar coordinate system with a certain number of pole pairs is constructed centered at the points in \(P\). i Taking the points in \(P\) as the center, i using the points in \(P\) as the reference points, i N concentric circles are established at logarithmic distance intervals within the region with the points in \(P\) as the center and a radius of \(r\). The region is divided into M equal parts along the circumferential direction to form region separations. Calculate the angles and distances of other points in the set of sampling points relative to point P i and map the points into the region of the pole-pair coordinate system, count the number of points in each region, and form a histogram; Normalize the statistically obtained histogram to obtain the set of shape context histograms {H A} and the set {H B}.

4. The method for self-calibrating the parameters of the RGB-T binocular camera system based on shape context according to claim 3, wherein: The matching point set includes a shape context histogram set {H A } i , calculate and shape context histogram set {H B }Element q j The distance of points less than the distance judgment threshold λ is taken as the set {H A } i The matching points are put into the matching point set {p||q}. The distance calculation formula between the elements in the set is: Where, C i,j is the distance between element p i and element q j , h i (k) is the shape context histogram of element p i , and h j (k) is the shape context histogram of element q j . Calculate the matching points for the images from other perspectives using the same process. When the number of the set of matching points {p||q} exceeds the set threshold σ, continue with the calculation of the external parameters; otherwise, the calculation fails and the object to be collected is searched for again 5. The method for self-calibrating the parameters of the RGB-T binocular camera system based on shape context according to claim 4, wherein: The obtaining the essential matrix of the matching points based on the RANSAC algorithm includes the following contents Randomly select 8 feature point pairs from the list of matching points, and the set of feature point pairs is represented as {p i ||q i | i = 1,..., 8}; Convert the pairs of feature points in the set to the camera space through the camera internal parameter matrices K1 and K2, form matrix x1 and matrix x2, and solve for the essential matrix E, that is, use the least squares method to make the essential matrix E satisfy the following constraint conditions, expressed as Construct the camera internal parameter matrix as where f x and f y are the focal lengths of the camera on the x-axis and y-axis respectively, and c x and c y are the principal point coordinates of the image on the x-axis and y-axis respectively; The point p is represented in homogeneous coordinates as: (x, y, 1), and its conversion result is For each pair of normalized matching points (p i , q i ), assuming p i = (u i v i 1) T , q i = (u' i v' i 1) T Expand the essential matrix E into a 9 - dimensional vector E = [e 11 e 12 e 13 e 21 e 22 e 23 e 31 e 32 e 33 ) T , then the equation is expressed as: [u' i u i u' i v i u' i v' i u i v' i v i v' i u i v i 1]·e = 0 Combine the equations of all the matching points into an m×9 matrix A, where m≥8, expressed as A·e = 0 Solve by the least squares method, that is, perform singular value decomposition (SVD) on the matrix, then the essential matrix E corresponds to the 8th row vector of V T ; A = UΣV T Calculate the remaining matching point pairs in the set {p||q}, and if they satisfy the residual constraint, mark them as inliers, and record the total number of inliers N and the corresponding essential matrix E; If the total number of interior points N calculated new > the existing total number of interior points N, then update the essential matrix E = E new , and update N = N new ; If the number of iteration times or the total tolerance error quantity exceeds the threshold μ, stop the iteration 6. The method for self-calibrating RGB-T binocular camera system parameters based on shape context according to claim 5, characterized in that: The singular value decomposition of the matrix includes using the singular value decomposition of the essential matrix E, setting the third singular value to 0 to make it satisfy the rank-2 constraint, and obtaining the initial camera rotation parameters and translation parameters according to the singular value decomposition result 7. The method for self-calibrating RGB-T binocular camera system parameters based on shape context according to claim 6, characterized in that: The optimizing the decomposed external parameters through a non-linear optimization technique includes the following contents Using non-linear optimization techniques, the points in the RGB image within the set of interior points are reprojected back to the points in the thermal image and the sum of the distances to their corresponding points is minimized to obtain the final rotation parameter matrix R and translation parameter vector t, that is, the optimized extrinsic parameters of the camera are obtained, and the self-calibration of the camera is completed.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the method for self-calibrating the RGB-T dual-camera system parameters based on shape context according to any one of claims 1 to 7 9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the method for self-calibrating the RGB-T dual-camera system parameters based on shape context according to any one of claims 1 to 7

Images