Internal reference calibration device and method based on camera module aligning technology

Abstract

The invention relates to an internal reference calibration device and method based on a camera module aligning technology, and the device comprises a profile main frame, and the two sides of the profile main frame are provided with linear lifting guide rails which are used for enabling a lifting support provided with a five-axis debugging platform to slide up and down along the linear lifting guide rails. The top of the five-axis debugging platform is provided with a fixing platform used for fixing a camera imaging module, and the top of the profile main frame is provided with an imaging calibration mechanism. And the imaging calibration mechanism comprises a glass checkerboard calibration plate and a point-like laser emission structure which is arranged at the central position of the glass checkerboard calibration plate and is perpendicular to the glass checkerboard calibration plate, so that compared with the prior art, the cost can be reduced, and the imaging accuracy of the camera can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of imaging equipment, and in particular to an internal parameter calibration device and method based on camera module centering technology. Background Art

[0002] With the rapid development of industrial cameras, technologies such as depth imaging and structured light imaging have been applied to industrial measurement, 3D modeling, machine vision, and face recognition. In addition, the research on depth imaging and structured light imaging is also very hot in China, especially in the fields of industrial measurement and machine vision, and the accuracy of the image is the key to determining the accuracy of ranging.

[0003] Currently, most module factories at home and abroad use fully automatic AA dispensing module centering technology to ensure the consistency of the center of the lens and the camera imaging sensor. Although this method is highly efficient, the R&D cost is huge, which is unacceptable to many small and medium-sized enterprises.

[0004] Therefore, there is an urgent need for a low-cost internal parameter calibration device based on camera module centering technology. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an internal parameter calibration device and method based on camera module centering technology, which solves the technical problem of relatively high cost in the prior art.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] In the first aspect, an embodiment of the present invention provides an internal parameter calibration device based on the camera module centering technology, including a profile main frame, both sides of the profile main frame are provided with linear lifting guide rails for allowing a lifting bracket with a five-axis debugging platform to slide up and down there along, and the top of the five-axis debugging platform is provided with a fixed platform for fixing the camera imaging module, and the top of the profile main frame is provided with an imaging calibration mechanism, and the imaging calibration mechanism includes a glass checkerboard calibration plate and a point laser emission structure arranged at the center position of the glass checkerboard calibration plate and perpendicular to the glass checkerboard calibration plate.

[0010] In a possible embodiment, the point laser emitting device includes a dichroic mirror, a point infrared band laser and a point visible light band laser, the central axis of the point infrared band laser is parallel to the glass checkerboard calibration plate, and the central axis of the point visible light band laser is perpendicular to the glass checkerboard calibration plate, and a dichroic mirror is arranged at the laser intersection of the point infrared band laser and the point visible light band laser, and the dichroic mirror is used to refract the infrared band light beam emitted by the point infrared band laser by 90 degrees or directly transmit the visible light band emitted by the point visible light band laser.

[0011] In a possible embodiment, the wavelength of the infrared band light beam emitted by the point infrared band laser is 850nm, 905nm or 940nm; the imaging calibration mechanism further includes a backlight source lamp board arranged above the glass chessboard calibration plate, and a plurality of infrared lamp beads are arranged on the side of the backlight source lamp board close to the glass chessboard calibration plate, and the plurality of infrared lamp beads include 850nm infrared lamp beads, 905nm infrared lamp beads, 940nm infrared lamp beads and visible white light infrared lamp beads.

[0012] In a possible embodiment, a light homogenizing film is attached to the back of the glass checkerboard calibration plate.

[0013] In a possible embodiment, both sides of the lifting bracket are fixedly connected to a lifting slide sliding in the linear lifting guide rail, and the lifting slide is provided with fixing hand wheels on both sides of the lifting slide in the sliding direction.

[0014] In a possible embodiment, the distance between the top of the five-axis debugging platform and the point-shaped laser emitting structure is 0.5 m-2 m.

[0015] In a possible embodiment, a black and white dot is provided at the center of the glass checkerboard calibration plate for enabling the camera to identify the center position during the internal parameter calibration.

[0016] In a possible embodiment, a roller sliding mechanism is provided at the bottom of the profile main frame.

[0017] In a possible embodiment, a set screw for fine-tuning the camera imaging module is provided on the fixed platform.

[0018] In a second aspect, an embodiment of the present invention provides an internal parameter calibration method of an internal parameter calibration device based on the camera module centering technology described in any one of the first aspects, the method comprising: when a point laser point coincides with the center position of the image sensor coordinates in the camera imaging module and after the fine-tuning of the camera imaging module is completed, determining the three-dimensional coordinates of each corner point of the glass chessboard calibration plate according to the chessboard specifications of the glass chessboard calibration plate and the distance from the optical center of the camera imaging module to the glass chessboard calibration plate; controlling the camera imaging module to collect an initial image of the glass chessboard calibration plate; preprocessing the initial image to obtain a preprocessed image; performing corner point detection on the preprocessed image using a corner point detection algorithm to obtain a candidate corner point set, and screening the candidate corner point set to obtain a target corner point set; according to the structure of the glass chessboard calibration plate, matching the target corner point set according to the rows and columns of the chessboard to obtain several groups of matched image corner point coordinates and three-dimensional coordinates; minimizing the processing of several groups of matched image corner point coordinates and three-dimensional coordinates using a nonlinear optimization algorithm to obtain the internal parameters and distortion coefficients of the camera imaging module.

[0019] (III) Beneficial effects

[0020] The beneficial effects of the present invention are:

[0021] The embodiment of the present application provides an internal parameter calibration device and method based on the camera module centering technology, the device includes a profile main frame, both sides of the profile main frame are provided with linear lifting guide rails for allowing a lifting bracket with a five-axis debugging platform to slide up and down along the guide rails, and the top of the five-axis debugging platform is provided with a fixed platform for the camera imaging module, and the top of the profile main frame is provided with an imaging calibration mechanism, and the imaging calibration mechanism includes a glass checkerboard calibration plate and a point laser emission structure arranged at the center position of the glass checkerboard calibration plate and perpendicular to the glass checkerboard calibration plate. Compared with the prior art, it can not only reduce the R&D cost, but also improve the accuracy of camera imaging.

[0022] In order to make the above-mentioned objectives, features and advantages to be achieved by the embodiments of the present application more obvious and understandable, the following specifically cites preferred embodiments and describes them in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1A structural block diagram of an internal parameter calibration device based on a camera module centering technology provided in an embodiment of the present application is shown;

[0025] Figure 2 A schematic diagram showing a front view of a glass chessboard calibration plate provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram showing a partial cross-sectional view of an imaging calibration mechanism provided in an embodiment of the present application;

[0027] Figure 4 A cross-sectional schematic diagram of a point-shaped laser emission structure provided in an embodiment of the present application is shown;

[0028] Figure 5 A bottom enlarged schematic diagram of an imaging calibration mechanism provided in an embodiment of the present application is shown;

[0029] Figure 6 A schematic diagram of a five-axis debugging platform provided in an embodiment of the present application is shown;

[0030] Figure 7 A schematic diagram showing an enlarged view of a backlight source lamp panel provided in an embodiment of the present application is shown;

[0031] Figure 8 A flow chart of an internal parameter calibration method of an internal parameter calibration device based on the camera module centering technology described in the first embodiment provided by an embodiment of the present application is shown;

[0032] Fig. 9A A schematic diagram of an uncommissioned system provided in an embodiment of the present application is shown;

[0033] Fig. 9B A schematic diagram showing a completed debugging provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0035] In order to solve the technical problem of relatively high R&D cost existing in the prior art, the embodiment of the present application provides an internal parameter calibration device and method based on camera module centering technology, the device includes a profile main frame, both sides of the profile main frame are provided with linear lifting guide rails for allowing a lifting bracket with a five-axis debugging platform to slide up and down along it, and a fixed platform for the camera imaging module is provided on the top of the five-axis debugging platform, and an imaging calibration mechanism is provided on the top of the profile main frame, and the imaging calibration mechanism includes a glass checkerboard calibration plate and a point laser emission structure arranged at the center position of the glass checkerboard calibration plate and perpendicular to the glass checkerboard calibration plate. Compared with the prior art, it can not only reduce the R&D cost, but also improve the accuracy of camera imaging.

[0036] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0037] In order to facilitate understanding of the embodiments of the present application, some terms involved in the present application are explained as follows:

[0038] "Camera module centering technology": It refers to ensuring precise alignment between the lens and image sensor (such as CMOS or CCD) during the camera module manufacturing process to achieve optimal optical performance and image quality.

[0039] First embodiment

[0040] See also Figure 1 , Figure 1 FIG. 1 shows a structural block diagram of an internal parameter calibration device based on a camera module centering technology provided in an embodiment of the present application. Figure 1 As shown, the internal reference calibration device comprises a profile main frame (1), both sides of the profile main frame (1) are provided with linear lifting guide rails (4) for enabling a lifting bracket (3) on which a five-axis debugging platform (2) is placed to slide up and down, and the top of the five-axis debugging platform (2) is provided with a fixing platform (21) for fixing a camera imaging module, and the top of the profile main frame (1) is provided with an imaging calibration mechanism (5), and the imaging calibration mechanism (5) comprises a glass checkerboard calibration plate and a point-shaped laser emission structure arranged at the center of the glass checkerboard calibration plate and arranged perpendicular to the glass checkerboard calibration plate. For details, see Figure 2 and Figure 3 .

[0041] It should be understood that the devices involved in the internal reference calibration device can be configured according to actual needs, and the embodiments of the present application are not limited thereto.

[0042] Optionally, continue to see Figure 2 , Figure 2 FIG. 1 is a schematic diagram showing a front view of a glass checkerboard calibration plate provided in an embodiment of the present application. Figure 2 As shown, the front of the imaging calibration mechanism (5) is a glass checkerboard calibration plate, and the glass checkerboard calibration plate can be composed of a plurality of 40mm*40mm black and white grids, and near the center is composed of two black and white dots with a diameter of 30mm, so as to facilitate the camera to identify the center position during the internal reference calibration. And, the center position of the glass checkerboard calibration plate is a point-shaped laser emission structure, and the Figure 2 The coordinates in the figure show the origin, X direction, and Y direction of the chessboard.

[0043] Optionally, continue to see Figure 3 , Figure 3 FIG. 1 is a schematic diagram showing a partial cross-sectional view of an imaging calibration mechanism provided in an embodiment of the present application. Figure 3 As shown, a layer of uniform light film is attached to the back of the glass chessboard calibration plate (51), so as to ensure that the light distribution of the infrared light source and the visible light source is uniform. In addition, the center of the glass chessboard calibration plate is a point-shaped laser emission structure (52), and the main structure of the imaging calibration mechanism (5) is a profile frame (53), and the top of the profile frame (53) includes a lamp holder structure (54) for fixing the backlight source and a backlight source lamp board (55), and according to the proportion of the chessboard area, the backlight source lamp board (55) can be evenly distributed in multiple pieces (for example, 20 pieces), and the circuit boards are connected in series using power cords. In addition, the backlight source circuit board is powered by a power adapter that converts 220V to 24V, and an external knob switch is connected to selectively specify a light source for fill light. In addition, the backlight source circuit board has a built-in welding nut, which is fixed to the lamp holder structure with screws, and the lamp holder structure is fixed to the profile of the top imaging calibration mechanism with screws.

[0044] In addition, the imaging calibration mechanism can be located on the top of the entire device and connected and fixed to the main profile frame using screws.

[0045] In addition, the glass chessboard calibration plate (51) can be embedded in the profile of the top imaging calibration mechanism, and the glass chessboard can be fixed through the card slots inside the profile.

[0046] Further, see Figure 4 , Figure 4 FIG. 4 is a cross-sectional schematic diagram of a point-shaped laser emission structure provided in an embodiment of the present application. Figure 4As shown, the point-shaped laser emission structure (52) comprises a two-phase color separation mirror (521), a point-shaped infrared band laser (522) and a point-shaped visible light band laser (523); the central axis of the point-shaped infrared band laser (522) is parallel to the glass checkerboard calibration plate (51); the central axis of the point-shaped visible light band laser (523) is perpendicular to the glass checkerboard calibration plate (51); and a two-phase color separation mirror (521) is arranged at the intersection of the laser beams of the point-shaped infrared band laser (522) and the point-shaped visible light band laser (523); and the two-phase color separation mirror (521) is used to refract the infrared band light beam emitted by the point-shaped infrared band laser (522) by 90 degrees or directly transmit the visible light band emitted by the point-shaped visible light band laser (523). Furthermore, the point laser emitting structure (52) can realize the switch control of the light source by using two sets of rocker switches to control the two point laser emitters through an external 12V power supply.

[0047] For example, Figure 4 As shown, a hole can be opened at the center of the glass checkerboard calibration plate (51) for placing a point-shaped laser emitting structure, and the point-shaped infrared band laser (522) in the point-shaped laser emitting structure (52) can be replaced with lasers of different infrared bands according to the band requirements of the camera imaging module. For example, the point-shaped infrared band laser (522) can be replaced with a point-shaped infrared band laser emitting a wavelength of 850nm; the point-shaped infrared band laser (522) can also be replaced with a point-shaped infrared band laser emitting a wavelength of 905nm; and the point-shaped infrared band laser (522) can also be replaced with a point-shaped infrared band laser emitting a wavelength of 940nm. And, the placement mechanism (524) of the point-shaped infrared band laser in the point-shaped laser emitting structure (52) and the point-shaped infrared band laser (522) are assembled with each other and fixed in the central opening of the glass checkerboard calibration plate. And the placement mechanism (525) of the point-shaped visible light band laser in the point-shaped laser emission structure (52) can be combined with the placement mechanism (524) of the point-shaped infrared band laser, and the wavelength of the visible light beam emitted by the point-shaped visible light band laser (523) is 520nm. And the two-phase color separation mirror (521) in the point-shaped laser emission structure (52) is fixed on the placement mechanism (524) of the point-shaped infrared band laser, and the two-phase color separation mirror (521) can be placed at 45 degrees, and the two-phase color separation mirror (521) refracts the infrared band (850nm, 905nm, 940nm) light beam by 90 degrees, while the 520nm visible light band laser can directly pass through. And, Figure 4 (526) in the figure is a schematic diagram of the light path directions of the point-shaped visible light band and the point-shaped infrared band, and the light path direction of the point-shaped infrared band and the light path direction of the point-shaped visible light band are emitted from a light hole and overlap with each other.

[0048] It should be noted here that when performing the camera intrinsic calibration method, if the intrinsic calibration of an infrared camera is performed, the point infrared band laser (522) can be turned on; if the intrinsic calibration of an infrared visible light camera is performed, the point visible light band laser (523) can be turned on.

[0049] Optionally, continue to see Figure 5 , Figure 5 FIG. 1 shows an enlarged bottom schematic diagram of an imaging calibration mechanism provided in an embodiment of the present application. Figure 5 As shown, the linear lifting guide rails (4) on the imaging calibration mechanism (1) can be distributed on both sides of the imaging calibration mechanism, and the lifting bracket (3) slides up and down along the linear lifting guide rails (4), and the two ends of the lifting bracket (3) can be fixedly provided with lifting slides (31), and the lifting slides (31) are provided with locking hand wheels (311) for facilitating locking after height adjustment.

[0050] Optionally, continue to see Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a five-axis debugging platform provided in an embodiment of the present application. Figure 6 As shown, E1 is the X-axis pitch angle debugging platform; E2 is the Y-axis pitch angle debugging platform; E3 is the rotation angle debugging platform; E4 is the X-axis translation debugging platform; E5 is the Y-axis translation debugging platform. Thus, the five-axis debugging platform can adjust the position of the camera imaging module in five dimensions, and ensure the vertical relationship between the imaging module and the point laser optical path.

[0051] It should be noted here that the distance between the top of the five-axis debugging platform and the point laser emitting structure is 0.5m-2m, that is, in order to meet the different focusing distances of different cameras, a lifting mechanism is set up, and linear guide rails are arranged on both sides to achieve a focusing distance of 0.5m-2m.

[0052] Furthermore, a fixed platform (21) of a camera imaging module is arranged on the top of the five-axis debugging platform, and nut micrometer heads (211) are distributed at four positions on the fixed platform (21), which can fine-tune the lens holder or lens part of the imaging module in the X and Y directions, so as to achieve the coincidence of the point-shaped laser point with the coordinate center position of the image sensor of the imaging module.

[0053] Optionally, continue to see Figure 7 , Figure 7 FIG. 1 is a schematic diagram showing an enlarged view of a backlight source panel provided in an embodiment of the present application. Figure 7As shown, a side of the backlight source lamp board close to the glass checkerboard calibration board (i.e., the front side of the glass checkerboard calibration board) is provided with a plurality of infrared lamp beads, and the plurality of infrared lamp beads may include 850nm infrared lamp beads (F1), 905nm infrared lamp beads (F2), 940nm infrared lamp beads (F3) and visible white light infrared lamp beads (F4). In addition, a side of the backlight source lamp board away from the glass checkerboard calibration board (i.e., the back side of the glass checkerboard calibration board) is provided with a series power connection socket.

[0054] For example, the backlight circuit board can be powered by a 24V power supply. In addition, 70 850nm infrared lamp beads are arranged in a single light source circuit board, and the light angle of each lamp is 120 degrees; 30 905nm infrared lamp beads are arranged in a single light source circuit board, and the light angle of each lamp is 120 degrees; 70 940nm infrared lamp beads are arranged in a single light source circuit board, and the light angle of each lamp is 120 degrees; 70 visible white light lamp beads are arranged in a single light source circuit board, and the light angle of each lamp is 120 degrees.

[0055] In summary, with the help of the above technical solutions, compared with the prior art, the present application can not only reduce the R&D cost, but also improve the accuracy of camera imaging.

[0056] It should be understood that the above-mentioned internal parameter calibration device based on the camera module centering technology is only exemplary, and those skilled in the art can make various modifications according to the above-mentioned method, and the modified scheme also belongs to the protection scope of this application.

[0057] Second embodiment

[0058] See also Figure 8 , Figure 8 A flowchart of an internal parameter calibration method of an internal parameter calibration device based on the camera module centering technology described in the first embodiment provided by an embodiment of the present application is shown, specifically:

[0059] Step S810, when the point laser spot coincides with the center position of the image sensor coordinates in the camera imaging module and the camera imaging module is fine-tuned, determine the three-dimensional coordinates of each corner point of the glass checkerboard calibration plate according to the chessboard specifications of the glass checkerboard calibration plate and the distance from the optical center of the camera imaging module to the glass checkerboard calibration plate.

[0060] It should be understood that the specific process of achieving the coincidence of the point-shaped laser point with the coordinate center position of the image sensor in the camera imaging module can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0061] Optionally, the camera imaging module is fixed on a camera imaging module fixing platform. At this time, due to assembly errors, the camera imaging module and the point laser are not in a perpendicular relationship. Direct debugging will affect the final optical center accuracy.

[0062] And, if Fig. 9A and Fig. 9B As shown, for the convenience of debugging, a point laser (for example, a point infrared band laser or a point visible light band laser) is emitted to the chip of the camera imaging module. The principle of laser reflection is used to fine-tune the five-axis debugging platform, and the incident light of the point laser and the reflected light on the chip surface of the camera imaging module are adjusted to overlap. At this time, the camera imaging module and the point laser are perpendicular to each other.

[0063] It should be noted here that lasers of different infrared bands can be switched according to the band requirements of the camera imaging module.

[0064] It should also be understood that the specific process of fine-tuning the camera imaging module can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0065] Optionally, fine-tune the XY dimensions, and use the host computer to automatically find the brightest point in the image in real time. The coordinates of the brightest point are the actual optical center, and the center coordinates are the actual optical center. The actual optical center is printed on the host computer. When the deviation between the actual optical center and the sensor pixel center coordinates is within a preset number of pixels (for example, 2 pixels), it is determined to be qualified, and the fine-tuning is ended. The depth camera and visible light camera lenses are glued or screwed in place. In addition, during debugging, the image can be obtained in the debugging host computer software.

[0066] It should also be understood that the specific process of step S810 can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0067] Optionally, when the point-shaped laser spot coincides with the center position of the image sensor coordinates in the camera imaging module and the camera imaging module is fine-tuned, the camera can be placed in front of the corner point at the center of the chessboard, and the distance from the optical center of the camera imaging module to the chessboard is measured as z, and the camera translation vector [0,0,z] relative to the chessboard center is obtained, and the three-dimensional coordinates [qi.x,qi.y,0] of each corner point can be obtained according to the chessboard specifications, and the three-dimensional coordinates Q of each corner point can be obtained in combination with the translation vector z. i =[qi.x,qi.y,z], where qi.x represents the x-coordinate of the i-th corner point; qi.y represents the y-coordinate of the i-th corner point.

[0068] Step S820, controlling the camera imaging module to collect an initial image of the glass checkerboard calibration plate.

[0069] Step S830, preprocessing the initial image to obtain a preprocessed image.

[0070] It should be understood that the specific process included in the preprocessing can be set according to actual needs, and the embodiments of the present application are not limited to this.

[0071] For example, the preprocessing may include converting the initial image into a grayscale image and performing noise reduction and contrast enhancement.

[0072] Step S840: Use a corner detection algorithm to perform corner detection on the preprocessed image to obtain a candidate corner point set, and then filter the candidate corner point set to obtain a target corner point set. Corner points are generally considered to be pixels with high angle changes in the image.

[0073] It should be understood that the specific algorithm of the corner detection algorithm can be set according to actual needs, and the embodiments of the present application are not limited thereto.

[0074] For example, the corner detection algorithm may be a Harris corner detection algorithm or a Shi-Tomasi corner detection algorithm, etc. These algorithms usually determine the corner positions based on intensity changes and local structures in an image.

[0075] It should also be understood that the specific screening algorithm for screening the candidate corner point set can also be set according to actual needs, and the embodiments of the present application are not limited thereto.

[0076] For example, since the corner detection algorithm may detect some falsely detected corners, it is necessary to filter and remove the detected corners. In addition, common filtering methods include setting values, sorting according to the quality index of the corners, or using techniques such as non-maximum suppression.

[0077] Step S850, according to the structure of the glass chessboard calibration plate, the target corner point set is matched according to the rows and columns of the chessboard to obtain several sets of matched image corner point coordinates and three-dimensional coordinates.

[0078] Specifically, according to the structure of the chessboard, the detected corner points are matched according to the rows and columns of the chessboard. Also, the adjacency and spacing of the corner points can usually be used for matching to obtain several sets of matched image corner point coordinates P i With the three-dimensional coordinates Q i .

[0079] Step S860, using a nonlinear optimization algorithm to minimize several sets of matched image corner point coordinates and three-dimensional coordinates to obtain the internal parameters and distortion coefficients of the camera imaging module.

[0080] It should be understood that the specific function of the nonlinear optimization algorithm can be set according to actual needs, and the embodiments of the present application are not limited thereto.

[0081] Optionally, the calculation expression of the nonlinear optimization algorithm is as follows:

[0082]

[0083] Where U represents the minimum value of the image corner coordinates and three-dimensional coordinates after several sets of matching; i represents different coordinate points; f(R*Q i ,α) function is the reprojection function, and it represents a three-dimensional coordinate Q i Use the internal parameters and distortion coefficient α to project to the image coordinate system; R is the rotation matrix to be optimized, and its initial value is the unit matrix; α is the internal parameter to be optimized and the distortion coefficient uses the theoretical value as the initial value, Q i is the three-dimensional coordinate, P i are the coordinates of the image corners.

[0084] Therefore, with the help of the above technical solution, the present application can solve the problem of optical center offset between infrared depth cameras, visible light camera lenses and camera imaging sensors, and can quickly improve the detection accuracy and efficiency of the lens module through the calibration of internal parameters.

[0085] It should be understood that the above internal reference calibration method is only exemplary, and those skilled in the art may make various modifications based on the above method, and the modified scheme also falls within the protection scope of the present application.

[0086] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0088] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0089] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0090] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0091] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. An internal parameter calibration device based on camera module centering technology, characterized in that: It comprises a profile main frame, both sides of which are provided with linear lifting guide rails for enabling a lifting bracket with a five-axis debugging platform to slide up and down, and the top of the five-axis debugging platform is provided with a fixed platform for fixing a camera imaging module, and the top of the profile main frame is provided with an imaging calibration mechanism, and the imaging calibration mechanism comprises a glass checkerboard calibration plate and a point-shaped laser emission structure which is arranged at the center position of the glass checkerboard calibration plate and is arranged perpendicular to the glass checkerboard calibration plate.

2. The internal reference calibration device according to claim 1, characterized in that: The point laser emitting device comprises a two-phase color dichroic mirror, a point infrared band laser and a point visible light band laser, wherein the central axis of the point infrared band laser is parallel to the glass checkerboard calibration plate, and the central axis of the point visible light band laser is perpendicular to the glass checkerboard calibration plate, and the two-phase color dichroic mirror is arranged at the laser intersection of the point infrared band laser and the point visible light band laser, and the two-phase color dichroic mirror is used to refract the infrared band light beam emitted by the point infrared band laser by 90 degrees or directly transmit the visible light band emitted by the point visible light band laser.

3. The internal reference calibration device according to claim 2, characterized in that: The wavelength of the infrared band light beam emitted by the point infrared band laser is 850nm, 905nm or 940nm; the imaging calibration mechanism further includes a backlight source lamp board arranged above the glass chessboard calibration plate, and a plurality of infrared lamp beads are arranged on the side of the backlight source lamp board close to the glass chessboard calibration plate, and the plurality of infrared lamp beads include 850nm infrared lamp beads, 905nm infrared lamp beads, 940nm infrared lamp beads and visible white light infrared lamp beads.

4. The internal reference calibration device according to claim 1, characterized in that: A light-homogenizing film is attached to the back of the glass chessboard calibration plate.

5. The internal reference calibration device according to claim 1, characterized in that: Both sides of the lifting bracket are fixedly connected to a lifting slide sliding in the linear lifting guide rail, and the lifting slide is provided with fixing hand wheels on both sides of the sliding direction thereof.

6. The internal reference calibration device according to claim 5, characterized in that: The distance between the top of the five-axis debugging platform and the point-shaped laser emitting structure is 0.5m-2m.

7. The internal reference calibration device according to claim 1, characterized in that: The center position of the glass chessboard calibration plate is provided with black and white dots for enabling the camera to identify the center position during internal parameter calibration.

8. The internal reference calibration device according to claim 1, characterized in that: A roller sliding mechanism is arranged at the bottom of the profile main frame.

9. The internal reference calibration device according to claim 1, characterized in that: The fixing platform is provided with a set screw for fine-tuning the camera imaging module.

10. An internal parameter calibration method based on the internal parameter calibration device based on the camera module centering technology according to any one of claims 1 to 9, characterized in that: include: When the point-shaped laser point coincides with the center position of the image sensor coordinates in the camera imaging module and the fine-tuning of the camera imaging module is completed, the three-dimensional coordinates of each corner point of the glass chessboard calibration plate are determined according to the chessboard specifications of the glass chessboard calibration plate and the distance from the optical center of the camera imaging module to the glass chessboard calibration plate; Controlling the camera imaging module to collect an initial image of the glass chessboard calibration plate; Preprocessing the initial image to obtain a preprocessed image; Performing corner point detection on the preprocessed image using a corner point detection algorithm to obtain a candidate corner point set, and screening the candidate corner point set to obtain a target corner point set; According to the structure of the glass chessboard calibration plate, the target corner point set is matched according to the rows and columns of the chessboard to obtain several sets of matched image corner point coordinates and three-dimensional coordinates; The plurality of groups of matched image corner point coordinates and three-dimensional coordinates are processed by minimizing the nonlinear optimization algorithm to obtain the internal parameters and distortion coefficients of the camera imaging module.

Images