Visual inspection information acquisition device and method based on pitch angle calibration image

Abstract

The invention discloses a visual inspection information acquisition device and method based on a pitch angle calibration image, and belongs to the technical field of machine visual inspection, the device comprises an industrial camera with a zoom lens, a holder device, a fixing device, an inclination angle sensor, a signal acquisition synchronizer and an image automatic calibration module; wherein the industrial camera is used for capturing an image of a surface area of a measured structure; the inclination angle sensor is used for determining a pitch angle formed between the camera lens axis and the surface normal of the measured structure; the signal acquisition synchronizer is used for ensuring data acquisition consistency between the industrial camera and the inclination angle sensor; and the image automatic calibration module is used for correcting distortion of the image captured by the camera due to the pitch angle to obtain a corrected image. By adopting the technical scheme of the invention, the accuracy of the image data can be ensured, so that the health detection and monitoring efficiency and precision of the engineering structure can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and in particular to a visual detection information acquisition device and method based on a pitch angle calibration image. Background Art

[0002] With the development of industrial automation and intelligence, machine vision technology is increasingly used in various structural inspection and monitoring fields. The application of traditional single sensor equipment in complex environments is greatly limited. Especially in the inspection process of large-scale engineering structures (such as bridges, tunnels, buildings, etc.), it is often necessary to perceive and identify surface cracks, deformations and other defects with high precision. However, traditional visual sensor equipment is limited by its single information collection method and it is difficult to provide comprehensive and accurate inspection results under changing environmental conditions.

[0003] In addition, existing visual inspection equipment usually relies on two-dimensional image data collected by industrial cameras and cannot effectively obtain depth information, resulting in a significant decrease in detection accuracy when the surface complexity is high or the lighting conditions are poor. At the same time, during the image acquisition process, the image distortion problem caused by the angle between the camera lens axis and the surface of the structure being tested is particularly obvious. The detection results often produce geometric distortion, affecting the reliability of the detection. In addition, in an environment with insufficient light, traditional equipment is difficult to ensure image clarity, further limiting its scope of application. Summary of the invention

[0004] The present invention provides a visual inspection information acquisition device and method based on pitch angle calibration images, so as to solve the technical problem that the image data collected in the prior art is not accurate enough, thus affecting the inspection accuracy of engineering structures.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] On the one hand, the present invention provides a visual inspection information acquisition device based on a pitch angle calibration image, which comprises: an industrial camera with a zoom lens, a pan-tilt device, a fixing device, an inclination angle sensor, a signal acquisition synchronizer and an image automatic calibration module; wherein the industrial camera is mounted on the pan-tilt device, and the pan-tilt device is mounted on the fixing device; the inclination angle sensor is mounted on the top of the industrial camera; the industrial camera and the inclination angle sensor are electrically connected to the signal acquisition synchronizer respectively;

[0007] The industrial camera is used to capture images of the surface area of ​​the structure being measured;

[0008] The tilt angle sensor is used to detect the tilt angle between the industrial camera and the horizontal plane, so as to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure;

[0009] The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor to ensure the consistency of data acquisition between the industrial camera and the tilt angle sensor;

[0010] The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, so as to obtain a corrected image.

[0011] Furthermore, the visual detection information collection device also includes an adaptive fill light;

[0012] The adaptive fill light is electrically connected to the signal acquisition synchronizer, and the signal acquisition synchronizer synchronously controls the start and stop of the adaptive fill light, the industrial camera and the tilt angle sensor;

[0013] The adaptive fill light can automatically adjust the brightness according to the intensity of the ambient light to ensure the clarity of the image captured by the industrial camera when capturing the image.

[0014] Furthermore, the visual detection information acquisition device also includes a multimodal data fusion module;

[0015] The multimodal data fusion module adopts a preset data fusion algorithm to synchronously process the image captured by the industrial camera and the angle information collected by the tilt angle sensor to achieve perception of the surface of the measured structure.

[0016] Furthermore, the image automatic calibration module is specifically used to: utilize the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor to first correct the object distance, and then use the horizontal center line of the image taken by the camera under the pitch angle condition as the boundary, and use the perspective transformation matrix to correct the distortion of the upper and lower areas of the image respectively; merge the corrected upper and lower areas of the image into a complete image to obtain a corrected image.

[0017] Furthermore, the object distance is first corrected by using the pitch angle between the axis of the camera lens and the surface normal of the structure under test collected by the tilt angle sensor, and then the horizontal center line of the image taken by the camera under the pitch angle condition is used as the boundary, and the perspective transformation matrix is ​​used to correct the distortion of the upper and lower areas of the image respectively; the upper and lower areas of the corrected image are merged into a complete image to obtain a corrected image, including:

[0018] The object distance is corrected by using the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor; wherein the corrected object distance S0 ′ for:

[0019]

[0020] Where S0 represents the original object distance; γ represents the pitch angle between the axis of the camera lens and the surface normal of the measured structure;

[0021] The upper half of the image is corrected by perspective transformation based on the horizontal center line of the image taken by the camera at a pitch angle. Before correction, four boundary points in the upper half of the image are selected, and their pixel coordinates are: (0,0), (w,0), (w,h / 2), (0,h / 2). After correction, the coordinates of the four boundary points become: (w,h / 2), (0,h / 2); where θ represents the camera angle of view; w represents the width of the image; h represents the height of the image; by changing the coordinates of the four boundary points in the upper half of the image and using the perspective transformation principle, the perspective transformation matrix H of the upper half of the image is obtained. 上 , and H 上 Applied to the upper area of ​​the image, it automatically corrects image deformation caused by distortion;

[0022] The horizontal center line of the image taken by the camera at a pitch angle is used as the boundary to perform perspective transformation correction on the lower half of the image. Before correction, four boundary points in the lower half of the image are selected, and their pixel coordinates are: (0,h / 2), (w,h / 2), (w,h), (0,h); after correction, the coordinates of the four boundary points become: (0,h / 2), (w,h / 2), According to the coordinate changes of the four boundary points in the lower half of the image, the perspective transformation matrix H of the lower half of the image is obtained. 下 , and H 下 Applied to the lower half of the image, it automatically corrects image deformation caused by distortion;

[0023] The upper half of the corrected image and the lower half of the image are merged into a complete image, ensuring a smooth transition between the two parts to obtain the corrected image.

[0024] On the other hand, the present invention also provides a method for collecting visual detection information based on a pitch angle calibration image, which is implemented by using the above-mentioned visual detection information collection device based on a pitch angle calibration image, and the method comprises:

[0025] The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor, and the industrial camera is used to capture an image of the surface area of ​​the measured structure; the tilt angle between the industrial camera and the horizontal plane is detected by the tilt angle sensor to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure;

[0026] The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, so as to obtain a corrected image.

[0027] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0028] The scheme of the present invention integrates multiple sensors such as industrial cameras, laser ranging sensors, high-precision tilt angle sensors and adaptive fill lights to develop a low-cost and high-precision visual inspection information acquisition device, which realizes the accurate acquisition and fusion processing of multi-dimensional data, and can significantly improve the perception accuracy of the structural surface. Through the built-in image automatic calibration module, the system can automatically correct the image distortion caused by the pitch angle deviation, ensure the accuracy of the image data, and thus significantly improve the measurement accuracy. The adaptive fill light function can intelligently adjust the brightness according to the ambient light to ensure that a clear image is still obtained in an environment with insufficient light. In addition, the multifunctional pan-tilt head has the ability to rotate at multiple angles and can flexibly adapt to the all-round detection needs of complex structures. The signal acquisition synchronizer ensures the synchronous data acquisition between the sensors, further improving the coordination and data consistency of the system. The entire device not only has the advantages of low cost and high precision, but also has good high-efficiency performance, and is particularly suitable for long-term detection and monitoring of large-scale engineering structures. The device can be widely used in buildings, bridges, tunnels and other fields, and can greatly improve the efficiency and reliability of engineering detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 It is a schematic diagram of a two-dimensional plane structure of a visual detection information acquisition device based on a pitch angle calibration image provided by an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of the three-dimensional structure of a visual detection information acquisition device based on a pitch angle calibration image provided by an embodiment of the present invention;

[0032] Figure 3 is a schematic diagram of the geometric relationship of image calibration when there is a pitch angle between the axis of the camera lens and the surface normal of the structure under test provided by an embodiment of the present invention;

[0033] Figure 4It is a schematic diagram of the geometric relationship of image calibration when there is a horizontal angle between the axis of the camera lens and the surface normal of the measured structure provided by an embodiment of the present invention.

[0034] Description of reference numerals:

[0035] 1. Industrial camera; 2. Zoom lens; 3. PTZ device; 4. Fixing device;

[0036] 5. Laser distance sensor; 6. Angle adjustment button; 7. Tilt angle sensor;

[0037] 8. Adaptive fill light; 9. Surface of the structure being measured. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0039] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0040] This embodiment provides a visual detection information acquisition device based on a pitch angle calibration image, such as Figure 1 and Figure 2 As shown, the device includes: an industrial camera 1, a zoom lens 2, a pan-tilt device 3, a fixing device 4 (a tripod in this embodiment), a variable-angle laser ranging sensor component, a tilt angle sensor 7, a signal acquisition synchronizer, an adaptive fill light 8, a multimodal data fusion module and an image automatic calibration module.

[0041] The industrial camera 1 is used to capture images of the surface area of ​​the structure to be measured, and a zoom lens 2 is installed at its front end, so that the focal length can be automatically adjusted according to the distance of the surface 9 of the structure to be measured to ensure the clarity of the image at different object distances. The bottom of the industrial camera 1 is fixed on the pan-tilt device 3, which has a 180-degree rotation function, supports the multi-angle free rotation and fixation of the industrial camera 1, and thus can realize multi-angle image acquisition, and supports installation on the fixing device 4 to meet the needs of different scenarios.

[0042] A variable-angle laser distance sensor assembly is installed in the middle part of the industrial camera 1, which is used to measure the distance information between the optical center of the camera and different positions of the target surface area in real time, thereby improving the depth perception accuracy and determining the horizontal angle formed between the axis of the camera lens and the surface normal of the measured structure; the variable-angle laser distance sensor assembly includes three laser distance sensors 5 and an angle adjustment button 6, one of which has an axis coincident with the optical axis of the industrial camera 1, and the other two sensors are symmetrically arranged on both sides of the optical axis of the industrial camera 1, and the angle can be adjusted by the angle adjustment button 6, thereby providing more comprehensive distance measurement information and being able to quickly determine the horizontal angle between the axis of the camera lens and the surface normal of the measured structure.

[0043] A tilt angle sensor 7 is installed at the rear end of the top of the industrial camera 1 to measure the tilt angle between the industrial camera 1 and the horizontal plane with high precision, ensure the measurement accuracy of the equipment at different angles, and quickly determine the pitch angle between the camera lens axis and the surface normal of the measured structure.

[0044] An adaptive fill light 8 is installed at the front end of the top of the industrial camera 1, which can intelligently adjust the brightness according to the ambient light conditions to ensure the acquisition of high-quality images in low-light environments and ensure the clarity of image acquisition.

[0045] After the hardware equipment is installed, a signal acquisition synchronizer is used to connect and synchronously control the start and stop of the industrial camera 1, the laser ranging sensor 5, the tilt angle sensor 7 and the adaptive fill light 8, so as to ensure the synchronous data acquisition of the industrial camera 1, the laser ranging sensor 5 and the tilt angle sensor 7, and realize the integration and data consistency of the whole system.

[0046] The multimodal data fusion module adopts a data fusion algorithm to fuse the image data acquired by the industrial camera 1, the distance information of the laser ranging sensor 5, and the angle information of the tilt angle sensor 7 in real time and synchronously process them to achieve accurate perception of the target area.

[0047] An image automatic calibration module is embedded on the basis of the above device to correct the distortion of the image captured by the industrial camera 1 due to the pitch angle between the lens axis of the industrial camera 1 and the normal of the surface 9 of the measured structure, so as to obtain a corrected image. Specifically, the image automatic calibration module uses a tilt angle sensor to measure the pitch angle between the camera lens axis and the normal of the surface of the measured structure, first corrects the object distance, and then uses the horizontal center line of the image taken by the camera under the pitch angle condition as the boundary, and uses the perspective transformation matrix to correct the distortion of the upper and lower areas of the image respectively.

[0048] When there is only a pitch angle between the camera lens axis and the surface normal of the structure being measured, such as Figure 3 As shown, Figure 3The leftmost image in the figure represents the imaging when the pitch angle is 0, the middle image represents the imaging when the elevation angle is γ, and the rightmost image represents the two-dimensional plane image when the pitch angle is γ. Figure 3 In the figure, O is the optical center of the camera, R and Q are the upper and lower boundaries of the image when the phase angle is θ, and P is the center point of the image. Therefore, the image at an elevation angle of γ needs to be calibrated in the following two aspects:

[0049] (1) As the object distance moves from OP to OP′, the conversion factor between pixels and physical information changes, so the object distance needs to be corrected. The adjusted object distance S0 ′ for:

[0050]

[0051] Among them, S0 represents the original object distance;

[0052] (2) When the focal length remains unchanged, the camera viewing angle θ also remains unchanged. When the pitch angle is 0, the upper and lower boundaries R and Q of the camera imaging are symmetrical along the imaging center point P. However, when there is an elevation angle γ, the upper and lower boundaries R′ and Q′ of the imaging are no longer symmetrical along the new imaging center point P′. Therefore, the image will be distorted to varying degrees in the upper and lower parts of the horizontal center line. In order to improve the accuracy of image distortion calibration, this embodiment uses a perspective transformation method to correct the image distortion. Among them, the perspective transformation matrix H is a 3×3 matrix used to map the coordinates (x, y) in the original image to the corrected coordinates (x′, y′):

[0053]

[0054] The perspective transformation matrix can be determined by four pairs of point coordinates, and its matrix form is as follows:

[0055]

[0056] In the formula, the perspective transformation matrix has 8 degrees of freedom (one of the 9 elements in the matrix can be set arbitrarily, such as setting h 33 =1), so the coordinates of four key points are needed to solve these degrees of freedom.

[0057] like Figure 3 As shown in the middle and right figures in the figure, the camera elevation angle measured by the high-precision tilt angle sensor is γ, the camera viewing angle is θ, the upper and lower boundaries of the camera imaging on the surface of the measured structure are R′ and Q′, and the corrected interface is R″P′Q″, which is perpendicular to the camera axis OP′, so the corrected upper and lower boundaries are R″ and Q″. According to the geometric relationship of the triangle, it can be obtained:

[0058] ∠POP ′ =∠R″P ′R ′ =γ

[0059]

[0060] According to the triangle sine theorem, we can get:

[0061]

[0062] Similarly, the ratio of Q′P′ to Q″P′ is:

[0063]

[0064] exist Figure 3 In the right figure shown, the camera imaging area is U′R′Q′W′, the width of the image is w, and the height is h. First, the upper half of the image U′R′P′V′ is corrected by perspective transformation. Before correction, four boundary points in the upper half of the image are selected, and their pixel coordinates are U′(0,0), R′(w,0), P′(w,h / 2), and V′(0,h / 2) respectively. After correction, the coordinates of these points become: (w,h / 2), (0,h / 2). By changing the coordinates of the four boundary points and using the perspective transformation principle, we can quickly obtain the perspective transformation matrix H 上 , and applies the perspective transformation matrix to the upper half of the image to automatically correct the image deformation caused by distortion.

[0065] Next, the lower half V′P′Q′W′ of the camera imaging area U′R′Q′W′ is corrected by perspective transformation. Before correction, the image pixel coordinates of the four boundary points are V′(0,h / 2), P′(w,h / 2), Q′(w,h), and W′(0,h). After correction, the coordinates of these points become: (0,h / 2), (w,h / 2), According to the coordinate changes of these points, quickly obtain the perspective transformation matrix H 下 , and applies the perspective transformation matrix to the lower half of the image to automatically correct the image deformation caused by distortion.

[0066] Finally, the corrected upper and lower parts are merged into a complete image, ensuring a smooth transition between the two parts to ensure the overall coherence and accuracy of the image, thereby obtaining a corrected image to ensure that the image accurately restores the true form of the measured structure.

[0067] On the other hand, the present embodiment further provides a method for collecting visual detection information based on a pitch angle calibration image, which is implemented by using the above-mentioned visual detection information collection device based on a pitch angle calibration image, and comprises:

[0068] The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor, and the industrial camera is used to capture an image of the surface area of ​​the measured structure; the tilt angle between the industrial camera and the horizontal plane is detected by the tilt angle sensor to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure;

[0069] The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, so as to obtain a corrected image.

[0070] In addition, it is worth mentioning that if there is a horizontal angle between the axis of the camera lens and the surface normal of the structure being measured, a principle similar to the "pitch angle correction" can be used to perform image correction based on the horizontal angle (if there are both pitch angles and horizontal angles between the axis of the camera lens and the surface normal of the structure being measured, the image can be calibrated first according to the pitch angle calibration method to eliminate the image distortion caused by the pitch angle. On this basis, the horizontal angle calibration method can be used to further correct the distortion caused by the horizontal angle).

[0071] When there is a horizontal angle between the camera lens axis and the surface normal of the structure being measured, such as Figure 4 As shown, O represents the optical center of the camera, OA and OC are the axes of the laser ranging sensors arranged symmetrically on both sides, and the measured distances from the optical center to the surface of the measured structure are S1 and S3 respectively. OB is the axis of the middle laser ranging sensor, and it coincides with the optical axis of the camera, and the measured object distance is S2. Since the angle α between the laser ranging sensors on the left and right sides and the middle laser ranging sensor is known, and the three laser ranging sensors are all located on the same horizontal plane, it is possible to determine whether a horizontal angle is formed between the axis of the camera lens and the normal of the surface of the measured structure based on the measured sizes of S1 and S3. When the measured S1 and S3 are equal, it is considered that the camera lens is parallel to the surface of the measured structure, and image calibration is not required; when the distances S1 and S3 are not equal, it means that there is a horizontal angle between the axis of the camera lens and the normal of the surface of the measured structure. At the same time, the direction of the angle can be determined based on the sizes of S1 and S3, and the corresponding image calibration is performed accordingly.

[0072] like Figure 4As shown in the figure, when there is a horizontal angle, the left and right boundaries A and C of the image are no longer symmetrical along the new imaging center point B. Therefore, the image will be distorted to varying degrees in the left and right areas of the vertical center line. Similarly, the perspective transformation method is used to calibrate the left and right areas of the image respectively. Before calibration, it is necessary to obtain the horizontal angle formed between the axis of the camera lens and the surface normal of the structure being measured. In order to improve the calibration accuracy, let ∠ABD be β1 and ∠CBE be β2 respectively. The calculation steps of the two angles are as follows:

[0073] First, if Figure 4 As shown in the left figure, it is known that ∠AOB=α, ∠COB=α, |OA|=S1, |OB|=S2, then according to the geometric relationship of the triangle:

[0074]

[0075] |BD|=S2 tanα

[0076]

[0077] In addition, by |AB| 2 +|BD| 2 -2×|AB|×|BD|×cosβ1=|AD| 2 The geometric relationship gives the angle between AB and DE:

[0078]

[0079] Similarly, the angle β2 between BC and DE is calculated as follows:

[0080]

[0081] When the horizontal angle is determined, the left and right sides of the image are calibrated along the vertical center line, and the corrected left and right sides are merged into a complete image. The basic steps are as follows:

[0082] First, the left half of the image U′B′BW′ is corrected by perspective transformation. Before correction, four boundary points of the left half of the image are selected, and their pixel coordinates are U′(0,0), B′(w / 2,0), B(w / 2,h), and W′(0,h). After correction, the coordinates of these points become: (w / 2,0), (w / 2,h), By changing the coordinates of the four boundary points and using the perspective transformation principle, the perspective transformation matrix H is quickly obtained. 左 , and applies the perspective transformation matrix to the left half of the image to automatically correct the image deformation caused by distortion.

[0083] Next, the right half B′R′Q′B of the camera imaging area U′R′Q′W′ is corrected by perspective transformation. Before correction, the image pixel coordinates of the four boundary points are B′(w / 2,0), R′(w,0), Q′(w,h), and B(w / 2,h). After correction, the coordinates of these points become: (w / 2,0), (w / 2,h). According to the coordinate changes of these points, quickly obtain the perspective transformation matrix H 右 , and applies the perspective transformation matrix to the right half of the image to automatically correct the image deformation caused by distortion.

[0084] Finally, the corrected left half and right half are merged into a complete image, ensuring a smooth transition between the two parts to ensure the overall coherence and accuracy of the image, thereby obtaining a corrected image to ensure that the image accurately restores the true form of the structure being measured. After obtaining the corrected image, the existing image detection algorithm can be used to achieve accurate perception of the surface of the structure to be measured, such as high-precision perception and identification of cracks, deformations and other defects on the surface of structures such as bridges, tunnels, and buildings.

[0085] In summary, this embodiment provides a visual detection information acquisition device based on pitch angle calibration images and a visual detection information acquisition method based on pitch angle calibration images implemented by the visual detection information acquisition device based on pitch angle calibration images. The technical solution of this embodiment can correct the image distortion caused by the angle between the camera lens axis and the surface normal of the measured structure, thereby significantly improving the measurement accuracy. This solution is widely applicable to the detection and monitoring of apparent defects, damage and deformation of engineering structures based on computer vision technology, and has the significant advantages of automation, high precision and multi-functional perception, and has broad application prospects.

[0086] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0087] In addition, the term "and / or" is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects associated with each other are in an "or" relationship, but it may also represent an "and / or" relationship, which can be understood by referring to the context. "At least one" means one or more, and "multiple" means two or more. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0088] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0089] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0090] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0091] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0092] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0093] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A visual detection information acquisition device based on pitch angle calibration image, characterized in that: include: An industrial camera with a zoom lens, a pan-tilt device, a fixing device, an inclination angle sensor, a signal acquisition synchronizer and an image automatic calibration module; wherein the industrial camera is mounted on the pan-tilt device, and the pan-tilt device is mounted on the fixing device; the inclination angle sensor is mounted on the top of the industrial camera; the industrial camera and the inclination angle sensor are electrically connected to the signal acquisition synchronizer respectively; The industrial camera is used to capture images of the surface area of ​​the structure being measured; The tilt angle sensor is used to detect the tilt angle between the industrial camera and the horizontal plane, so as to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure; The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor to ensure the consistency of data acquisition between the industrial camera and the tilt angle sensor; The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, so as to obtain a corrected image.

2. The visual detection information acquisition device based on the pitch angle calibration image according to claim 1, characterized in that: The visual detection information collection device also includes an adaptive fill light; The adaptive fill light is electrically connected to the signal acquisition synchronizer, and the signal acquisition synchronizer synchronously controls the start and stop of the adaptive fill light, the industrial camera and the tilt angle sensor; The adaptive fill light can automatically adjust the brightness according to the intensity of the ambient light to ensure the clarity of the image captured by the industrial camera when capturing the image.

3. The visual detection information acquisition device based on the pitch angle calibration image according to claim 2, characterized in that: The visual detection information acquisition device also includes a multimodal data fusion module; The multimodal data fusion module adopts a preset data fusion algorithm to synchronously process the image captured by the industrial camera and the angle information collected by the tilt angle sensor to achieve perception of the surface of the measured structure.

4. The visual detection information acquisition device based on the pitch angle calibration image according to claim 1, characterized in that: The image automatic calibration module is specifically used to: use the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor to first correct the object distance, and then use the horizontal center line of the image taken by the camera under the pitch angle condition as the boundary, and use the perspective transformation matrix to correct the distortion of the upper and lower areas of the image respectively; merge the corrected upper and lower areas of the image into a complete image to obtain a corrected image.

5. The visual detection information acquisition device based on the pitch angle calibration image according to claim 4, characterized in that: Using the pitch angle between the axis of the camera lens and the surface normal of the structure under test collected by the tilt angle sensor, the object distance is first corrected, and then the horizontal center line of the image taken by the camera under the pitch angle condition is used as the boundary, and the perspective transformation matrix is ​​used to correct the distortion of the upper and lower areas of the image respectively; Merge the upper and lower regions of the rectified image into a complete image to obtain a rectified image, including: The object distance is corrected by using the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor; wherein the corrected object distance S0 ′ for: Where S0 represents the original object distance; γ represents the pitch angle between the axis of the camera lens and the surface normal of the measured structure; The upper half of the image is corrected by perspective transformation based on the horizontal center line of the image taken by the camera at a pitch angle. Before correction, four boundary points in the upper half of the image are selected, and their pixel coordinates are: (0,0), (w,0), (w,h / 2), (0,h / 2). After correction, the coordinates of the four boundary points become: (w,h / 2), (0,h / 2); where θ represents the camera angle of view; w represents the width of the image; h represents the height of the image; by changing the coordinates of the four boundary points in the upper half of the image and using the perspective transformation principle, the perspective transformation matrix H of the upper half of the image is obtained. 上 , and H 上 Applied to the upper area of ​​the image, it automatically corrects image deformation caused by distortion; The horizontal center line of the image taken by the camera at a pitch angle is used as the boundary to perform perspective transformation correction on the lower half of the image. Before correction, four boundary points in the lower half of the image are selected, and their pixel coordinates are: (0,h / 2), (w,h / 2), (w,h), (0,h); after correction, the coordinates of the four boundary points become: (0,h / 2), (w,h / 2), According to the coordinate changes of the four boundary points in the lower half of the image, the perspective transformation matrix H of the lower half of the image is obtained. 下 , and H 下 Applied to the lower half of the image, it automatically corrects image deformation caused by distortion; The upper half of the corrected image and the lower half of the image are merged into a complete image, ensuring a smooth transition between the two parts to obtain the corrected image.

6. A method for collecting visual inspection information based on a pitch angle calibration image, implemented by using the visual inspection information collection device based on a pitch angle calibration image as claimed in any one of claims 1 to 5, characterized in that: The visual detection information acquisition method based on the pitch angle calibration image includes: The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor, and the industrial camera is used to capture an image of the surface area of ​​the measured structure; the tilt angle between the industrial camera and the horizontal plane is detected by the tilt angle sensor to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure; The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, so as to obtain a corrected image.

Images