A visual detection information acquisition device and method based on pitch angle calibration image

Abstract

The present invention discloses a visual inspection information acquisition device and method based on pitch angle calibration images, belonging to the field of machine vision inspection technology. The device includes: an industrial camera with a zoom lens, a pan-tilt device, a fixing device, a tilt angle sensor, a signal acquisition synchronizer, and an image automatic calibration module; wherein the industrial camera is used to capture images of the surface area of the structure under test; the tilt angle sensor is used to determine the pitch angle formed between the axis of the camera lens and the surface normal of the structure under test; the signal acquisition synchronizer is used to ensure data acquisition consistency between the industrial camera and the tilt angle sensor; the image automatic calibration module is used to correct the distortion of the image captured by the camera due to the pitch angle to obtain a corrected image. The technical solution of the present invention can ensure the accuracy of image data, thereby improving the efficiency and accuracy of engineering structure health detection and monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and in particular to a device and method for collecting visual detection information based on pitch angle calibration images. Background Art

[0002] With the advancement of industrial automation and intelligentization, machine vision technology is increasingly being used in various structural inspection and monitoring fields. However, the application of traditional single-sensor devices in complex environments is significantly limited. This is especially true when inspecting large engineering structures (such as bridges, tunnels, and buildings), where high-precision sensing and identification of surface cracks, deformations, and other defects is often required. However, traditional visual sensor devices are limited by their single information collection method, making it difficult to provide comprehensive and accurate inspection results under changing environmental conditions.

[0003] Furthermore, existing visual inspection equipment typically relies on two-dimensional image data captured by industrial cameras, which cannot effectively capture depth information. This results in a significant decrease in inspection accuracy when the surface complexity is high or lighting conditions are poor. Furthermore, during image acquisition, image distortion caused by the angle between the camera lens axis and the surface of the structure being inspected is particularly noticeable, often resulting in geometric distortion in the inspection results, affecting inspection reliability. Furthermore, traditional equipment struggles to maintain image clarity in low-light environments, further limiting its application. Summary of the Invention

[0004] The present invention provides a device and method for collecting visual inspection information based on pitch angle calibration images, so as to solve the technical problem that the image data collected by the prior art is not accurate enough, which affects the accuracy of engineering structure inspection.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides a visual inspection information acquisition device based on a pitch angle calibration image, comprising: an industrial camera with a zoom lens, a pan-tilt device, a fixing device, a tilt angle sensor, a signal acquisition synchronizer, and an image automatic calibration module; wherein the industrial camera is mounted on the pan-tilt device, which is mounted on the fixing device; the tilt angle sensor is mounted on the top of the industrial camera; the industrial camera and the tilt angle sensor are respectively electrically connected to the signal acquisition synchronizer;

[0007] The industrial camera is used to capture images of the surface area of the structure being measured;

[0008] The tilt angle sensor is used to detect the tilt angle between the industrial camera and the horizontal plane, so as to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure;

[0009] The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor to ensure the consistency of data acquisition between the industrial camera and the tilt angle sensor;

[0010] The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, so as to obtain a corrected image.

[0011] Furthermore, the visual detection information collection device also includes an adaptive fill light;

[0012] The adaptive fill light is electrically connected to the signal acquisition synchronizer, and the signal acquisition synchronizer synchronously controls the start and stop of the adaptive fill light, the industrial camera, and the tilt angle sensor;

[0013] The adaptive fill light can automatically adjust the brightness according to the intensity of the ambient light to ensure the clarity of the image captured by the industrial camera.

[0014] Furthermore, the visual detection information acquisition device also includes a multimodal data fusion module;

[0015] The multimodal data fusion module uses a preset data fusion algorithm to synchronously process the images captured by the industrial camera and the angle information collected by the tilt angle sensor to achieve perception of the surface of the measured structure.

[0016] Furthermore, the image automatic calibration module is specifically used to: use the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor to first correct the object distance, and then use the horizontal center line of the image taken by the camera under the pitch angle condition as the boundary, and use the perspective transformation matrix to correct the distortion of the upper and lower areas of the image respectively; merge the corrected upper and lower areas of the image into a complete image to obtain a corrected image.

[0017] Furthermore, the object distance is first corrected using the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor. Then, the horizontal center line of the image captured by the camera under the pitch angle condition is used as the boundary, and the perspective transformation matrix is used to correct the distortion of the upper and lower areas of the image respectively; the upper and lower areas of the corrected image are merged into a complete image to obtain a corrected image, including:

[0018] The object distance is corrected using the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor; wherein the corrected object distance S0 ′ for:

[0019]

[0020] Where S0 represents the original object distance; γ represents the pitch angle between the camera lens axis and the surface normal of the measured structure;

[0021] The upper half of the image is corrected by perspective transformation using the horizontal center line of the image captured by the camera at a pitch angle. Before correction, four boundary points in the upper half of the image are selected, and their pixel coordinates are: (0, 0), (w, 0), (w, h / 2), (0, h / 2). After correction, the coordinates of the four boundary points become: (w,h / 2), (0,h / 2); where θ represents the camera angle of view; w represents the width of the image; h represents the height of the image; by changing the coordinates of the four boundary points in the upper half of the image and using the perspective transformation principle, we can obtain the perspective transformation matrix H of the upper half of the image. 上 , and H 上 Applied to the upper half of the image, it automatically corrects image deformation caused by distortion;

[0022] The perspective transformation correction is performed on the lower half of the image with the horizontal center line of the image taken by the camera at a pitch angle as the boundary. Before correction, the four boundary points in the lower half of the image are selected, and their pixel coordinates are: (0,h / 2), (w,h / 2), (w,h), (0,h); after correction, the coordinates of the four boundary points become: (0,h / 2), (w,h / 2), According to the coordinate changes of the four boundary points in the lower half of the image, the perspective transformation matrix H of the lower half of the image is obtained. 下 , and H 下 Applied to the lower half of the image, it automatically corrects image deformation caused by distortion;

[0023] The upper half of the corrected image and the lower half of the image are merged into a complete image, ensuring a smooth transition between the two parts to obtain the corrected image.

[0024] On the other hand, the present invention also provides a method for collecting visual detection information based on a pitch angle calibration image, which is implemented by using the above-mentioned visual detection information collection device based on a pitch angle calibration image, and the method comprises:

[0025] The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor, and the industrial camera is used to capture an image of the surface area of the measured structure; the tilt angle between the industrial camera and the horizontal plane is detected by the tilt angle sensor to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure;

[0026] The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, thereby obtaining a corrected image.

[0027] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0028] This solution integrates multiple sensors, including an industrial camera, a laser rangefinder, a high-precision tilt sensor, and adaptive fill light, to develop a low-cost, high-precision visual inspection information acquisition device. This device achieves precise acquisition and fusion processing of multi-dimensional data, significantly improving surface perception accuracy. A built-in automatic image calibration module automatically corrects image distortion caused by pitch deviation, ensuring image data accuracy and significantly improving measurement precision. The adaptive fill light function intelligently adjusts brightness based on ambient light, ensuring clear images even in low-light environments. Furthermore, the multifunctional pan / tilt head features multi-angle rotation, flexibly adapting to the comprehensive inspection needs of complex structures. A signal acquisition synchronizer ensures synchronized data acquisition between sensors, further improving system coordination and data consistency. The entire device not only offers low cost and high precision, but also boasts excellent performance, making it particularly suitable for long-term inspection and monitoring of large-scale engineering structures. This device can be widely used in buildings, bridges, tunnels, and other fields, significantly improving the efficiency and reliability of engineering inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 2. It is a schematic diagram of the two-dimensional structure of a visual detection information acquisition device based on a pitch angle calibration image provided by an embodiment of the present invention;

[0031] Figure 2 3D schematic diagram of the structure of a visual detection information acquisition device based on pitch angle calibration images provided by an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of the geometric relationship of image calibration when there is a pitch angle between the axis of the camera lens and the surface normal of the structure under test, provided by an embodiment of the present invention;

[0033] Figure 4This is a schematic diagram of the geometric relationship of image calibration when there is a horizontal angle between the camera lens axis and the surface normal of the measured structure provided by an embodiment of the present invention.

[0034] Description of reference numerals:

[0035] 1. Industrial camera; 2. Zoom lens; 3. PTZ device; 4. Fixing device;

[0036] 5. Laser distance sensor; 6. Angle adjustment button; 7. Tilt angle sensor;

[0037] 8. Adaptive fill light; 9. Surface of the structure being measured. DETAILED DESCRIPTION

[0038] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0039] First, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present concepts in a concrete manner. In addition, in the embodiments of the present invention, the meaning of "and / or" can be both or either of the two.

[0040] This embodiment provides a visual detection information acquisition device based on pitch angle calibration images, such as Figure 1 and Figure 2 As shown, the device includes: an industrial camera 1, a zoom lens 2, a pan-tilt device 3, a fixing device 4 (a tripod in this embodiment), a variable-angle laser ranging sensor component, a tilt angle sensor 7, a signal acquisition synchronizer, an adaptive fill light 8, a multimodal data fusion module, and an image automatic calibration module.

[0041] Industrial camera 1 is used to capture images of the surface area of the structure being measured. A zoom lens 2 is mounted on its front end, automatically adjusting its focal length based on the distance to the surface 9 of the structure being measured to ensure image clarity at varying object distances. The bottom of industrial camera 1 is fixed to a pan / tilt device 3, which features 180-degree rotation, allowing for multi-angle rotation and fixation of industrial camera 1. This enables multi-angle image acquisition and supports installation on a fixture 4 to accommodate diverse scenarios.

[0042] A variable-angle laser ranging sensor assembly is installed in the middle part of the industrial camera 1, which is used to measure the distance information between the optical center of the camera and different positions of the target surface area in real time, thereby improving the depth perception accuracy and determining the horizontal angle formed between the axis of the camera lens and the surface normal of the measured structure; the variable-angle laser ranging sensor assembly includes three laser ranging sensors 5 and an angle adjustment button 6, one of which has its axis coincident with the optical axis of the industrial camera 1, and the other two sensors are symmetrically arranged on both sides of the optical axis of the industrial camera 1, and the angles can be adjusted by the angle adjustment button 6, thereby providing more comprehensive distance measurement information and being able to quickly determine the horizontal angle between the axis of the camera lens and the surface normal of the measured structure.

[0043] A tilt angle sensor 7 is installed at the top rear end of the industrial camera 1 to measure the tilt angle between the industrial camera 1 and the horizontal plane with high precision, ensure the measurement accuracy of the equipment at different angles, and quickly determine the pitch angle between the camera lens axis and the surface normal of the structure being measured.

[0044] An adaptive fill light 8 is installed at the top front end of the industrial camera 1, which can intelligently adjust the brightness according to the ambient light conditions to ensure high-quality images in low-light environments and ensure the clarity of image acquisition.

[0045] After the hardware equipment is installed, a signal acquisition synchronizer is used to connect and synchronously control the start and stop of the industrial camera 1, laser ranging sensor 5, tilt angle sensor 7 and adaptive fill light 8 to ensure the synchronous data acquisition of devices such as the industrial camera 1, laser ranging sensor 5 and tilt angle sensor 7, thereby achieving the integration and data consistency of the entire system.

[0046] The multimodal data fusion module adopts a data fusion algorithm to fuse the image data acquired by the industrial camera 1, the distance information of the laser ranging sensor 5, and the angle information of the tilt angle sensor 7 in real time and synchronously process them to achieve accurate perception of the target area.

[0047] An automatic image calibration module is embedded within the aforementioned apparatus to correct distortion in the image captured by industrial camera 1 caused by the pitch angle between the lens axis of industrial camera 1 and the normal of the surface 9 of the structure being measured, thereby obtaining a corrected image. Specifically, the automatic image calibration module uses a tilt angle sensor to measure the pitch angle between the camera lens axis and the normal of the surface of the structure being measured. This module first corrects the object distance and then uses a perspective transformation matrix to correct distortion in the upper and lower regions of the image, using the horizontal centerline of the image captured by the camera at the pitch angle as the boundary.

[0048] When there is only a pitch angle between the camera lens axis and the surface normal of the structure being measured, such as Figure 3 As shown, Figure 3The leftmost image represents the imaging when the pitch angle is 0, the middle image represents the imaging when the elevation angle is γ, and the rightmost image represents the two-dimensional plane image when the pitch angle is γ. Figure 3 In the equation, O is the optical center of the camera, R and Q are the upper and lower boundaries of the image when the phase angle is θ, and P is the center point of the image. Therefore, the image at an elevation angle of γ requires the following two aspects of calibration:

[0049] (1) Since the object distance moves from OP to OP′, the conversion coefficient between pixels and physical information changes, so the object distance needs to be corrected. The adjusted object distance S0 ′ for:

[0050]

[0051] Among them, S0 represents the original object distance;

[0052] (2) When the focal length remains unchanged, the camera angle of view θ also remains unchanged. When the pitch angle is 0, the upper and lower boundaries R and Q of the camera imaging are symmetrical along the imaging center point P. However, when there is an elevation angle γ, the upper and lower boundaries R′ and Q′ of the imaging are no longer symmetrical along the new imaging center point P′. Therefore, the image will be distorted to varying degrees in the upper and lower parts of the horizontal center line. In order to improve the accuracy of image distortion calibration, this embodiment uses a perspective transformation method to correct the image distortion. Among them, the perspective transformation matrix H is a 3×3 matrix used to map the coordinates (x, y) in the original image to the corrected coordinates (x′, y′):

[0053]

[0054] The perspective transformation matrix can be determined by four pairs of point coordinates, and its matrix form is as follows:

[0055]

[0056] In the formula, the perspective transformation matrix has 8 degrees of freedom (one of the 9 elements in the matrix can be set arbitrarily, such as setting h 33 =1), so the coordinates of four key points are needed to solve these degrees of freedom.

[0057] like Figure 3 As shown in the middle and right figures, the camera elevation angle measured by the high-precision tilt sensor is γ, the camera viewing angle is θ, the upper and lower boundaries of the camera imaging on the surface of the measured structure are R′ and Q′, and the corrected interface is R″P′Q″. This interface is perpendicular to the camera axis OP′, so the corrected upper and lower boundaries are R″ and Q″. According to the triangle geometric relationship, we can get:

[0058] ∠POP ′ =∠R″P ′R ′ =γ

[0059]

[0060] According to the triangle sine theorem, we can get:

[0061]

[0062] Similarly, the ratio of Q′P′ to Q″P′ is:

[0063]

[0064] exist Figure 3 In the right figure shown, the camera imaging area is U′R′Q′W′, the image width is w, and the height is h. First, the upper half of the image U′R′P′V′ is corrected by perspective transformation. Before correction, four boundary points in the upper half of the image are selected, and their pixel coordinates are U′(0,0), R′(w,0), P′(w,h / 2), and V′(0,h / 2). After correction, the coordinates of these points become: (w,h / 2), (0,h / 2). By changing the coordinates of the four boundary points and using the perspective transformation principle, we can quickly obtain the perspective transformation matrix H. 上 , and applies the perspective transformation matrix to the upper half of the image to automatically correct the image deformation caused by distortion.

[0065] Next, the perspective transformation correction is performed on the lower half V′P′Q′W′ of the camera imaging area U′R′Q′W′. Before correction, the pixel coordinates of the four boundary points are V′(0,h / 2), P′(w,h / 2), Q′(w,h), and W′(0,h). After correction, the coordinates of these points become: (0,h / 2), (w,h / 2), According to the coordinate changes of these points, quickly obtain the perspective transformation matrix H 下 , and applies the perspective transformation matrix to the lower half of the image to automatically correct the image deformation caused by distortion.

[0066] Finally, the corrected upper and lower parts are merged into a complete image, ensuring a smooth transition between the two parts to ensure the overall coherence and accuracy of the image, thereby obtaining a corrected image to ensure that the image accurately restores the true form of the measured structure.

[0067] On the other hand, this embodiment further provides a method for collecting visual detection information based on a pitch angle calibration image, which is implemented by using the above-mentioned visual detection information collection device based on a pitch angle calibration image, and the method includes:

[0068] The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor, and the industrial camera is used to capture an image of the surface area of the measured structure; the tilt angle between the industrial camera and the horizontal plane is detected by the tilt angle sensor to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure;

[0069] The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, thereby obtaining a corrected image.

[0070] In addition, it is worth mentioning that if there is a horizontal angle between the axis of the camera lens and the surface normal of the structure being measured, a principle similar to the "pitch angle correction" can be used to perform image correction based on the horizontal angle (if there are both pitch angles and horizontal angles between the axis of the camera lens and the surface normal of the structure being measured, the image can be calibrated first according to the pitch angle calibration method to eliminate the image distortion caused by the pitch angle. On this basis, the horizontal angle calibration method can be used to further correct the distortion caused by the horizontal angle).

[0071] When there is a horizontal angle between the camera lens axis and the surface normal of the structure being measured, such as Figure 4 As shown in the figure, O represents the optical center of the camera, OA and OC represent the axes of the symmetrically arranged laser ranging sensors on either side. The measured distances from the optical center to the surface of the structure being measured are S1 and S3, respectively. OB is the axis of the center laser ranging sensor, which coincides with the camera's optical axis. The measured object distance is S2. Since the angle α between the left and right laser ranging sensors and the center laser ranging sensor is known, and all three laser ranging sensors are located on the same horizontal plane, the measured values of S1 and S3 can be used to determine whether a horizontal angle is formed between the axis of the camera lens and the surface normal of the structure being measured. When the measured values of S1 and S3 are equal, the camera lens is assumed to be parallel to the surface of the structure being measured, and image calibration is not required. When the measured values of S1 and S3 are unequal, it indicates that a horizontal angle exists between the axis of the camera lens and the surface normal of the structure being measured. Furthermore, the values of S1 and S3 can be used to determine the direction of the angle, allowing for appropriate image calibration.

[0072] like Figure 4As shown in the figure, when there is a horizontal angle, the left and right boundaries A and C of the image are no longer symmetrical about the new imaging center point B. Therefore, the image will be distorted to varying degrees in the left and right areas of the vertical center line. Similarly, the perspective transformation method is used to calibrate the left and right areas of the image respectively. Before calibration, it is necessary to obtain the horizontal angle formed between the axis of the camera lens and the surface normal of the structure being measured. In order to improve the calibration accuracy, let ∠ABD be β1 and ∠CBE be β2. The calculation steps of the two angles are as follows:

[0073] First, if Figure 4 As shown in the left figure, it is known that ∠AOB=α, ∠COB=α, |OA|=S1, |OB|=S2, then according to the geometric relationship of the triangle:

[0074]

[0075] |BD|=S2 tanα

[0076]

[0077] In addition, by |AB| 2 +|BD| 2 -2×|AB|×|BD|×cosβ1=|AD| 2 The geometric relationship gives the angle between AB and DE:

[0078]

[0079] Similarly, the angle β2 between BC and DE is calculated as follows:

[0080]

[0081] Once the horizontal angle is determined, calibrate the left and right sides of the image along the vertical center line separately, and merge the corrected left and right sides into a complete image. The basic steps are as follows:

[0082] First, perform perspective transformation correction on the left half of the image, U′B′BW′. Before correction, select four boundary points in the left half of the image, whose pixel coordinates are U′(0,0), B′(w / 2,0), B(w / 2,h), and W′(0,h). After correction, the coordinates of these points become: (w / 2,0), (w / 2,h), By changing the coordinates of the four boundary points and using the perspective transformation principle, we can quickly obtain the perspective transformation matrix H. 左 , and applies the perspective transformation matrix to the left half of the image to automatically correct the image deformation caused by distortion.

[0083] Next, the right half B′R′Q′B of the camera imaging area U′R′Q′W′ is corrected by perspective transformation. Before correction, the pixel coordinates of the four boundary points are B′(w / 2,0), R′(w,0), Q′(w,h), and B(w / 2,h). After correction, the coordinates of these points become: (w / 2,0), (w / 2,h). According to the coordinate changes of these points, quickly obtain the perspective transformation matrix H 右 , and applies the perspective transformation matrix to the right half of the image to automatically correct the image deformation caused by distortion.

[0084] Finally, the corrected left and right halves are merged into a complete image, ensuring a smooth transition between the two parts to maintain overall image coherence and accuracy. This results in a corrected image that accurately restores the true form of the structure being measured. Once this corrected image is obtained, existing image detection algorithms can be used to accurately perceive the surface of the structure being measured. For example, cracks, deformations, and other defects on the surfaces of structures like bridges, tunnels, and buildings can be detected and identified with high precision.

[0085] In summary, this embodiment provides a visual inspection information acquisition device based on pitch-angle calibration images and a visual inspection information acquisition method based on pitch-angle calibration images implemented using this device. The technical solution of this embodiment can correct image distortion caused by the angle between the camera lens axis and the surface normal of the structure being measured, thereby significantly improving measurement accuracy. This solution is widely applicable to the detection and monitoring of apparent defects, damage, and deformation of engineering structures based on computer vision technology. It has the significant advantages of automation, high precision, and multifunctional perception, and has broad application prospects.

[0086] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal device comprising the element.

[0087] In addition, the term "and / or" is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after the association are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" refers to one or more, and "multiple" refers to two or more. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0088] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0089] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0090] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of functional modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0091] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0092] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0093] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. It should be noted that, although preferred embodiments of the present invention have been described, those skilled in the art, once understanding the basic inventive concepts of the present invention, may make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all variations and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A visual detection information acquisition device based on pitch angle calibration images, characterized in that: include: An industrial camera with a zoom lens, a pan-tilt device, a fixing device, a tilt angle sensor, a signal acquisition synchronizer, and an image automatic calibration module; wherein the industrial camera is mounted on the pan-tilt device, and the pan-tilt device is mounted on the fixing device; the tilt angle sensor is mounted on the top of the industrial camera; the industrial camera and the tilt angle sensor are respectively electrically connected to the signal acquisition synchronizer; The industrial camera is used to capture images of the surface area of the structure being measured; The tilt angle sensor is used to detect the tilt angle between the industrial camera and the horizontal plane, so as to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure; The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor to ensure the consistency of data acquisition between the industrial camera and the tilt angle sensor; The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, thereby obtaining a corrected image; The image automatic calibration module is specifically used to: use the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor to first correct the object distance, and then use the horizontal center line of the image taken by the camera under the pitch angle condition as the boundary to use the perspective transformation matrix to correct the distortion of the upper and lower areas of the image respectively; merge the corrected upper and lower areas of the image into a complete image to obtain a corrected image.

2. The visual detection information acquisition device based on the pitch angle calibration image according to claim 1, characterized in that: The visual detection information collection device also includes an adaptive fill light; The adaptive fill light is electrically connected to the signal acquisition synchronizer, and the signal acquisition synchronizer synchronously controls the start and stop of the adaptive fill light, the industrial camera, and the tilt angle sensor; The adaptive fill light can automatically adjust the brightness according to the intensity of the ambient light to ensure the clarity of the image captured by the industrial camera.

3. The visual detection information acquisition device based on the pitch angle calibration image according to claim 2, characterized in that: The visual detection information acquisition device also includes a multimodal data fusion module; The multimodal data fusion module uses a preset data fusion algorithm to synchronously process the images captured by the industrial camera and the angle information collected by the tilt angle sensor to achieve perception of the surface of the measured structure.

4. The visual detection information acquisition device based on the pitch angle calibration image according to claim 1, characterized in that: Using the pitch angle between the camera lens axis and the surface normal of the structure being measured, as acquired by the tilt angle sensor, the object distance is first corrected. Then, using the horizontal center line of the image captured by the camera at the pitch angle as the boundary, the perspective transformation matrix is used to correct the distortion of the upper and lower areas of the image respectively. Merge the upper and lower regions of the rectified image into a complete image to obtain the rectified image, including: The object distance is corrected using the pitch angle between the camera lens axis and the surface normal of the measured structure collected by the tilt angle sensor; wherein the corrected object distance S0 ′ for: Where S0 represents the original object distance; γ represents the pitch angle between the camera lens axis and the surface normal of the measured structure; The upper half of the image is corrected by perspective transformation using the horizontal center line of the image captured by the camera at a pitch angle. Before correction, four boundary points in the upper half of the image are selected, and their pixel coordinates are: (0, 0), (w, 0), (w, h / 2), (0, h / 2). After correction, the coordinates of the four boundary points become: Where θ represents the camera angle of view; w represents the width of the image; h represents the height of the image; by changing the coordinates of the four boundary points in the upper half of the image, using the perspective transformation principle, we can obtain the perspective transformation matrix H of the upper half of the image. 上 , and H 上 Applied to the upper half of the image, it automatically corrects image deformation caused by distortion; The perspective transformation correction is performed on the lower half of the image, with the horizontal center line of the image taken by the camera at a pitch angle as the boundary. Before correction, four boundary points in the lower half of the image are selected, and their pixel coordinates are: (0, h / 2), (w, h / 2), (w, h), (0, h). After correction, the coordinates of the four boundary points become: According to the coordinate changes of the four boundary points in the lower half of the image, the perspective transformation matrix H of the lower half of the image is obtained. 下 , and H 下 Applied to the lower half of the image, it automatically corrects image deformation caused by distortion; The upper half of the corrected image and the lower half of the image are merged into a complete image, ensuring a smooth transition between the two parts to obtain the corrected image.

5. A method for collecting visual inspection information based on a pitch angle calibration image, implemented by the visual inspection information collection device based on a pitch angle calibration image according to any one of claims 1 to 4, characterized in that: The method for collecting visual detection information based on the pitch angle calibration image includes: The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the tilt angle sensor, and the industrial camera is used to capture an image of the surface area of the measured structure; the tilt angle between the industrial camera and the horizontal plane is detected by the tilt angle sensor to determine the pitch angle formed between the lens axis of the industrial camera and the surface normal of the measured structure; The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the pitch angle between the lens axis of the industrial camera and the surface normal of the measured structure, thereby obtaining a corrected image.

Images