Visual Detection Information Acquisition Device and Method for Calibrating Images Based on Horizontal Angle

Abstract

The present invention discloses a visual detection information acquisition device and method for calibrating an image based on a horizontal angle, belonging to the technical field of machine vision detection. The device includes: an industrial camera with a zoom lens, a pan-tilt device, a fixing device, a laser ranging sensor assembly, a signal acquisition synchronizer, and an image automatic calibration module. Among them, the industrial camera is used to capture an image of the surface area of the structure to be measured; the laser ranging sensor assembly is used to determine the horizontal angle formed between the axis of the camera lens and the normal of the surface of the structure to be measured; the signal acquisition synchronizer is used to ensure the consistency of data acquisition between the camera and the laser ranging sensor assembly; the image automatic calibration module is used to correct the distortion of the image captured by the camera caused by the horizontal angle to obtain a corrected image. By adopting the technical solution of the present invention, the accuracy of image data can be ensured, thereby improving the efficiency and accuracy of engineering structure health detection and monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and particularly to a visual detection information acquisition device and method based on calibrating images with a horizontal angle. Background Art

[0002] With the development of industrial automation and intelligence, machine vision technology has been more widely applied in various fields of structure detection and monitoring. The application of traditional single-sensor devices has been greatly limited in complex environments. Especially during the detection process of large-scale engineering structures (such as bridges, tunnels, buildings, etc.), it is often necessary to highly accurately perceive and identify cracks, deformations, and other defects on the surface. However, traditional visual sensor devices are limited by their single information acquisition method and are difficult to provide comprehensive and accurate detection results under changing environmental conditions.

[0003] In addition, existing visual detection devices usually rely on two-dimensional image data collected by industrial cameras and cannot effectively obtain depth information, resulting in a significant decrease in detection accuracy when the surface complexity is high or the lighting conditions are poor. At the same time, during the image acquisition process, the problem of image distortion caused by the angle between the camera lens axis and the surface of the structure to be measured is particularly obvious, and the detection results often produce geometric distortion, affecting the reliability of the detection. In addition, in an environment with insufficient light, traditional devices are difficult to ensure image clarity, further limiting their application scope. Summary of the Invention

[0004] The present invention provides a visual detection information acquisition device and method based on calibrating images with a horizontal angle to solve the technical problem that the image data collected by the prior art is not accurate enough and affects the detection accuracy of engineering structures.

[0005] To solve the above technical problems, the present invention provides the following technical solutions:

[0006] On the one hand, the present invention provides a visual detection information acquisition device based on calibrating images with a horizontal angle, which includes: an industrial camera with a zoom lens, a pan-tilt device, a fixing device, a laser range sensor assembly, a signal acquisition synchronizer, and an image automatic calibration module; wherein, the industrial camera is installed on the pan-tilt device, and the pan-tilt device is installed on the fixing device; the laser range sensor assembly is installed on the top of the industrial camera; the industrial camera and the laser range sensor assembly are respectively electrically connected to the signal acquisition synchronizer;

[0007] The industrial camera is used to capture images of the surface area of the structure to be measured;

[0008] The laser ranging sensor assembly is used to measure the distance information between the optical center of the industrial camera and different positions on the surface area of the structure to be measured in real time, and determine the horizontal angle formed between the lens axis of the industrial camera and the surface normal of the structure to be measured;

[0009] The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the laser ranging sensor assembly to ensure data acquisition consistency between the industrial camera and the laser ranging sensor assembly;

[0010] The image automatic calibration module is used to correct the distortion of the image captured by the industrial camera caused by the horizontal angle between the lens axis of the industrial camera and the surface normal of the structure to be measured, and obtain the corrected image.

[0011] Further, the visual detection information acquisition device further includes an adaptive fill light;

[0012] The adaptive fill light is electrically connected to the signal acquisition synchronizer, and the signal acquisition synchronizer synchronously controls the start and stop of the adaptive fill light, the industrial camera, and the laser ranging sensor assembly;

[0013] The adaptive fill light can automatically adjust its brightness according to the intensity of the ambient light to ensure the clarity of the image captured when the industrial camera performs image acquisition.

[0014] Further, the laser ranging sensor assembly includes: a first laser ranging sensor, a second laser ranging sensor, a third laser ranging sensor, a first angle adjustment button, and a second angle adjustment button;

[0015] Among them, the second laser ranging sensor coincides with the optical axis of the industrial camera, the first laser ranging sensor and the third laser ranging sensor are symmetrically arranged on both sides of the optical axis of the industrial camera, and a first angle adjustment button is provided on the first laser ranging sensor, and a second angle adjustment button is provided on the third laser ranging sensor. The first angle adjustment button is mechanically connected to the first laser ranging sensor and is used to adjust the angle of the first laser ranging sensor; the second angle adjustment button is mechanically connected to the third laser ranging sensor and is used to adjust the angle of the third laser ranging sensor.

[0016] Further, the visual detection information acquisition device further includes a multi-modal data fusion module; the multi-modal data fusion module uses a preset data fusion algorithm to synchronously process the image captured by the industrial camera and the distance information collected by the laser ranging sensor assembly to realize the perception of the surface of the structure to be measured.

[0017] Further, the image automatic calibration module is specifically configured to: measure the distances between the optical center of the industrial camera and different positions on the surface area of the structure to be measured by using the laser ranging sensor assembly, obtain the horizontal angle between the axis of the camera lens and the normal of the surface of the structure to be measured, and take the vertical center line of the image captured by the camera as the boundary, and use the perspective transformation matrix to correct the distortion of the left and right regions of the image respectively to obtain the corrected image.

[0018] Further, measuring the distances between the optical center of the industrial camera and different positions on the surface area of the structure to be measured by using the laser ranging sensor assembly, obtaining the horizontal angle between the axis of the camera lens and the normal of the surface of the structure to be measured, and taking the vertical center line of the image captured by the camera as the boundary, and using the perspective transformation matrix to correct the distortion of the left and right regions of the image respectively to obtain the corrected image, including:

[0019] Obtaining the horizontal angle between the axis of the camera lens and the normal of the surface of the structure to be measured, which is expressed by the formula:

[0020]

[0021] wherein, S1 represents the distance from the optical center to the surface of the structure to be measured measured by the first laser ranging sensor; S3 represents the distance from the optical center to the surface of the structure to be measured measured by the third laser ranging sensor; S2 represents the distance from the optical center to the surface of the structure to be measured measured by the second laser ranging sensor; the angles between the axis of the first laser ranging sensor and the axis of the second laser ranging sensor and between the axis of the third laser ranging sensor and the axis of the second laser ranging sensor are both α;

[0022] Taking the vertical center line of the image captured by the camera as the boundary, performing perspective transformation correction on the left half region of the image. Before correction, four boundary points of the left half region of the image are selected, and their pixel coordinates are successively: (0,0), (w / 2,0), (w / 2,h), (0,h); after correction, the coordinates of the four boundary points become: (w / 2,0), (w / 2,h), wherein, θ represents the camera view angle; w represents the width of the image; h represents the height of the image; through the coordinate changes of the four boundary points in the left half region of the image, using the principle of perspective transformation, the perspective transformation matrix H of the left half region of the image is obtained 左 , and applying H 左 to the left half region of the image to automatically correct the image deformation caused by distortion;

[0023] Taking the vertical center line of the image captured by the camera as the boundary, perform perspective transformation correction on the right half area of the image. Before correction, select four boundary points of the right half area of the image, and their pixel coordinates are successively: (w / 2, 0), (w, 0), (w, h), (w / 2, h); after correction, the coordinates of the four boundary points become: (w / 2, 0), (w / 2, h); according to the coordinate changes of the four boundary points of the right half area of the image, obtain the perspective transformation matrix H of the right half area of the image 右 , and apply H 右 to the right half area of the image to automatically correct the image deformation caused by distortion;

[0024] Finally, combine the left half area of the corrected image and the right half area of the image into a complete image, ensure a smooth transition of the seam between the two parts, and obtain the corrected image.

[0025] On the other hand, the present invention also provides a method for collecting visual detection information for calibrating an image based on a horizontal angle, which is implemented by using the above-described visual detection information collection device for calibrating an image based on a horizontal angle, and includes:

[0026] Use the signal acquisition synchronizer to synchronously control the start and stop of the industrial camera and the laser range sensor assembly, use the laser range sensor assembly to measure the distance information between the optical center of the industrial camera and different positions on the surface area of the measured structure in real time, and determine the horizontal angle formed between the lens axis of the industrial camera and the normal line of the surface of the measured structure;

[0027] Use the image automatic calibration module to correct the distortion generated by the horizontal angle between the lens axis of the industrial camera and the normal line of the surface of the measured structure in the image captured by the industrial camera, and obtain the corrected image.

[0028] The beneficial effects brought by the technical solution provided by the present invention at least include:

[0029] The solution of the present invention develops a low-cost and high-precision visual detection information acquisition device by integrating various sensors such as industrial cameras, laser range sensors, high-precision tilt angle sensors, and adaptive fill lights, realizing the accurate acquisition and fusion processing of multi-dimensional data, and significantly improving the perception accuracy of the structure surface. Through the built-in image automatic calibration module, the system can automatically correct the image distortion caused by the horizontal angle deviation, ensuring the accuracy of the image data, thereby significantly improving the measurement accuracy. The adaptive fill light function can intelligently adjust the brightness according to the ambient light, ensuring clear images in low-light environments. In addition, the multi-functional pan-tilt head has the ability to rotate at multiple angles, flexibly adapting to the all-round detection requirements of complex structures. The signal acquisition synchronizer ensures the synchronous data acquisition between sensors, further improving the coordination and data consistency of the system. The entire device not only has the advantages of low cost and high precision, but also has good high-performance performance, especially suitable for long-term detection and monitoring of large-scale engineering structures. The device can be widely applied in fields such as buildings, bridges, and tunnels, greatly improving the efficiency and reliability of engineering detection. Brief Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0031] Figure 1 It is a two-dimensional plane structure schematic diagram of the visual detection information acquisition device based on horizontal angle calibrated image provided by the embodiment of the present invention;

[0032] Figure 2 It is a three-dimensional solid structure schematic diagram of the visual detection information acquisition device based on horizontal angle calibrated image provided by the embodiment of the present invention;

[0033] Figure 3 It is a schematic diagram of the image calibration geometric relationship when there is a horizontal angle between the camera lens axis and the normal line of the measured structure surface provided by the embodiment of the present invention;

[0034] Figure 4 It is a schematic diagram of the image calibration geometric relationship when there is a pitch angle between the camera lens axis and the normal line of the measured structure surface provided by the embodiment of the present invention.

[0035] Description of the Reference Numerals:

[0036] 1. Industrial camera; 2. Zoom lens; 3. Pan-tilt head device; 4. Fixing device;

[0037] 5. Laser distance measurement sensor; 6. Angle adjustment button; 7. Tilt angle sensor;

[0038] 8. Adaptive fill light; 9. Surface of the structure to be measured. Specific implementation manner

[0039] To make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0040] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, the use of the word "exemplarily" is intended to present concepts in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two.

[0041] This embodiment provides a visual detection information acquisition device based on horizontal angle calibration images, as Figure 1 and Figure 2 shown. The device includes: an industrial camera 1, a zoom lens 2, a pan-tilt device 3, a fixing device 4 (a tripod in this embodiment), a variable-angle laser distance measurement sensor assembly, a tilt angle sensor 7, a signal acquisition synchronizer, an adaptive fill light 8, a multi-modal data fusion module, and an image automatic calibration module.

[0042] Among them, the industrial camera 1 is used to capture images of the surface area of the structure to be measured. A zoom lens 2 is installed at the front end thereof, so as to be able to automatically adjust the focal length according to the distance of the surface 9 of the structure to be measured, so as to ensure the clarity of the image at different object distances. The bottom of the industrial camera 1 is fixed on the pan-tilt device 3. The pan-tilt device 3 has a 180-degree rotation function, supports the multi-angle free rotation and fixation of the industrial camera 1, and thus can realize multi-angle image acquisition, and supports installation on the fixing device 4 to meet the requirements of different scenarios.

[0043] A variable-angle laser distance measurement sensor assembly is installed in the middle part of the industrial camera 1, which is used to measure the distance information between the optical center of the camera and different positions of the target surface area in real time, so as to improve the depth perception accuracy and determine the horizontal angle formed between the camera lens axis and the normal line of the surface of the structure to be measured; the variable-angle laser distance measurement sensor assembly includes three laser distance measurement sensors 5 and an angle adjustment button 6. The axis of one sensor coincides with the optical axis of the industrial camera 1, and the other two sensors are symmetrically arranged on both sides of the optical axis of the industrial camera 1, and the angles can be adjusted through the angle adjustment button 6, so as to provide more comprehensive distance measurement information and quickly determine the horizontal angle between the camera lens axis and the normal line of the surface of the structure to be measured.

[0044] An inclination angle sensor 7 is installed at the rear end of the top of the industrial camera 1, which is used to measure the inclination angle between the industrial camera 1 and the horizontal plane with high precision, ensure the measurement accuracy of the device at different angles, and quickly determine the pitch angle between the camera lens axis and the normal line of the surface of the measured structure.

[0045] An adaptive fill light 8 is installed at the front end of the top of the industrial camera 1, which can intelligently adjust the brightness according to the ambient light conditions, ensure high-quality images are obtained in low-light environments, and guarantee the clarity of image acquisition.

[0046] After the hardware devices are installed, a signal acquisition synchronizer is used to connect and synchronously control the startup and shutdown of the industrial camera 1, the laser range finder sensor 5, the inclination angle sensor 7, and the adaptive fill light 8, ensuring the synchronous acquisition of data from devices such as the industrial camera 1, the laser range finder sensor 5, and the inclination angle sensor 7, and realizing the integration and data consistency of the entire system.

[0047] The multi-modal data fusion module uses a data fusion algorithm to perform real-time fusion and synchronous processing on the image data obtained by the industrial camera 1, the distance information of the laser range finder sensor 5, and the angle information of the inclination angle sensor 7, achieving precise perception of the target area.

[0048] An image automatic calibration module is embedded on the basis of the above device, which is used to correct the distortion of the image captured by the industrial camera 1 caused by the horizontal angle between the camera lens axis and the normal line of the surface 9 of the measured structure, and obtain a corrected image. Specifically, the image automatic calibration module measures the distances between the optical center of the industrial camera and different positions on the surface area of the measured structure using the variable-angle laser range finder sensor assembly, obtains the horizontal angle between the camera lens axis and the normal line of the surface of the measured structure, and takes the vertical center line of the image captured by the camera as the boundary, and uses a perspective transformation matrix to correct the distortion of the left and right areas of the image respectively to obtain a corrected image.

[0049] When there is a horizontal angle between the camera lens axis and the normal line of the surface of the measured structure, such as Figure 3As shown in the figure, O represents the optical center of the camera. OA and OC are the axes of the laser rangefinders symmetrically arranged on both sides. The measured distances from the optical center to the surface of the measured structure are S1 and S3 respectively. OB is the axis of the middle laser rangefinder and coincides with the camera optical axis. The measured object distance is S2. Since the angles α between the laser rangefinders on the left and right sides and the middle laser rangefinder are known, and the three laser rangefinders are all on the same horizontal plane, it is possible to determine whether a horizontal angle is formed between the axis of the camera lens and the normal of the surface of the measured structure according to the magnitudes of the measured S1 and S3. When the measured S1 and S3 are equal, it is considered that the camera lens is parallel to the surface of the measured structure, and no image calibration is required. When the measured S1 and S3 are not equal, it indicates that there is a horizontal angle between the axis of the camera lens and the normal of the surface of the measured structure. At the same time, the direction of the angle can be determined according to the magnitudes of S1 and S3, and corresponding image calibration can be carried out accordingly.

[0050] As Figure 3 shown, when there is a horizontal angle, the left and right boundaries A and C of the image no longer symmetrically along the new imaging center point B. Therefore, different degrees of distortion will occur in the left and right regions of the vertical center line of the image. Similarly, the perspective transformation method is used to calibrate the left and right regions of the image respectively. Before calibration, it is necessary to obtain the horizontal angle formed between the axis of the camera lens and the normal of the surface of the measured structure. To improve the calibration accuracy, let ∠ABD be β1 and ∠CBE be β2 respectively. The calculation steps of the two angles are as follows:

[0051] First, as Figure 3 shown in the left figure of, given ∠AOB = α, ∠COB = α, |OA| = S1, |OB| = S2, then according to the triangle geometric relationship:

[0052]

[0053] |BD| = S2 tanα

[0054]

[0055] In addition, from |AB| 2 +|BD| 2 -2×|AB|×|BD|×cosβ1 = |AD| 2 The geometric relationship gives the angle between AB and DE:

[0056]

[0057] Similarly, the calculation formula for the angle β2 between BC and DE is as follows:

[0058]

[0059] After the horizontal angle is determined, the image is calibrated separately on the left and right sides of the vertical center line, and the corrected left and right sides are combined into a complete image. Among them, to improve the accuracy of image distortion calibration, this embodiment uses the perspective transformation method to correct the image distortion. Among them, the perspective transformation matrix H is a 3×3 matrix, which is used to map the coordinates (x, y) in the original image to the corrected coordinates (x′, y′):

[0060]

[0061] The perspective transformation matrix can be determined by the coordinates of four pairs of points, and its matrix form is as follows:

[0062]

[0063] In the formula, the perspective transformation matrix has 8 degrees of freedom (1 element in the 9 elements of the matrix can be set arbitrarily, for example, set h 33 = 1), so the coordinates of four key points are required to solve these degrees of freedom.

[0064] Based on the above, the specific correction steps are as follows:

[0065] First, perform perspective transformation correction on the left half U′B′BW′ of the image. Before correction, four boundary points in the left half area of the image are selected, and their pixel coordinates are U′(0,0), B′(w / 2,0), B(w / 2,h), W′(0,h) in sequence. After correction, the coordinates of these points become: (w / 2,0), (w / 2,h), Through the coordinate changes of the four boundary points, using the principle of perspective transformation, quickly obtain the perspective transformation matrix H 左 , and apply this perspective transformation matrix to the left half area of the image to automatically correct the image deformation caused by distortion.

[0066] Next, perform perspective transformation correction on the right half B′R′Q′B of the camera imaging area U′R′Q′W′. Before correction, the pixel coordinates of the four boundary points of the image are B′(w / 2,0), R′(w,0), Q′(w,h), B(w / 2,h) in sequence. After correction, the coordinates of these points become: (w / 2,0), (w / 2,h). According to the coordinate changes of these points, quickly obtain the perspective transformation matrix H 右 , and apply this perspective transformation matrix to the right half area of the image to automatically correct the image deformation caused by distortion.

[0067] Finally, merge the corrected left half and the right half into a complete image, ensuring a smooth transition at the seam between the two parts to guarantee the overall coherence and accuracy of the image, thereby obtaining the corrected image to ensure that the image accurately restores the true form of the measured structure.

[0068] On the other hand, this embodiment also provides a method for collecting visual detection information based on horizontal angle calibration of an image implemented by using the above-mentioned visual detection information collection device based on horizontal angle calibration of an image, which includes:

[0069] Use the signal acquisition synchronizer to synchronously control the startup and shutdown of the industrial camera and the variable-angle laser ranging sensor assembly, use the variable-angle laser ranging sensor assembly to measure the distance information between the optical center of the industrial camera and different positions on the surface area of the measured structure in real time, and determine the horizontal angle formed between the lens axis of the industrial camera and the surface normal of the measured structure;

[0070] Use the image automatic calibration module to correct the distortion of the image captured by the industrial camera caused by the horizontal angle between the lens axis of the industrial camera and the surface normal of the measured structure, and obtain the corrected image.

[0071] In addition, it is worth mentioning that if there is also a pitch angle between the camera lens axis and the surface normal of the measured structure, then an image correction can be performed based on the pitch angle using a principle similar to "horizontal angle correction" (if there are both a pitch angle and a horizontal angle between the camera lens axis and the surface normal of the measured structure, then the image can be calibrated first according to the pitch angle calibration method to eliminate the image distortion caused by the pitch angle. On this basis, the horizontal angle calibration method can be used to further correct the distortion caused by the horizontal angle).

[0072] When there is only a pitch angle between the camera lens axis and the surface normal of the measured structure, as Figure 4 shown, Figure 4 the leftmost image in represents the imaging when the pitch angle is 0, the middle image represents the imaging when the elevation angle is γ, and the rightmost image represents the two-dimensional plan view of the imaging when the elevation angle is γ. Figure 4 In, O is the optical center of the camera, R and Q are the upper and lower boundaries of the imaging when the phase angle is θ respectively, and P is the center point of the imaging. Therefore, the imaging image at the elevation angle of γ needs to be calibrated in the following two aspects:

[0073] (1) Since the object distance moves from OP to OP′, the conversion coefficient between pixels and physical information changes, so the object distance needs to be corrected, and the adjusted object distance S0 ′ is:

[0074]

[0075] Among them, S0 represents the original object distance;

[0076] (2) When the focal length remains unchanged, the camera viewing angle θ also remains unchanged. In the case where the pitch angle is 0, the upper and lower boundaries R and Q of the camera imaging are symmetric along the imaging center point P. However, when there is an elevation angle γ, the upper and lower boundaries R' and Q' of the imaging are no longer symmetric along the new imaging center point P'. Therefore, different degrees of distortion will occur in the upper and lower parts of the horizontal center line of the image. As shown in the middle and right figures of Figure 4 , the elevation angle of the camera measured by the high-precision tilt angle sensor is γ, the camera viewing angle is θ, the upper and lower boundaries of the camera imaging on the surface of the structure to be measured are R' and Q', and the corrected interface is R″P′Q″, which is perpendicular to the camera axis OP'. Therefore, the corrected upper and lower boundaries are R″ and Q″. According to the triangle geometric relationship, we can obtain:

[0077] ∠POP ′ =∠R″P ′ R ′ =γ

[0078]

[0079] Then, according to the sine theorem of the triangle, we can obtain:

[0080]

[0081] Similarly, the ratio of Q′P′ to Q″P′ is:

[0082]

[0083] In Figure 4 the right figure shown, the camera imaging area is U′R′Q′W′, the width of the image is w, and the height is h. First, perform perspective transformation correction on the upper half U′R′P′V′ of the image. Before correction, four boundary points in the upper half area of the image are selected, and their pixel coordinates are U′(0,0), R′(w,0), P′(w,h / 2), V′(0,h / 2) in sequence. After correction, the coordinates of these points become: (w,h / 2), (0,h / 2). Through the coordinate changes of the four boundary points, using the principle of perspective transformation, the perspective transformation matrix H 上 is quickly obtained, and this perspective transformation matrix is applied to the upper half area of the image to automatically correct the image deformation caused by distortion.

[0084] Next, perform perspective transformation correction on the lower half V′P′Q′W′ of the camera imaging region U′R′Q′W′. Before correction, the image pixel coordinates of the four boundary points are V′(0, h / 2), P′(w, h / 2), Q′(w, h), and W′(0, h) in sequence. After correction, the coordinates of these points become: (0, h / 2), (w, h / 2), Based on the coordinate changes of these points, quickly obtain the perspective transformation matrix H 下 , and apply this perspective transformation matrix to the lower half region of the image to automatically correct the image distortion caused by aberration.

[0085] Finally, merge the corrected upper half and the lower half into a complete image, ensuring a smooth transition at the seam between the two parts to guarantee the overall coherence and accuracy of the image, thereby obtaining the corrected image to ensure the accurate restoration of the true form of the measured structure. After obtaining the corrected image, existing image detection algorithms can be used to achieve precise perception of the measured structural surface, such as high-precision perception and recognition of cracks, deformations, and other defects on the surfaces of structures such as bridges, tunnels, and buildings.

[0086] In summary, this embodiment provides a visual detection information acquisition device based on horizontal angle calibration of an image and a visual detection information acquisition method based on horizontal angle calibration of an image implemented using this visual detection information acquisition device based on horizontal angle calibration of an image. The technical solution of this embodiment can correct the image distortion caused by the angle between the camera lens axis and the normal of the measured structure surface, thereby significantly improving the measurement accuracy. This solution is widely applicable to the detection and monitoring of apparent defects, damages, and deformations of engineering structures based on computer vision technology, and has the significant advantages of automation, high precision, and multi-functional perception.

[0087] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or terminal device comprising the element.

[0088] In addition, the term "and / or" is merely a description of the relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, both A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, in this text, the character " / " generally indicates an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, and specific understanding can be made by referring to the context before and after. "At least one" means one or more, and "a plurality" means two or more. "At least one of the following (items)" or similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0089] In addition, it can be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0090] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0091] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of functional modules / units is only a logical functional division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be in an electrical, mechanical, or other form. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, in each embodiment of the present invention, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0093] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0094] Finally, it should be noted that the above description is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiments of the present invention have been described, for those of ordinary skill in the art, once they know the basic creative concept of the present invention, without departing from the principle described in the present invention, several improvements and refinements can still be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A visual detection information acquisition device for calibrating an image based on a horizontal angle, characterized in that, Including: An industrial camera with a zoom lens, a pan-tilt device, a fixing device, a laser ranging sensor assembly, a signal acquisition synchronizer, and an image automatic calibration module; among them, the industrial camera is installed on the pan-tilt device, and the pan-tilt device is installed on the fixing device; the laser ranging sensor assembly is installed on the top of the industrial camera; the industrial camera and the laser ranging sensor assembly are respectively electrically connected to the signal acquisition synchronizer; The industrial camera is used to capture images of the surface area of the structure to be measured; The laser ranging sensor assembly is used to measure the distance information between the optical center of the industrial camera and different positions on the surface area of the structure to be measured in real time, and determine the horizontal angle formed between the lens axis of the industrial camera and the surface normal of the structure to be measured; the laser ranging sensor assembly includes a first laser ranging sensor, a second laser ranging sensor, and a third laser ranging sensor; among them, the second laser ranging sensor coincides with the optical axis of the industrial camera, the first laser ranging sensor and the third laser ranging sensor are symmetrically arranged on both sides of the optical axis of the industrial camera, and a first angle adjustment button is provided on the first laser ranging sensor, and a second angle adjustment button is provided on the third laser ranging sensor. The first angle adjustment button is mechanically connected to the first laser ranging sensor for adjusting the angle of the first laser ranging sensor; the second angle adjustment button is mechanically connected to the third laser ranging sensor for adjusting the angle of the third laser ranging sensor; The signal acquisition synchronizer is used to synchronously control the start and stop of the industrial camera and the laser ranging sensor assembly to ensure the consistency of data acquisition between the industrial camera and the laser ranging sensor assembly; The image automatic calibration module is used for: Obtaining the horizontal angle between the camera lens axis and the surface normal of the structure to be measured, which is expressed by the formula: Where S1 is the distance from the optical center measured by the first laser ranging sensor to the surface of the structure to be measured; S3 is the distance from the optical center measured by the third laser ranging sensor to the surface of the structure to be measured; S2 is the distance from the optical center measured by the second laser ranging sensor to the surface of the structure to be measured; the angles between the axes of the first laser ranging sensor and the second laser ranging sensor and between the axes of the third laser ranging sensor and the second laser ranging sensor are both α; Taking the vertical center line of the image captured by the camera as the boundary, perform perspective transformation correction on the left half area of the image. Before correction, select four boundary points in the left half area of the image, and their pixel coordinates are successively: (0, 0), (w / 2, 0), (w / 2, h), (0, h); after correction, the coordinates of the four boundary points become: (w / 2, 0), (w / 2, h), where θ is the camera view angle; w is the width of the image; h is the height of the image; through the coordinate changes of the four boundary points in the left half area of the image, using the principle of perspective transformation, obtain the perspective transformation matrix H of the left half area of the image 左 , and apply H 左 to the left half area of the image to automatically correct the image deformation caused by distortion; Taking the vertical center line of the image captured by the camera as the boundary, perform perspective transformation correction on the right half area of the image. Before correction, select four boundary points of the right half area of the image, and their pixel coordinates are successively: (w / 2, 0), (w, 0), (w, h), (w / 2, h); after correction, the coordinates of the four boundary points become: (w / 2, 0), (w / 2, h); According to the coordinate changes of the four boundary points of the right half area of the image, obtain the perspective transformation matrix H of the right half area of the image 右 , and apply H 右 to the right half area of the image to automatically correct the image deformation caused by distortion; Combining the left half area and the right half area of the corrected image into a complete image, ensuring a smooth transition of the seam between the two parts to obtain the corrected image.

2. The visual detection information acquisition device for calibrating an image based on a horizontal angle as claimed in claim 1, wherein, The visual detection information acquisition device further includes an adaptive fill light; The adaptive fill light is electrically connected to the signal acquisition synchronizer, and the signal acquisition synchronizer synchronously controls the start and stop of the adaptive fill light, the industrial camera, and the laser ranging sensor assembly; The adaptive fill light can automatically adjust the brightness according to the intensity of the ambient light to ensure the clarity of the images captured by the industrial camera during image acquisition.

3. The visual detection information acquisition device for calibrating an image based on a horizontal angle as claimed in claim 1, wherein, The visual detection information acquisition device further includes a multi-modal data fusion module; the multi-modal data fusion module uses a preset data fusion algorithm to synchronously process the image captured by the industrial camera and the distance information collected by the laser ranging sensor assembly, so as to realize the perception of the surface of the structure to be measured.

4. A method for collecting visual detection information with image calibration based on horizontal angle, which is realized by using the visual detection information collection device with image calibration based on horizontal angle according to any one of claims 1 to 3, characterized in that, The visual detection information acquisition method based on calibrating the image by the horizontal angle includes: Using the signal acquisition synchronizer to synchronously control the start and stop of the industrial camera and the laser ranging sensor assembly, using the laser ranging sensor assembly to measure the distance information between the optical center of the industrial camera and different positions on the surface area of the structure to be measured in real time, and determining the horizontal angle formed between the lens axis of the industrial camera and the normal line of the surface of the structure to be measured; Using the image automatic calibration module to correct the distortion of the image captured by the industrial camera caused by the horizontal angle between the lens axis of the industrial camera and the normal line of the surface of the structure to be measured, and obtaining the corrected image.

Images