A bridge guardrail punching positioning method and system based on infrared ranging and visual fusion

Abstract

The present application relates to the technical field of image processing, and proposes a bridge guardrail punching positioning method and system based on infrared ranging and visual fusion, which comprises the following steps: collecting a visual work image of a bridge guardrail and recording candidate hole positions and risk components in the image, obtaining a visual fine measurement demand area of each candidate hole position; obtaining an infrared ranging prediction fine measurement window and a window following offset degree; obtaining a plurality of effective infrared ranging points to obtain an infrared effective coverage degree of the candidate hole position; obtaining a regional distance continuity of the candidate hole position; obtaining a dynamic fine measurement bearing degree of the candidate hole position; performing threshold judgment on the dynamic fine measurement bearing degree to obtain an infrared ranging fine measurement window of the candidate hole position; and correcting the candidate hole position based on the visual fine measurement demand area and the infrared ranging fine measurement window to obtain a target hole position of the candidate hole position. The present application aims to solve the problem that the fusion of visual collection and infrared ranging is affected by the device posture and offset and produces an offset.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for positioning and drilling holes in bridge railings by fusing infrared ranging and vision. Background Technology

[0002] During the installation, renovation, or reinforcement of bridge railings, drilling equipment typically moves along the length of the railing and needs to create holes on the sides, top, or in the area where connectors are installed. To improve positioning efficiency, existing equipment can use cameras to acquire images of a large area of ​​the railing work, from which candidate holes, railing edges, posts, old holes, cracks, joints, and obstructed areas can be identified. Visual recognition can provide a two-dimensional relationship between the hole and surrounding components, but it cannot reliably provide the true distance, local tilt, protrusions, or depressions at candidate hole locations.

[0003] Infrared ranging can obtain local distance information, but its range is small. If the distance is measured point by point on the entire visual working surface, a large amount of distance data unrelated to drilling will be generated, and it may measure columns, bolts, edges of old holes, reflective parts, or obstructions. More importantly, bridge railing drilling equipment is in a mobile working state. The camera usually sees the candidate hole position first, and the infrared ranging module then arrives at that local area. Due to the installation offset between the camera and the infrared module, and the displacement, vibration, and attitude changes that occur during the movement of the equipment, the hole position area identified by vision may have already shifted at the time of infrared ranging. Existing vision and infrared fusion solutions mostly rely on static position superposition, that is, projecting the infrared ranging point onto the visual image and determining whether it falls near the candidate hole position.

[0004] However, in the scenario of moving holes in bridge railings, there is a time difference, installation offset, and posture change between visual recognition and infrared ranging. Static projection cannot determine whether the infrared ranging has accepted the precise measurement requirements identified by visual recognition at the correct time, in the correct position, and on the correct railing surface. Therefore, it is necessary to migrate the area of ​​visual precise measurement requirements to the predicted precise measurement window at the time of infrared ranging based on the movement status of the equipment, and determine whether the infrared ranging point dynamically accepts the window, thereby improving the reliability of hole position correction. Summary of the Invention

[0005] This invention provides a method and system for positioning holes in bridge railings by fusing infrared ranging and vision, to solve the problem of deviation caused by equipment posture and offset in existing vision acquisition and infrared ranging fusion methods. The specific technical solution adopted is as follows: This invention proposes a method for positioning holes in bridge railings by fusing infrared ranging and vision. The method includes the following steps: Collect visual images of bridge railings and record candidate holes and risky components to obtain the visual precision measurement requirement area for each candidate hole. The influence of device displacement and attitude changes under the time difference between visual acquisition and infrared ranging is analyzed to obtain the infrared ranging prediction and precision measurement window corresponding to the visual precision measurement requirement area, as well as its window following offset. Based on the distribution of infrared ranging points in the infrared ranging prediction and precision measurement window, several effective infrared ranging points are obtained and combined with the window following offset to obtain the effective infrared coverage of the candidate aperture. Analyze the distance performance between effective infrared ranging points in the infrared ranging prediction precision measurement window to obtain the regional distance continuity of candidate aperture locations; combine the effective infrared coverage and the regional distance continuity to obtain the dynamic precision measurement acceptance of candidate aperture locations; perform threshold judgment on the dynamic precision measurement acceptance, and adjust the infrared ranging precision measurement window of candidate aperture locations by combining the window following offset, the regional distance continuity, and the number of effective infrared ranging points; Based on the visual precision measurement requirement area and the infrared ranging precision measurement window, the candidate hole positions are corrected to obtain the target hole positions of the candidate holes.

[0006] Optionally, the specific method for obtaining the visual precision measurement requirement area for each candidate aperture is as follows: Based on the distribution of candidate hole locations and risky components in the visual operation image, the local region of any candidate hole location and the visual recognition confidence of each local location therein are obtained; the minimum distance between any local location and the risky component is obtained. A preset safety boundary influence range is defined, and the ratio of the safety boundary influence range to the minimum distance is obtained. Based on the ratio and the visual recognition confidence level, the visual precision measurement requirement at any local location is obtained. The visual precision measurement requirement is positively correlated with both the ratio and the visual recognition confidence level. The local locations in a local area of ​​any candidate aperture where the visual precision measurement requirement is greater than a preset requirement threshold constitute the visual precision measurement requirement area of ​​that candidate aperture.

[0007] Optionally, the infrared ranging prediction precision measurement window corresponding to the visual precision measurement requirement area is obtained using the following method: For any candidate aperture position, the visual frame acquisition time and infrared ranging time are obtained. Based on the visual frame acquisition time and the infrared ranging time, the displacement vector and attitude change of the device are obtained, and the installation offset vector of the camera module and the infrared ranging module is obtained. Based on the displacement vector and attitude change, and combined with the installation offset vector, the visual precision measurement requirement area of ​​the candidate hole is moved and adjusted to obtain the infrared ranging prediction precision measurement window corresponding to the visual precision measurement requirement area.

[0008] Optionally, the window follows the offset, and the specific method for obtaining this offset is as follows: Obtain the distance between the actual infrared ranging center of any candidate aperture and the center of its infrared ranging prediction and fine measurement window. Use the ratio of this distance to the radius of the infrared ranging prediction and fine measurement window, and normalize the result, as the window following offset of the candidate aperture.

[0009] Optionally, the specific method for obtaining the infrared effective coverage of the candidate aperture by acquiring several effective infrared ranging points and combining them with the window following offset is as follows: Infrared ranging points are projected onto the visual operation image. The infrared ranging points within the infrared ranging prediction and fine measurement window of any candidate hole are projected as several effective infrared ranging points of the candidate hole. The infrared ranging points located in the corresponding area of ​​the risk component in the local area of ​​the candidate hole are projected as several positions to be confirmed of the candidate hole. Obtain the ratio of the number of effective infrared ranging points to the number of locations to be confirmed; based on this ratio and the window following offset of the candidate aperture, obtain the effective infrared coverage of the candidate aperture. The effective infrared coverage is positively correlated with the ratio and negatively correlated with the window following offset.

[0010] Optionally, the method for obtaining the regional distance continuity of the candidate aperture locations is as follows: For any candidate aperture location with several effective infrared ranging points, the distance between adjacent effective infrared ranging points corresponding to the sequentially arranged infrared ranging points is obtained and used as the distribution distance of the next effective infrared ranging point among the adjacent effective infrared ranging points; the maximum value of the absolute value of the difference between the distribution distances of any two adjacent effective infrared ranging points is obtained, and the inverse proportional normalization result of the ratio of the maximum value to the preset local distance reference value is used as the regional distance continuity of the candidate aperture location.

[0011] Optionally, the specific method for obtaining the dynamic precision bearing capacity of candidate hole positions includes: The dynamic precision measurement acceptance of any candidate aperture is obtained by multiplying the effective infrared coverage of the aperture with the continuity of the regional distance.

[0012] Optionally, the specific method for obtaining the infrared ranging precision measurement window for candidate aperture positions includes: If the dynamic precision measurement acceptance rate of any candidate aperture is greater than or equal to the preset threshold, the infrared ranging prediction precision measurement window of the candidate aperture will be used as the infrared ranging precision window of the candidate aperture. If the dynamic precision measurement acceptance is less than the set threshold, analyze the window following offset, regional distance continuity and number of effective infrared ranging points of the candidate hole, and compare it with the average window following offset, average regional distance continuity and number of effective infrared ranging points of all candidate holes. If the window following offset is greater than the average, the infrared ranging module is corrected and the device is allowed to stabilize. If the area distance continuity is less than the average, the flatness of the local area of ​​the candidate hole is confirmed. If the number of effective infrared ranging points is less than the average, infrared ranging points are added. Finally, the adjustment is made so that the adjusted dynamic precision measurement acceptance is greater than or equal to the set threshold, and the adjusted infrared ranging prediction precision measurement window is used as the infrared ranging precision window for the candidate hole.

[0013] Optionally, the specific method for correcting candidate hole positions based on the visual precision measurement requirement area and the infrared ranging precision measurement window to obtain the target hole position of the candidate hole position includes: For any candidate hole, obtain the displacement vector of the center point of the visual precision measurement requirement area of ​​the candidate hole pointing to the infrared ranging precision measurement window, and take the position of the candidate hole after adjustment according to the displacement vector as the target hole.

[0014] The present invention also proposes a bridge railing drilling and positioning system that integrates infrared ranging and vision. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the above method.

[0015] The beneficial effects of this invention are as follows: Compared with a simple visual positioning scheme, this invention can supplement the actual distance, local tilt, and protrusion information at the hole position; compared with a scheme that involves dense infrared scanning of the entire working surface, it can reduce irrelevant ranging points and improve infrared ranging efficiency; compared with a static fusion scheme, it can consider the window offset caused by equipment movement, so that infrared ranging truly covers the precise measurement requirement area identified by visual recognition, thereby improving the stability and hole position correction accuracy of bridge railing moving drilling positioning; it does not simply superimpose visual images and infrared distances, but first forms the precise measurement requirement area around the hole position through visual recognition, and then generates an infrared ranging prediction precise measurement window based on equipment displacement, camera and infrared module installation offset, and equipment attitude changes; subsequently, it determines whether the effective infrared ranging points cover the hole center, safety boundary, and adjacent area of ​​risky components in the window, and forms a dynamic precise measurement acceptance degree accordingly, and finally obtains the infrared ranging precise measurement window and uses it for drilling positioning of bridge railings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This is a schematic diagram of a bridge railing drilling and positioning method based on infrared ranging and vision fusion, provided in one embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 The diagram illustrates a flowchart of a bridge railing drilling and positioning method based on infrared ranging and vision fusion, according to an embodiment of the present invention. The method includes the following steps: Step S001: Collect visual operation images of bridge railings and record candidate hole positions and risky components, and obtain the visual precision measurement requirement area for each candidate hole position.

[0020] The purpose of this embodiment is to enable the infrared ranging module to perform precise measurements only on key local areas identified by vision during the continuous operation of the bridge railing mobile drilling equipment, and to determine whether the precise measurement process has effectively inherited the visual recognition results after movement compensation, thereby reducing invalid ranging and improving the reliability of hole position correction.

[0021] It should be noted that the drilling and positioning equipment moves along the length of the bridge railing. The equipment is equipped with a vision camera, an infrared ranging module, a displacement detection unit, an attitude detection unit, and a drilling execution mechanism. The vision camera is located in front of or to the side of the infrared ranging module. Therefore, the camera first acquires a large-area image of the candidate hole location, and the infrared ranging module performs local ranging near the hole location as the equipment continues to move. In actual construction, the equipment may move close to the railing along temporary guide rails, suction wheels, or traveling wheels. The railing surface may also have posts, end joints, old holes, bolts, reflective coatings, construction dust, and localized corrosion. The working area acquired by the vision camera at one time is usually significantly larger than the coverage area of ​​the infrared ranging, enabling rapid identification of candidate hole locations and surrounding risky components. The role of infrared ranging is to confirm the actual distance near the hole location, local protrusions, depressions, or tilting. If the infrared ranging points are directly superimposed onto the original hole location area in the vision image, it is easy to overlook the window offset caused by equipment movement and attitude changes, and it is also easy to mistakenly identify ranging results where infrared points fall on the edge of old holes, the side of posts, or reflective obstructions as valid hole location rangings.

[0022] Specifically, visual operation images are acquired using the vision camera in the camera module. After acquiring the visual operation images, the visual center point of the candidate hole position is identified, and a local area of ​​the candidate hole position is established based on the visual center point. The local area is not the entire image, but the local area around the candidate hole position that truly affects the drilling reliability. The local area is a circle with the visual center point as the center. The radius is not specifically limited in this embodiment and can be set by the implementer according to the construction situation. The hole center area, safety boundary area, guardrail edge adjacent area, old hole crack adjacent area, column obstruction adjacent area, and low confidence identification area are extracted. The hole center determines the drill bit entry position, the safety boundary determines whether the drilling avoids the edge, old hole, crack, or column, and the low confidence identification area indicates the position that cannot be reliably confirmed by visual images alone. Risk components are marked, including guardrail edges, column boundaries, old hole edges, crack ends, and obstruction area boundaries.

[0023] It should be noted that if a location is closer to a risky component and the confidence level of visual recognition is lower, then infrared ranging confirmation is more necessary for that location; therefore, the visual recognition results need to be organized into areas requiring precise measurement, rather than being used directly as the final drilling location.

[0024] Furthermore, based on the distribution of candidate hole locations and risky components in the visual operation image, the local region of any candidate hole location and the visual recognition confidence of each local location are obtained (the visual recognition confidence of each local location is directly obtained during the extraction of low-confidence recognition regions); the minimum distance between any local location and the risky component is obtained; the safety boundary influence range is preset, and in this embodiment, the safety boundary influence range is obtained based on the equivalent radius of the safety boundary region, which will not be elaborated further in this embodiment; the ratio of the safety boundary influence range to the minimum distance is obtained, and based on the ratio and the visual recognition confidence, the visual precision measurement requirement of any local location is obtained, and the visual precision measurement requirement is positively correlated with both the ratio and the visual recognition confidence.

[0025] As an example, this embodiment uses the product of the ratio and the visual recognition confidence level and the result of normalization as the visual detection demand of the corresponding local position. The normalization adopts the maximum and minimum value normalization, and the normalization object is the product of all local positions in the local area of ​​the candidate hole position.

[0026] Furthermore, the local locations in any candidate hole location where the visual precision measurement requirement is greater than a preset requirement threshold are defined as the visual precision measurement requirement area for that candidate hole location. In this embodiment, the requirement threshold is described as 0.5, which is set according to the judgment process of the low confidence recognition area corresponding to the visual recognition confidence of the bridge railing. The specific process will not be described in detail.

[0027] It should be noted that the visual camera and the infrared ranging module do not acquire data on the same area at the same time and in the same spatial location. The camera first identifies candidate holes, and the infrared module then measures the distance. The visual image has a larger coverage area, which is suitable for early detection of candidate holes and risky components during the movement of the equipment. The infrared ranging has a smaller coverage area, which is suitable for obtaining fine distances near candidate holes. If the equipment moves, vibrates, or changes its posture between two acquisitions, the area requiring visual precision measurement needs to be migrated to the moment of infrared ranging before determining whether the infrared ranging effectively covers it. Therefore, a predictive precision measurement window is formed first, and then it is determined whether the infrared ranging point effectively covers the window. This is the key difference from static visual infrared overlay.

[0028] Step S002: Analyze the impact of device displacement and attitude changes under the time difference between visual acquisition and infrared ranging, obtain the infrared ranging prediction and precision measurement window corresponding to the visual precision measurement requirement area, and its window following offset; based on the distribution performance of infrared ranging points in the infrared ranging prediction and precision measurement window, obtain several effective infrared ranging points and combine them with the window following offset to obtain the effective infrared coverage of the candidate aperture.

[0029] It should be noted that the local demand area obtained by visual recognition is converted to the actual time and location of infrared ranging. Due to the small range of infrared ranging, if the migrated window is incorrect, even if there are enough subsequent ranging points, it is possible to measure the pillar next to the candidate hole, the edge of the old hole, or the non-target guardrail surface. Especially when the equipment moves close to the guardrail, short-term pitch changes will amplify the deviation of the infrared ranging direction, causing the ranging point that should fall near the center of the hole to deviate to the edge area. Therefore, it is necessary to first form a predictive precision measurement window based on the time difference, displacement, installation offset, and attitude change.

[0030] Preferably, in one embodiment of the present invention, the method for analyzing the influence of device displacement and attitude changes due to the time difference between visual acquisition and infrared ranging, and obtaining the infrared ranging prediction precision measurement window corresponding to the visual precision measurement requirement area, as well as its window following offset, includes: For any candidate hole location, the visual frame acquisition time and infrared ranging time are obtained. Based on the visual frame acquisition time and the infrared ranging time, the displacement vector and attitude change of the device are obtained, and the installation offset vector of the camera module and the infrared ranging module is also obtained. The displacement vector of the device is obtained by an encoder, odometer, or track movement detection unit, and is used to represent the positional change of the visual area after the device moves along the guardrail direction. The attitude change is obtained by an tilt sensor, IMU, or device attitude detection unit, and is used to represent the window rotation and offset caused by device vibration, tilt, or changes in the angle of contact with the guardrail. The installation offset vector is obtained during the device calibration stage and is used to represent the natural spatial misalignment when the two sensors observe the same guardrail area. Based on the displacement vector and attitude change, combined with the installation offset vector, the visual precision measurement requirement area of ​​the candidate hole location is moved and adjusted to obtain the infrared ranging prediction precision measurement window corresponding to the visual precision measurement requirement area.

[0031] Furthermore, the distance between the actual infrared ranging center of any candidate aperture and the center of its infrared ranging prediction and fine measurement window is obtained. The ratio of this distance to the radius of the infrared ranging prediction and fine measurement window, after normalization, is used as the window following offset of the candidate aperture. The normalization adopts a linear normalization method, and the normalization object is the ratio corresponding to all candidate apertures.

[0032] It should be noted that the greater the offset between the center of the predicted precision measurement window and the center of the actual infrared ranging window, the worse the infrared window follows the visual requirement area; the greater the deviation of the actual infrared ranging window from the predicted precision measurement window, the more it indicates that the visual requirement area is not being stably followed.

[0033] It should be further noted that even if the infrared ranging point falls within the prediction precision measurement window, it may not be able to be used for hole position correction. For example, the ranging point may fall on the post, reflector, edge of old hole, or obstruction, or it may measure a non-target guardrail surface due to changes in equipment posture. Therefore, it is necessary to first analyze whether the effective infrared ranging point covers the area required for visual precision measurement, and then analyze whether its distance change conforms to the continuous performance of the same guardrail target surface, thereby obtaining the dynamic precision measurement acceptance.

[0034] Preferably, in one embodiment of the present invention, based on the distribution of infrared ranging points within the infrared ranging prediction precision window, several effective infrared ranging points are obtained and combined with the window following offset to obtain the effective infrared coverage of the candidate aperture position. The specific method includes: It should be noted that if the infrared ranging point only falls on the edge of the predicted precision measurement window or in non-target areas such as pillars, old holes, cracks, or obstructions, it cannot be said that the infrared ranging has truly confirmed the area required for visual precision measurement. For candidate hole positions, the infrared ranging should at least cover the vicinity of the hole center and the vicinity of the safety boundary, and this coverage needs to be based on the premise that the window following offset is small.

[0035] Specifically, infrared ranging points are projected onto the visual operation image. Infrared ranging points within the infrared ranging prediction and fine measurement window of any candidate hole are projected as several effective infrared ranging points for that candidate hole. Infrared ranging points located in the corresponding area of ​​the risk component in a local area of ​​the candidate hole are projected as several locations to be confirmed for that candidate hole. The ratio of the number of effective infrared ranging points to the number of locations to be confirmed is obtained. Based on this ratio and the window following offset of the candidate hole, the effective infrared coverage of the candidate hole is obtained. The effective infrared coverage is positively correlated with the ratio and negatively correlated with the window following offset.

[0036] As an example, this embodiment obtains the inversely proportional normalized result of the window following offset, and uses the product of the obtained result and the ratio as the window following offset of the candidate hole position. In this embodiment, the following method is used: The model is used to represent the inverse proportional relationship and for normalization processing. As input to the model, This represents an exponential function with the natural constant as the base. Implementers can set inverse proportional functions and normalization functions according to the actual situation.

[0037] It should be noted that even if the number of infrared points is large, the effective coverage will decrease if the window tracking offset is large.

[0038] Thus, the infrared ranging prediction precision window and effective infrared coverage of the candidate aperture positions are obtained.

[0039] Step S003: Analyze the distance performance between effective infrared ranging points in the infrared ranging prediction precision measurement window to obtain the regional distance continuity of candidate aperture locations; combine the effective infrared coverage and the regional distance continuity to obtain the dynamic precision measurement acceptance of candidate aperture locations; perform threshold judgment on the dynamic precision measurement acceptance, and adjust the infrared ranging precision measurement window of candidate aperture locations by combining the window following offset, the regional distance continuity, and the number of effective infrared ranging points.

[0040] It should be noted that distance continuity is used to determine whether infrared ranging points come from the same guardrail target surface; if the distance between adjacent effective ranging points changes a lot, it is more likely that there are edges, protrusions, obstructions or non-target surfaces for ranging.

[0041] Preferably, in one embodiment of the present invention, the method for analyzing the distance performance between effective infrared ranging points in the infrared ranging prediction precision measurement window and obtaining the regional distance continuity of candidate aperture positions includes: For any candidate aperture location with several effective infrared ranging points, the distance between adjacent effective infrared ranging points corresponding to the sequentially arranged infrared ranging points is obtained and used as the distribution distance of the next effective infrared ranging point among the adjacent effective infrared ranging points; the maximum value of the absolute value of the difference between the distribution distances of any two adjacent effective infrared ranging points is obtained, and the inverse proportional normalization result of the ratio of the maximum value to the preset local distance reference value is used as the regional distance continuity of the candidate aperture location.

[0042] It should be noted that the greater the abrupt change in adjacent distances, the smaller the continuity of regional distances, indicating that the ranging results within this window are not suitable for direct use in hole position correction.

[0043] Preferably, in one embodiment of the present invention, the dynamic precision measurement acceptance of candidate aperture locations is obtained by combining the effective infrared coverage and the continuity of the regional distance, including the following specific method: The dynamic precision measurement acceptance degree of any candidate aperture is obtained by multiplying the effective infrared coverage of the aperture with the continuity of the regional distance. As an example, this embodiment uses the obtained product and normalized result as the dynamic precision measurement acceptance degree of the candidate aperture. The normalization adopts the maximum and minimum value normalization, and the normalization object is the product obtained for all candidate apertures.

[0044] It should be noted that by multiplying the effective infrared coverage by the continuity of the area distance, both effective infrared coverage and the continuity of the distance to the same target surface are necessary conditions. If either condition is insufficient, the reliability of infrared ranging in meeting the requirements of visual precision measurement will be reduced.

[0045] Preferably, in one embodiment of the present invention, the infrared ranging precision measurement window for candidate aperture positions is obtained by threshold judgment of the dynamic precision measurement acceptance degree and adjustment in combination with the window following offset, the regional distance continuity and the number of effective infrared ranging points. The specific method includes: If the dynamic precision measurement acceptance of any candidate aperture is greater than or equal to a preset threshold, the infrared ranging prediction precision measurement window of that candidate aperture is used as the infrared ranging accuracy window for that candidate aperture. If the dynamic precision measurement acceptance is less than the preset threshold, the window following offset, regional distance continuity, and number of effective infrared ranging points of the candidate aperture are analyzed and compared with the average window following offset, average regional distance continuity, and number of effective infrared ranging points of all candidate apertures (past candidate apertures). If the window following offset is greater than the average, the infrared ranging module is corrected and the device is allowed to stabilize. If the regional distance continuity is less than the average, it is confirmed whether the local area of ​​the candidate aperture is flat. If the number of effective infrared ranging points is less than the average, infrared ranging points are added. Finally, adjustments are made to ensure that the adjusted dynamic precision measurement acceptance is greater than or equal to the preset threshold, and the adjusted infrared ranging prediction precision measurement window is used as the infrared ranging accuracy window for that candidate aperture.

[0046] It should be noted that when the dynamic precision measurement acceptance reaches the set threshold, it means that the infrared small window has effectively accepted the visual precision measurement requirements after the equipment movement compensation; when it does not reach the threshold, the system further distinguishes the reasons for the deficiency; if the window tracking offset is large, the window position is corrected first or the equipment is allowed to stabilize; if the number of effective infrared ranging points is insufficient, ranging points are added for the hole center or safety boundary; if the area distance continuity is low, it is necessary to confirm whether there are local protrusions, edge reflections or non-target surface ranging.

[0047] At this point, the infrared ranging precision measurement window for the candidate aperture position is obtained.

[0048] Step S004: Based on the visual precision measurement requirement area and the infrared ranging precision measurement window, the candidate hole positions are corrected to obtain the target hole positions of the candidate hole positions.

[0049] It should be noted that once the dynamic precision measurement of the bearing capacity meets the standard, the system uses effective infrared ranging points to fit the local guardrail surface near the candidate hole position; the local guardrail surface can be determined by effective infrared ranging points near the hole center and at the safety boundary, thus obtaining the lateral, longitudinal and normal deviations of the hole position relative to the actual guardrail surface.

[0050] Specifically, for any candidate hole position, the displacement vector of the center point of the visual precision measurement requirement area of ​​the candidate hole position pointing to the infrared ranging precision measurement window is obtained, and the position of the candidate hole position after adjustment according to the displacement vector is taken as the target hole position of the candidate hole position.

[0051] It should be noted that the system adjusts the drilling mechanism by adjusting its lateral position, longitudinal position, normal distance, or attitude angle, i.e., by using a displacement vector, so that the drill bit is aligned with the target hole on the actual guardrail surface.

[0052] Furthermore, if the dynamic precision measurement accuracy does not meet the standard, the system will not directly execute drilling. Instead, it will perform additional ranging, window repositioning, or wait for the equipment to stabilize, depending on the reason for the deficiency. This avoids window offset caused by equipment movement, incorrect hole position correction caused by ranging from non-target surfaces, and drilling risks caused by unconfirmed sudden changes in local distance. Finally, the system obtains the target hole position from all candidate hole positions in the visual operation image to eliminate the deviation generated during the visual fusion infrared ranging process, realizes drilling positioning, and provides a basis for subsequent drilling.

[0053] This concludes the embodiment.

[0054] Another embodiment of the present invention provides a bridge railing drilling and positioning system that combines infrared ranging and vision. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the above-described method steps S001 to S004.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for positioning and drilling holes in bridge railings by fusing infrared ranging and vision, characterized in that, The method includes the following steps: Collect visual images of bridge railings and record candidate holes and risky components to obtain the visual precision measurement requirement area for each candidate hole. The influence of device displacement and attitude changes under the time difference between visual acquisition and infrared ranging is analyzed to obtain the infrared ranging prediction and precision measurement window corresponding to the visual precision measurement requirement area, as well as its window following offset. Based on the distribution of infrared ranging points in the infrared ranging prediction and precision measurement window, several effective infrared ranging points are obtained and combined with the window following offset to obtain the effective infrared coverage of the candidate aperture. Analyze the distance performance between effective infrared ranging points in the infrared ranging prediction precision measurement window to obtain the regional distance continuity of candidate aperture locations; combine the effective infrared coverage and the regional distance continuity to obtain the dynamic precision measurement acceptance of candidate aperture locations; perform threshold judgment on the dynamic precision measurement acceptance, and adjust the infrared ranging precision measurement window of candidate aperture locations by combining the window following offset, the regional distance continuity, and the number of effective infrared ranging points; Based on the visual precision measurement requirement area and the infrared ranging precision measurement window, the candidate hole positions are corrected to obtain the target hole positions of the candidate holes. The specific method for obtaining the infrared effective coverage of candidate aperture positions by acquiring several effective infrared ranging points and combining them with the window following offset is as follows: Infrared ranging points are projected onto the visual operation image. The infrared ranging points within the infrared ranging prediction and fine measurement window of any candidate hole are projected as several effective infrared ranging points of the candidate hole. The infrared ranging points located in the corresponding area of ​​the risk component in the local area of ​​the candidate hole are projected as several positions to be confirmed of the candidate hole. Obtain the ratio of the number of effective infrared ranging points to the number of locations to be confirmed; based on this ratio and the window following offset of the candidate aperture, obtain the effective infrared coverage of the candidate aperture. The effective infrared coverage is positively correlated with the ratio and negatively correlated with the window following offset. The specific method for obtaining the regional distance continuity of the candidate aperture positions is as follows: For any candidate aperture location with several effective infrared ranging points, the distance between adjacent effective infrared ranging points corresponding to the sequentially arranged infrared ranging points is obtained and used as the distribution distance of the next effective infrared ranging point among the adjacent effective infrared ranging points; the maximum value of the absolute value of the difference between the distribution distances of any two adjacent effective infrared ranging points is obtained, and the inverse proportional normalization result of the ratio of the maximum value to the preset local distance reference value is used as the regional distance continuity of the candidate aperture location. The specific method for obtaining the dynamic precision bearing capacity of candidate borehole positions is as follows: The dynamic precision measurement acceptance of any candidate aperture is obtained by multiplying the effective infrared coverage of the aperture with the continuity of the regional distance.

2. The method for positioning and drilling holes in bridge railings by fusing infrared ranging and vision according to claim 1, characterized in that, The specific method for obtaining the visual precision measurement requirement area for each candidate aperture is as follows: Based on the distribution of candidate hole locations and risky components in the visual operation image, the local region of any candidate hole location and the visual recognition confidence of each local location therein are obtained; the minimum distance between any local location and the risky component is obtained. A preset safety boundary influence range is defined, and the ratio of the safety boundary influence range to the minimum distance is obtained. Based on the ratio and the visual recognition confidence level, the visual precision measurement requirement at any local location is obtained. The visual precision measurement requirement is positively correlated with both the ratio and the visual recognition confidence level. The local locations in a local area of ​​any candidate aperture where the visual precision measurement requirement is greater than a preset requirement threshold constitute the visual precision measurement requirement area of ​​that candidate aperture.

3. The method for positioning and drilling holes in bridge railings by fusing infrared ranging and vision according to claim 1, characterized in that, The infrared ranging prediction precision measurement window corresponding to the visual precision measurement requirement area is obtained as follows: For any candidate aperture position, the visual frame acquisition time and infrared ranging time are obtained. Based on the visual frame acquisition time and the infrared ranging time, the displacement vector and attitude change of the device are obtained, and the installation offset vector of the camera module and the infrared ranging module is obtained. Based on the displacement vector and attitude change, and combined with the installation offset vector, the visual precision measurement requirement area of ​​the candidate hole is moved and adjusted to obtain the infrared ranging prediction precision measurement window corresponding to the visual precision measurement requirement area.

4. The bridge railing drilling and positioning method based on infrared ranging and vision fusion according to claim 3, characterized in that, The window following offset is obtained using the following method: Obtain the distance between the actual infrared ranging center of any candidate aperture and the center of its infrared ranging prediction and fine measurement window. Use the ratio of this distance to the radius of the infrared ranging prediction and fine measurement window, and normalize the result, as the window following offset of the candidate aperture.

5. The method for positioning and drilling holes in bridge railings by fusing infrared ranging and vision according to claim 1, characterized in that, The specific method for obtaining the infrared ranging precision measurement window for candidate aperture positions is as follows: If the dynamic precision measurement acceptance rate of any candidate aperture is greater than or equal to the preset threshold, the infrared ranging prediction precision measurement window of the candidate aperture will be used as the infrared ranging precision window of the candidate aperture. If the dynamic precision measurement acceptance is less than the set threshold, analyze the window following offset, regional distance continuity and number of effective infrared ranging points of the candidate hole, and compare it with the average window following offset, average regional distance continuity and number of effective infrared ranging points of all candidate holes. If the window following offset is greater than the mean, correct the infrared ranging module and wait for the device to stabilize. If the continuity of the regional distance is less than the average, confirm whether the local area of ​​the candidate hole is flat; if the number of effective infrared ranging points is less than the average, supplement the infrared ranging points; finally, adjust so that the adjusted dynamic precision measurement acceptance is greater than or equal to the set threshold, and use the adjusted infrared ranging prediction precision measurement window as the infrared ranging precision window of the candidate hole.

6. The method for positioning and drilling holes in bridge railings by combining infrared ranging and vision according to claim 1, characterized in that, The method for correcting candidate hole positions based on the visual precision measurement requirement area and the infrared ranging precision measurement window to obtain the target hole position includes the following specific methods: For any candidate hole, obtain the displacement vector of the center point of the visual precision measurement requirement area of ​​the candidate hole pointing to the infrared ranging precision measurement window, and take the position of the candidate hole after adjustment according to the displacement vector as the target hole.

7. A bridge railing drilling and positioning system integrating infrared ranging and vision, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the bridge railing drilling and positioning method according to any one of claims 1-6.

Images