Bridge large section adaptive intelligent assembling construction method based on machine vision

Abstract

This invention discloses an adaptive intelligent assembly construction method for large bridge segments based on machine vision, belonging to the field of bridge assembly construction technology. This invention is implemented based on a drone, a base station, and a processing unit. The base station includes a first image acquisition module, an Ethernet module, and a wireless module. The drone includes a flight control module, a positioning module, an image transmission module, an encoding module, and a second image acquisition module. The method includes the following steps: Step 1, deploying the drone and base station and calibrating their parameters to establish a coordinate system; Step 2, the base station and drone collaboratively acquire images and transmit them to the processing unit. This invention achieves a balance between the real-time performance and stitching accuracy of the digital twin model by collaboratively acquiring image information through a fixed ground base station and a mobile drone. It obtains planar position information through the positioning module and combines this with machine vision analysis of the image information to obtain altitude position information, improving spatial positioning accuracy and eliminating problems of model stitching misalignment and data distortion.

Description

Technical Field

[0001] This invention relates to the field of bridge assembly and construction technology, specifically to an adaptive intelligent assembly and construction method for large bridge segments based on machine vision. Background Technology

[0002] The assembly and construction of large bridge segments is a core process in the construction of long-span bridges. Its construction accuracy and work efficiency directly determine the overall construction quality and service life of the bridge. Machine vision technology acquires the spatial location and morphological parameters of the construction target through image acquisition and feature extraction, while digital twin technology constructs a virtual model synchronized with the physical site through multi-source data fusion. The combination of the two can realize the visualization and digital control of the bridge assembly and construction process, and is widely used in various segmental assembly and construction scenarios of long-span bridges, providing technical support for automated and intelligent construction.

[0003] Current technologies for large-segment bridge assembly construction still suffer from several unresolved technical shortcomings. Firstly, image acquisition in existing large-segment bridge assembly construction often relies on fixed camera positions or single mobile acquisition devices. Fixed camera positions have blind spots due to their fixed viewing angles, while single mobile acquisition devices cannot balance acquisition efficiency and coverage. Secondly, image transmission uses a fixed compression ratio, which cannot meet the varying real-time and accuracy requirements of different assembly stages, easily leading to excessive bandwidth consumption or insufficient model splicing accuracy. Thirdly, existing technologies often use a single satellite positioning module for spatial positioning of mobile acquisition devices. Planar positioning accuracy cannot meet the needs of refined construction, and elevation positioning accuracy has significant deviations, easily resulting in misalignment of digital twin models and data distortion. Finally, existing large-segment bridge assembly construction processes largely rely on manual experience, failing to achieve rapid reasoning of the globally optimal solution based on real-time on-site conditions. This leads to delayed monitoring of assembly deviations and a lack of timely warnings and handling of abnormal situations, easily resulting in accumulated assembly deviations and high rework rates. Therefore, designing an adaptive intelligent assembly construction method for large-segment bridges based on machine vision technology is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an adaptive intelligent assembly construction method for large bridge segments based on machine vision, which solves the problems mentioned in the background section.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a machine vision-based adaptive intelligent assembly construction method for large bridge segments, implemented using a drone, a base station, and a processing unit. The base station includes a first image acquisition module, an Ethernet module, and a wireless module. The drone includes a flight control module, a positioning module, an image transmission module, an encoding module, and a second image acquisition module. The method comprises the following steps: Step 1: Deploy drones and base stations, calibrate parameters, and establish a coordinate system; Step 2: The base station and the UAV cooperate to acquire images and transmit them to the processing unit. The acquired images include first image information and second image information. The second image information is adaptively encoded and transmitted. Step 3: The processing unit constructs a digital twin model and executes the assembly process reasoning; Step 4: The processing unit monitors the construction and assembly deviations in real time and sends out early warnings when abnormalities occur.

[0006] Furthermore, deploying drones and base stations, calibrating parameters, and establishing a coordinate system specifically includes the following steps: Step 101: The processing unit sets up the operating area and the no-fly zone. The plane range of the operating area includes the diameter range of the bridge segment construction area of ​​20 meters to 50 meters, and the height range of the operating area includes the ground height range of 10 meters to 100 meters. The area within the diameter range of the bridge segment construction area of ​​20 meters and the area below the ground height of 10 meters are no-fly zones. Several base stations are set up on the ground within the plane range of the UAV operating area, and the number of base stations is not less than 3. Step 102: Power on the base station, drone, and processing unit, and check that the communication between the base station, drone, and processing unit is normal and there are no hardware faults; Step 103: Establish a coordinate system with the bridge splicing direction as the x-axis, the bridge large segment splicing surface as the y-axis, and the bridge elevation direction as the z-axis. The unit of the coordinate system is meters. Step 104: Set up several targets in the bridge construction area, with no fewer than 6 targets.

[0007] Furthermore, the collaborative image acquisition and transmission to the processing unit by the base station and the drone includes a first image information transmission, a second image information transmission, and a drone positioning process. The drone positioning process includes the following steps: Step 201: The processing unit sends flight control commands to the Ethernet module of the base station. The Ethernet module forwards the flight control commands to the wireless module. The wireless module sends the flight control commands to the image transmission module of the UAV wirelessly. The image transmission module transmits the flight control commands to the flight control module. The flight control module controls the UAV to take off and wait in the work area according to the flight control commands. Step 202: The positioning module transmits the positioning information to the flight control module. The second image acquisition module acquires the second image information of the area where the UAV is located and transmits it to the encoding module. The encoding module compresses the second image information according to the highest frame rate and the lowest quality and transmits it to the flight control module. Initially, the main task of the UAV is to determine its own positioning, so high-definition second image information is not required. Only a high frame rate is needed to maintain the shortest response time. Step 203: The flight control module transmits the positioning information, signal connection strength Q, and compressed second image information to the processing unit sequentially through the wireless module and the Ethernet module. The signal connection strength Q is the wireless connection strength between the wireless module and the flight control module. The base station is wired to the processing unit through the Ethernet module. The data transmission delay is smaller and the bandwidth is higher. Therefore, the base station acts as a data concentrator or data relay to forward the second image information acquired by the UAV to the processing unit.

[0008] Step 204: The processing unit processes the data according to the formula... Calculate the altitude H of the drone, where, The bridge deck reference elevation is measured by construction personnel and manually input into the processing unit. The processing unit is a computing terminal connected to a large cloud model. f is the lens focal length of the second image acquisition module, L is the actual distance between the targets, which is measured by construction personnel after the targets are deployed and manually input into the processing unit. d1 is the pixel size between the targets in the second image information. It should be noted that the encoding module only compresses the imaging quality of the second image information, while the size and number of pixels of the second image information remain unchanged, which is convenient for analyzing and calculating the height H of the UAV. Step 205: The processing unit calculates the predicted position of the UAV based on the signal connection strength; Step 206: The processing unit connects the predicted position with the positioning information and marks the midpoint of the connecting line segment as the horizontal position of the UAV; Step 207: The processing unit inputs the horizontal position and altitude H of the UAV into the coordinate system to obtain the coordinates of the UAV, completes the positioning of the UAV, and then simultaneously executes the first image information transmission and the second image information transmission, that is, simultaneously executes steps 208a and 208b, and the two branches continue to execute simultaneously.

[0009] Furthermore, the processing unit constructs the digital twin model specifically through the following steps: Step 301: The processing unit inputs the first image information and the corresponding base station planar position into the coordinate system, inputs the second image information and the corresponding UAV coordinates into the coordinate system, aligns the first image information and the second image information in the coordinate system and performs pixel fusion to obtain the first model. The fused first model is the fixed view of the base station and the UAV, which has visual blind spots or missing local features. The UAV needs to move its view to collect the second image information again to supplement it. Step 302: The processing unit determines whether there is a missing region in the first model. If there is a missing region, proceed to the next step 303; if there is no missing region, jump to step 305. Step 303: The processing unit marks the missing areas in the first model, generates the shooting position of the drone based on the position of the missing areas, forwards the flight control command to the drone's flight control module through the base station, and the flight control module controls the drone to reach the designated position, and proceeds to the next step 304. Step 304: The second image acquisition module acquires the second image information again and transmits it back to the processing unit. The processing unit inputs the acquired second image information into the coordinate system. The second image information is converted into supplementary features and fused with the first model. Then, proceed to step 302. Step 305: The processing unit marks the first model as a digital twin model.

[0010] Furthermore, the specific steps involved in performing the assembly process reasoning are as follows: Step 306: The processing unit extracts the construction parameters of the bridge from the digital twin model. The construction parameters include, but are not limited to, the dimensions of the main bridge, the attitude data of large segments, the spatial dimensions of large segments, and the splicing surface parameters. Step 307: The processing unit establishes construction requirements, including but not limited to splicing deviation, construction time, collision risk, and resource utilization rate. The construction requirements and digital twin model are uploaded to the cloud-based large model. At the same time, the movable path range and attitude adjustment range of the large segment are calibrated. The minimum comprehensive value of the construction requirements is used as a constraint. The movable path range and attitude adjustment range are referenced from past construction drawings. The construction personnel manually input the data into the processing unit. The cloud-based large model uses an exhaustive method to calculate each splicing path and attitude adjustment scheme. The cloud-based large model adopts a general large model inference mode to calculate the splicing deviation, construction time, and collision risk coefficient corresponding to each scheme. After completing the quantitative evaluation of all feasible schemes, the feasible optimal construction process is obtained and sent to the processing unit. Step 308: The processing unit marks the optimal construction process as the assembly process and outputs it. The construction personnel carry out the construction according to the assembly process.

[0011] Furthermore, the real-time monitoring of construction assembly deviations by the processing unit specifically includes the following steps: Step 401: When construction is carried out according to the assembly process, the processing unit marks the area where the bridge large segment splicing is located as the monitoring area. The processing unit sends flight control commands to all drones through the base station. All drones are evenly distributed around the monitoring area. Step 402: The second image acquisition module acquires the second image information and transmits it back to the processing unit through the base station. The processing unit extracts the outline of the large segment splicing surface in the second image information, marks the bends in the outline as docking points, and connects the matching nodes of the two splicing surfaces to obtain a reference line. Step 403: The processing unit calculates the angle between the reference line and the axis where the bridge splicing direction is located. The larger the angle, the greater the docking deviation. The processing unit generates and outputs the corresponding adjustment procedure based on the deflection direction of the angle. The purpose of the adjustment procedure is to reduce the value of the angle. The adjustment procedure is provided to the construction personnel as a reference. It should be noted that the shorter the reference line, the shorter the distance between the two docking surfaces. In order to make the second image information collected by the UAV clearer, it is necessary to reduce the corresponding monitoring area, that is, the smaller the degree of convergence of all UAVs, the shorter the distance between the UAVs. The specific steps for sending alerts when an anomaly occurs include: Step 404: After the adjustment procedure output by the processing unit is implemented by the construction personnel, the processing unit executes step 402 again to obtain the reference line of the splicing surface. If the angle between the extended line of the axis where the bridge splicing direction is located and the extended line of the reference line increases, it indicates that an abnormality has occurred in the output adjustment procedure or when the adjustment procedure is implemented. The processing unit establishes an abnormal event and pushes an early warning to promptly remind the safety officer or construction personnel.

[0012] Furthermore, the second image information undergoes adaptive encoding and transmission. This includes the synchronous change of the drone's coordinates in the coordinate system as the drone moves. The processing unit calculates the drone's movement distance and transmits it in real time to the drone's encoding module via a base station. The encoding module sets the frame rate range and imaging quality range for the second image information. The frame rate range is 30Hz to 120Hz, and the imaging quality range is 25% to 100%. The processing unit sets a first threshold and a second threshold. If the drone's displacement distance is less than the first threshold, the encoding module uses the maximum imaging quality and the minimum frame rate. If the drone's displacement distance is greater than or equal to the first threshold and less than or equal to the second threshold, the encoding module dynamically adjusts the frame rate and imaging quality uniformly according to the displacement value. If the drone's displacement distance is greater than the second threshold, the encoding module uses the minimum imaging quality and the maximum frame rate.

[0013] Furthermore, the first image information transmission includes the following steps: Step 208a: The first image acquisition module acquires the first image information of the area where the base station is located and transmits it to the Ethernet module, then proceeds to the next step 209a; Step 209a: The Ethernet module transmits the first image information and its own network address to the processing unit. The processing unit assigns a different network address to each base station, and the network address is used to distinguish the base stations. The second image information transmission includes the following steps: Step 208b: The second image acquisition module acquires the second image information of the area where the UAV is located and transmits it to the encoding module. The encoding module compresses the second image information and forwards it to the image transmission module. The compression adopts adaptive encoding. The function of the encoding module is to dynamically adjust the compression ratio according to the movement range of the UAV, thereby reducing the transmission pressure of the image transmission module. When the UAV moves significantly, it means that the large-segment splicing of the bridge is in the initial stage. The precision requirement is not high, but the movement range is large. At this time, the timeliness of image transmission is of higher priority. The encoding module compresses the second image information in a way that maintains a high frame rate and low-quality image to keep the digital twin model updated in real time. When the UAV moves slightly, it means that the large-segment splicing of the bridge is in the final stage. The movement range is not large, but the precision requirement is high. At this time, the accuracy of image transmission is of higher priority. The encoding module compresses the second image information in a way that maintains a low frame rate and high-quality image to reduce the splicing deviation of the digital twin model and proceed to the next step 209b. Step 209b: The image transmission module transmits the second image information back to the processing unit sequentially through the wireless module and the Ethernet module.

[0014] Furthermore, the processing unit calculates the predicted position of the UAV based on the signal connection strength, including the processing unit according to the formula... The distance s between each base station and the drone is calculated, where d2 is the reference distance, n is the path loss factor (the specific value is determined based on the wireless transmission environment of the drone), q is the wireless signal connection strength between the flight control module and the wireless module at the reference distance, and X is the Gaussian white noise correction term. The processing unit draws a circle with the plane position of each base station as the center and the calculated distance s as the radius, and marks all overlapping parts of the circles as the predicted positions. The plane position of each base station is measured by the construction personnel and manually input into the processing unit.

[0015] Furthermore, the output terminal of the first image acquisition module is electrically connected to the input terminal of the Ethernet module, the port of the Ethernet module establishes bidirectional communication with the ports of the processing unit and the wireless module respectively, the port of the wireless module establishes bidirectional communication with the port of the image transmission module, the output terminal of the image transmission module is electrically connected to the input terminal of the flight control module, the output terminal of the positioning module is electrically connected to the input terminal of the image transmission module, the port of the image transmission module establishes bidirectional communication with the port of the encoding module, and the output terminal of the second image acquisition module is electrically connected to the input terminal of the encoding module.

[0016] The present invention has the following beneficial effects: 1. By coordinating image information acquisition with fixed ground base stations and mobile drones, and dynamically adjusting the image compression ratio based on the drone's movement range, this invention achieves adaptive matching of image transmission strategies at different assembly stages. Compared with existing fixed-position acquisition and fixed-compression-ratio transmission technologies, this invention fills in the blind spots of the acquisition viewpoint, reduces the pressure of image transmission, and simultaneously ensures the real-time performance and stitching accuracy of the digital twin model.

[0017] 2. By acquiring planar position information through the UAV positioning module and combining it with machine vision analysis of image information to obtain altitude position information, a digital twin model is constructed based on the UAV's spatial position calculation. Compared with existing modeling technologies using single devices or single measurement methods, this invention improves spatial positioning accuracy and eliminates problems such as model splicing misalignment and data distortion.

[0018] 3. By inputting parameters through a digital twin model, the optimal assembly and construction process is inferred using an exhaustive method or a general large model. Combined with real-time image data, splicing deviations and anomaly warnings are calculated, realizing intelligent control of the entire assembly and construction process. Compared with existing technologies that rely on human experience to formulate construction processes, this invention improves the global optimality of the assembly scheme and reduces the accumulation of assembly deviations.

[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a system block diagram of the drone, base station, and processing unit of the present invention; Figure 2 This is a flowchart illustrating the adaptive intelligent assembly construction method for large bridge segments based on machine vision, as described in this invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figure 1-2This invention provides a technical solution: an adaptive intelligent assembly construction method for large bridge segments based on machine vision, such as... Figure 2 As shown, this is implemented based on a drone, a base station, and a processing unit. The base station includes a first image acquisition module, an Ethernet module, and a wireless module. The drone includes a flight control module, a positioning module, an image transmission module, an encoding module, and a second image acquisition module. The process includes the following steps: Step 1: Deploy drones and base stations, calibrate parameters, and establish a coordinate system; Step 2: The base station and the drone work together to acquire images and transmit them to the processing unit. The acquired images include first image information and second image information. The second image information is adaptively encoded and transmitted. Step 3: The processing unit constructs a digital twin model and executes the assembly process reasoning; Step 4: The processing unit monitors the construction and assembly deviations in real time and sends out early warnings when abnormalities occur.

[0024] The process of deploying drones and base stations, calibrating parameters, and establishing a coordinate system includes the following steps: Step 101: The processing unit sets up the work area and no-fly zone. The planar range of the work area includes the diameter range of the bridge segment construction area, which is 20 to 50 meters. If the planar range is too close, it will affect the bridge splicing construction. If the planar range is too far, it will be difficult for the second image acquisition module to obtain the second image information. The height range of the work area includes the ground height, which is 10 to 100 meters. If the height is too low, the drone is likely to collide with the construction equipment. Generally, the ground bridge structure will not be higher than 50 meters. Limiting the height of the work area to below 100 meters can meet the image acquisition requirements. The diameter range of the bridge segment construction area within 20 meters and the ground height below 10 meters are no-fly zones. Several base stations are set up on the ground within the planar range of the drone's work area. The number of base stations is not less than 3. If there are fewer than 3 base stations, the drone cannot use the signal connection strength to assist in positioning. Step 102: Power on the base station, drone, and processing unit, and check that the communication between the base station, drone, and processing unit is normal and there are no hardware faults; Step 103: Establish a coordinate system with the bridge splicing direction as the x-axis, the bridge large segment splicing surface as the y-axis, and the bridge elevation direction as the z-axis. The unit of the coordinate system is meters. Step 104: Set up several targets in the bridge construction area. The number of targets should not be less than 6. Setting up targets can serve as feature points for the first and second image information after image acquisition. The more targets there are, the smaller the calculated error will be. This will facilitate the extraction of image features and the monitoring of the correction accuracy when assembling deviations are monitored.

[0025] The process of base station and drone collaboratively acquiring and transmitting images to the processing unit includes first image information transmission, second image information transmission, and drone positioning. The drone positioning process includes the following steps: Step 201: The processing unit sends flight control commands to the Ethernet module of the base station. The Ethernet module forwards the flight control commands to the wireless module. The wireless module sends the flight control commands to the image transmission module of the UAV wirelessly. The image transmission module transmits the flight control commands to the flight control module. The flight control module controls the UAV to take off and wait in the work area according to the flight control commands. Step 202: The positioning module transmits the positioning information to the flight control module. The second image acquisition module acquires the second image information of the area where the UAV is located and transmits it to the encoding module. The encoding module compresses the second image information according to the highest frame rate and the lowest quality and transmits it to the flight control module. Initially, the main task of the UAV is to determine its own positioning, so high-definition second image information is not required. Only a high frame rate is needed to maintain the shortest response time. Step 203: The flight control module transmits the positioning information, signal connection strength Q, and compressed second image information to the processing unit sequentially through the wireless module and the Ethernet module. The signal connection strength Q is the wireless connection strength between the wireless module and the flight control module. The base station is wired to the processing unit through the Ethernet module. The data transmission has lower latency and higher bandwidth. Therefore, the base station acts as a data concentrator or data relay to forward the second image information acquired by the UAV to the processing unit. Step 204: The processing unit processes the data according to the formula. Calculate the altitude H of the drone, where, The bridge deck reference elevation is measured by construction personnel and manually input into the processing unit. The processing unit is a computing terminal connected to the cloud-based large model. f is the lens focal length of the second image acquisition module, L is the actual distance between the targets, which is measured by construction personnel after the targets are deployed and manually input into the processing unit. d1 is the pixel size between the targets in the second image information. It should be noted that the encoding module only compresses the imaging quality of the second image information, while the size and number of pixels of the second image information remain unchanged, which is convenient for analyzing and calculating the height H of the UAV. The following is an example of calculating height H: The actual distance L between the targets is 2 meters, and the bridge deck reference elevation is... With a target height of 10 meters, a focal length of 24 millimeters, and a pixel size of 240 pixels, the drone's height H is calculated to be 10.2 meters using the formula. This calculation example is not the optimal result and is only used to explain the formula calculation. Step 205: The processing unit calculates the predicted position of the UAV based on the signal connection strength; Step 206: The processing unit connects the predicted position with the positioning information and marks the midpoint of the connecting line segment as the horizontal position of the UAV; Step 207: The processing unit inputs the horizontal position and altitude H of the UAV into the coordinate system to obtain the coordinates of the UAV, and completes the positioning of the UAV. Subsequently, the first image information transmission and the second image information transmission are executed simultaneously, that is, steps 208a and 208b are executed simultaneously, and the two branches are executed continuously at the same time.

[0026] The processing unit constructs a digital twin model, specifically including the following steps: Step 301: The processing unit inputs the first image information and the corresponding base station planar position into the coordinate system, inputs the second image information and the corresponding UAV coordinates into the coordinate system, aligns the first image information and the second image information in the coordinate system and performs pixel fusion to obtain the first model. The fused first model is the fixed view of the base station and the UAV, which has visual blind spots or missing local features. The UAV needs to move its view to collect the second image information again to supplement it. Step 302: The processing unit determines whether there is a missing region in the first model. If there is a missing region, proceed to the next step 303. If there is no missing region, jump to step 305. Step 303: The processing unit marks the missing areas in the first model, generates the shooting position of the drone based on the location of the missing areas, forwards the flight control command to the drone's flight control module through the base station, and controls the drone to reach the designated position, and proceeds to the next step 304. Step 304: The second image acquisition module acquires the second image information again and transmits it back to the processing unit. The processing unit inputs the acquired second image information into the coordinate system, and the second image information is converted into supplementary features and fused with the first model. Then, proceed to step 302. Step 305: The processing unit marks the first model as a digital twin model.

[0027] The specific steps involved in performing the assembly process reasoning are as follows: Step 306: The processing unit extracts the construction parameters of the bridge from the digital twin model. The construction parameters include, but are not limited to, the dimensions of the main bridge, the attitude data of large segments, the spatial dimensions of large segments, and the splicing surface parameters. Step 307: The processing unit establishes construction requirements, which include, but are not limited to, splicing deviation, construction time, collision risk, and resource utilization rate. The construction requirements and digital twin model are uploaded to the cloud-based large model. At the same time, the movable path range and attitude adjustment range of the large segments are calibrated. The minimum comprehensive value of the construction requirements is used as a constraint. The movable path range and attitude adjustment range are referenced from past construction drawings. The construction personnel manually input the data into the processing unit. The cloud-based large model uses an exhaustive method to calculate each splicing path and attitude adjustment scheme. The cloud-based large model adopts a general large model inference mode to calculate the splicing deviation, construction time, and collision risk coefficient corresponding to each scheme. After completing the quantitative evaluation of all feasible schemes, the feasible optimal construction process is obtained and sent to the processing unit. Step 308: The processing unit marks the optimal construction process as the assembly process and outputs it. The construction personnel carry out the construction according to the assembly process.

[0028] The processing unit's real-time monitoring of construction assembly deviations specifically includes the following steps: Step 401: When construction is carried out according to the assembly process, the processing unit marks the area where the bridge large segment splicing is located as the monitoring area. The processing unit sends flight control commands to all drones through the base station. All drones are evenly distributed around the monitoring area. Step 402: The second image acquisition module acquires the second image information and transmits it back to the processing unit through the base station. The processing unit extracts the outline of the large segment splicing surface in the second image information, marks the bends in the outline as docking points, and connects the matching nodes of the two splicing surfaces to obtain a reference line. Step 403: The processing unit calculates the angle between the reference line and the axis where the bridge splicing direction is located. The larger the angle, the greater the docking deviation. The processing unit generates and outputs the corresponding adjustment procedure based on the deflection direction of the angle. The purpose of the adjustment procedure is to reduce the value of the angle. The adjustment procedure is provided to the construction personnel as a reference. It should be noted that the shorter the reference line, the shorter the distance between the two docking surfaces. In order to make the second image information collected by the UAV clearer, it is necessary to reduce the corresponding monitoring area, that is, the smaller the degree of convergence of all UAVs, the shorter the distance between the UAVs. The specific steps for sending alerts when an anomaly occurs include: Step 404: After the adjustment procedure output by the processing unit is implemented by the construction personnel, the processing unit executes step 402 again to obtain the reference line of the splicing surface. If the angle between the extended line of the axis where the bridge splicing direction is located and the extended line of the reference line increases, it means that an abnormality has occurred in the output adjustment procedure or when the adjustment procedure is implemented. The processing unit establishes an abnormal event and pushes an early warning to promptly remind the safety officer or construction personnel.

[0029] The second image information undergoes adaptive encoding and transmission. This includes the synchronous change of the drone's coordinates in the coordinate system as the drone moves. The processing unit calculates the drone's movement distance and transmits it in real-time to the drone's encoding module via a base station. The encoding module sets the frame rate range and image quality range for the second image information. The frame rate range is 30Hz–120Hz, and the image quality range is 25%–100%. 100% represents uncompressed image quality, and 25% represents compressed image quality to 25% of its original value. The processing unit sets a first threshold of 10 meters and a second threshold of 20 meters. If the drone's displacement distance is less than the first threshold, the encoding module uses the maximum image quality and the minimum frame rate to ensure clear imaging of the second image information. If the drone's displacement distance is greater than or equal to the first threshold and less than or equal to the second threshold, the encoding module dynamically adjusts the frame rate and image quality uniformly according to the displacement value. For example, if the drone's displacement distance is 15 meters, the frame rate is 75Hz and the image quality is 50%. If the drone's displacement distance is greater than the second threshold, the encoding module uses the minimum image quality and the maximum frame rate to ensure timely transmission of the second image information.

[0030] The first image information transmission includes the following steps: Step 208a: The first image acquisition module acquires the first image information of the area where the base station is located and transmits it to the Ethernet module, then proceeds to the next step 209a; Step 209a: The Ethernet module transmits the first image information and its own network address to the processing unit. The processing unit assigns a different network address to each base station, and the network address is used to distinguish the base stations. The second image information transmission includes the following steps: Step 208b: The second image acquisition module acquires the second image information of the area where the UAV is located and transmits it to the encoding module. The encoding module compresses the second image information and forwards it to the image transmission module. The compression adopts adaptive encoding. The role of the encoding module is to dynamically adjust the compression ratio according to the movement range of the UAV, thereby reducing the transmission pressure of the image transmission module. When the UAV moves significantly, it means that the large-segment splicing of the bridge is in the initial stage. The precision requirement is not high, but the movement range is large. At this time, the timeliness of image transmission is of higher priority. The encoding module compresses the second image information in a way that maintains a high frame rate and low quality image to keep the digital twin model updated in real time. When the UAV moves slightly, it means that the large-segment splicing of the bridge is in the final stage. The movement range is not large, but the precision requirement is high. At this time, the accuracy of image transmission is of higher priority. The encoding module compresses the second image information in a way that maintains a low frame rate and high quality image to reduce the splicing deviation of the digital twin model and proceed to the next step 209b. Step 209b: The image transmission module transmits the second image information back to the processing unit sequentially through the wireless module and the Ethernet module.

[0031] The processing unit calculates the predicted position of the UAV based on the signal connection strength, including the processing unit according to the formula... The distance s between each base station and the drone is calculated, where d2 is the reference distance of 1 meter, n is the path loss factor, the specific value is determined according to the wireless transmission environment of the drone, q is the wireless signal connection strength between the flight control module and the wireless module at the reference distance, and X is a Gaussian white noise correction term used to handle random noise in the field environment. The processing unit draws a circle with the plane position of each base station as the center and the distance s as the radius, and marks all overlapping parts of the circles as the predicted position. The plane position of each base station is measured by the construction personnel and manually input into the processing unit. The following is an example of calculating distance s; With a reference distance d2 of 2 meters, a signal strength Q of -75dBm, a reference distance connection strength q of -45dBm, a path loss n of 3, and a Gaussian white noise correction term X of 2dBm, the distance s is approximately 8.57 meters when substituted into the formula. This calculation example is not the best result and is only used to explain the formula calculation.

[0032] Among them, such as Figure 1 As shown, the output of the first image acquisition module is electrically connected to the input of the Ethernet module. The ports of the Ethernet module establish bidirectional communication with the ports of the processing unit and the wireless module, respectively. The ports of the wireless module establish bidirectional communication with the port of the image transmission module. The output of the image transmission module is electrically connected to the input of the flight control module. The output of the positioning module is electrically connected to the input of the image transmission module. The port of the image transmission module establishes bidirectional communication with the port of the encoding module. The output of the second image acquisition module is electrically connected to the input of the encoding module.

[0033] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A bridge large section adaptive intelligent assembly construction method based on machine vision, characterized in that Based on a drone, a base station, and a processing unit, the base station includes a first image acquisition module, an Ethernet module, and a wireless module. The drone includes a flight control module, a positioning module, an image transmission module, an encoding module, and a second image acquisition module. The process includes the following steps: Step 1: Deploy drones and base stations, calibrate parameters, and establish a coordinate system; Step 2: The base station and the UAV cooperate to acquire images and transmit them to the processing unit. The acquired images include first image information and second image information. The second image information is adaptively encoded and transmitted. Step 3: Construct a digital twin model and execute the assembly process reasoning; Step 4: Monitor construction and assembly deviations in real time and send out early warnings when abnormalities occur.

2. The machine vision-based adaptive intelligent assembling construction method for bridge large segments according to claim 1, characterized in that, Deploying drones and base stations, calibrating parameters, and establishing a coordinate system specifically includes the following steps: Step 101: The processing unit sets up the operating area and no-fly zone, and sets up several base stations on the ground within the plane of the drone's operating area; Step 102: Power on the base station, drone, and processing unit, and check that the communication between the base station, drone, and processing unit is normal and there are no hardware faults; Step 103: Establish a coordinate system with the bridge splicing direction as the x-axis, the bridge large segment splicing surface as the y-axis, and the bridge elevation direction as the z-axis; Step 104: Set up several targets within the bridge construction area. 3.The machine vision based adaptive intelligent assembling construction method for bridge large section according to claim 1, characterized in that, The base station and the drone collaborate to acquire and transmit images to the processing unit, which includes a first image information transmission, a second image information transmission, and a drone positioning process. The drone positioning process includes the following steps: Step 201: The processing unit sends flight control commands to the Ethernet module, the Ethernet module forwards the flight control commands to the wireless module, the wireless module sends the flight control commands to the image transmission module, the image transmission module transmits the flight control commands to the flight control module, and the flight control module controls the UAV to take off according to the flight control commands; Step 202: The positioning module transmits the positioning information to the flight control module. The second image acquisition module acquires the second image information of the area where the UAV is located and transmits it to the encoding module. The encoding module compresses the second image information according to the highest frame rate and the lowest quality and transmits it to the flight control module. Step 203: The flight control module transmits the positioning information, signal connection strength Q, and second image information to the processing unit sequentially through the wireless module and the Ethernet module. The signal connection strength Q is the wireless connection strength between the wireless module and the flight control module. Step 204: calculating the height H of the unmanned aerial vehicle according to the formula H = f * L / d1 is the reference elevation of the bridge surface, f is the focal length of the lens of the second image acquisition module, L is the actual distance between the targets, L is manually input into the processing unit by the construction personnel after the target deployment is completed, and d1 is the pixel size between the targets in the second image information; Step 205: Calculate the predicted location of the UAV based on signal connection strength; Step 206: Connect the predicted position with the positioning information, and mark the midpoint of the connecting line segment as the horizontal position of the UAV; Step 207: Input the horizontal position and altitude H of the UAV into the coordinate system to obtain the coordinates of the UAV, complete the positioning of the UAV, and then simultaneously perform the first image information transmission and the second image information transmission.

4. The machine vision-based adaptive intelligent assembly construction method for large bridge segments according to claim 1, characterized in that, The processing unit constructs the digital twin model, specifically including the following steps: Step 301: The processing unit inputs the first image information and the base station planar position into the coordinate system, inputs the second image information and the UAV coordinates into the coordinate system, aligns the first image information and the second image information in the coordinate system and performs pixel fusion to obtain the first model; Step 302: Determine whether there is a missing region in the first model. If there is a missing region, proceed to the next step 303. If there is no missing region, jump to step 305. Step 303: Mark the missing areas in the first model, generate the shooting position of the drone based on the position of the missing areas, forward the flight control command to the drone's flight control module through the base station, and control the drone to reach the designated position, then proceed to the next step 304; Step 304: The second image acquisition module acquires the second image information again and transmits it back to the processing unit. The processing unit inputs the acquired second image information into the coordinate system. The second image information is converted into supplementary features and fused with the first model. Then, proceed to step 302. Step 305: Mark the first model as a digital twin model.

5. The machine vision-based adaptive intelligent assembly construction method for large bridge segments according to claim 1, characterized in that, The specific steps involved in performing the assembly process reasoning are as follows: Step 306: The processing unit extracts the bridge's construction parameters from the digital twin model; Step 307: Establish construction requirements, upload the construction requirements and digital twin model to the cloud big model, and mark the movable path range and attitude adjustment range of the large segment. With the lowest comprehensive value of the construction requirements as the constraint, the cloud big model uses an exhaustive method to calculate each assembly path and attitude adjustment scheme, and after obtaining the feasible best construction process, it is sent to the processing unit. Step 308: Mark the optimal construction process as the assembly process and output it. Carry out the construction according to the assembly process.

6. The machine vision-based adaptive intelligent assembly construction method for large bridge segments according to claim 1, characterized in that, The processing unit performs real-time monitoring of construction assembly deviations, specifically including the following steps: Step 401: When construction is carried out according to the assembly process, the processing unit marks the area where the bridge large segment splicing is located as the monitoring area. The processing unit sends flight control commands to all drones through the base station. All drones are evenly distributed around the monitoring area. Step 402: The second image acquisition module acquires the second image information and transmits it back to the processing unit through the base station. It extracts the outline of the large segment splicing surface in the second image information, marks the bends in the outline as docking points, and connects the matching nodes of the two splicing surfaces to obtain a reference line. Step 403: Calculate the angle between the extended reference line and the axis where the bridge splicing direction is located, and generate and output the corresponding adjustment procedure based on the deflection direction of the angle. The specific steps for sending alerts when an anomaly occurs are as follows: Step 404: After the processing unit outputs the adjustment procedure, the reference line of the splicing surface is obtained by executing step 402 again. If the angle between the extended line of the axis where the bridge splicing direction is located and the extended line of the reference line increases, an abnormal event is established and an early warning is pushed.

7. The machine vision-based adaptive intelligent assembly construction method for large bridge segments according to claim 1, characterized in that, The second image information is adaptively encoded and transmitted. This includes the synchronous change of the drone's coordinates in the coordinate system when the drone moves. The processing unit calculates the drone's moving distance and transmits it to the drone's encoding module in real time through the base station. The encoding module sets the frame number range and imaging quality range of the second image information, and sets a first threshold and a second threshold. If the drone's displacement distance is less than the first threshold, the encoding module uses the maximum imaging quality and the minimum frame number. If the drone's displacement distance is greater than or equal to the first threshold and less than or equal to the second threshold, the encoding module dynamically adjusts the frame number and imaging quality uniformly according to the displacement value. If the drone's displacement distance is greater than the second threshold, the encoding module uses the minimum imaging quality and the maximum frame number.

8. The machine vision-based adaptive intelligent assembly construction method for large bridge segments according to claim 3, characterized in that, The first image information transmission includes the following steps: Step 208a: The first image acquisition module acquires the first image information of the area where the base station is located and transmits it to the Ethernet module, then proceeds to the next step 209a; Step 209a: The Ethernet module transmits the first image information and its own network address to the processing unit; The second image information transmission includes the following steps: Step 208b: The second image acquisition module acquires the second image information of the area where the UAV is located and transmits it to the encoding module. The encoding module compresses the second image information and forwards it to the image transmission module. The compression adopts adaptive encoding. Then, proceed to the next step 209b. Step 209b: The image transmission module transmits the second image information back to the processing unit sequentially through the wireless module and the Ethernet module.

9. The machine vision-based adaptive intelligent assembly construction method for large bridge segments according to claim 3, characterized in that, The processing unit calculates the predicted position of the UAV based on the signal connection strength, including the processing unit according to the formula... The distance s between each base station and the drone is calculated, where d2 is the reference distance, n is the path loss factor, q is the wireless signal connection strength between the flight control module and the wireless module at the reference distance, and X is the Gaussian white noise correction term. Based on the planar position of each base station as the center and the calculated distance s as the radius, all overlapping parts of the circles are marked as the predicted positions.

10. The machine vision-based adaptive intelligent assembly construction method for large bridge segments according to claim 1, characterized in that, The output of the first image acquisition module is electrically connected to the input of the Ethernet module. The ports of the Ethernet module establish bidirectional communication with the ports of the processing unit and the wireless module, respectively. The ports of the wireless module establish bidirectional communication with the port of the image transmission module. The output of the image transmission module is electrically connected to the input of the flight control module. The output of the positioning module is electrically connected to the input of the image transmission module. The port of the image transmission module establishes bidirectional communication with the port of the encoding module. The output of the second image acquisition module is electrically connected to the input of the encoding module.

Images