Bridge, tunnel and culvert disaster monitoring method and system using image and video technology

Abstract

The present invention relates to the field of visual detection technology, and more specifically to a bridge, tunnel, and culvert disaster monitoring method and system utilizing image and video technology. The method comprises obtaining monitoring video of a bridge, tunnel, and culvert to be monitored and determining first consecutive video frames corresponding to the same suspected floating object; analyzing each video frame in the first consecutive video frames to determine the likelihood that the suspected floating object corresponding to the first consecutive video frames is a floating object, and screening out the actual floating objects; obtaining second consecutive video frames corresponding to each actual floating object, and determining the bridge, tunnel, and culvert collision damage index corresponding to each actual floating object based on the difference in the regional position of the actual floating object in the corresponding first consecutive video frames and the second consecutive video frames, thereby performing a bridge, tunnel, and culvert safety assessment. The present invention effectively improves the reliability of bridge, tunnel, and culvert monitoring.

Description

Technical Field

[0001] The present invention relates to the field of visual detection technology, and in particular to a bridge, tunnel and culvert disaster monitoring method and system using image and video technology. Background Art

[0002] Flash flood debris impacting bridges, tunnels, and culverts can cause a variety of hazards, including bridge damage, traffic disruptions, and increased maintenance costs. Therefore, during flash floods, it is often necessary to monitor whether flash flood debris has impacted bridges, tunnels, and culverts. This impact analysis can be used to ensure the safety and proper operation of bridges, tunnels, and culverts.

[0003] In the prior art, image and video technology is typically used to monitor bridges, tunnels, and culverts and their surroundings. Image segmentation is then used to determine the connected domain between floating objects and the bridges, tunnels, and culverts. Whether a collision has occurred is then determined based on whether the connected domains are connected. However, due to changes in the optical properties of water (such as light refraction and scattering), the water appears different in color than the surrounding water in the monitoring image, making it easy to misidentify it as a floating object during image segmentation. Furthermore, when floating objects are close to the bridges, tunnels, and culverts, even if there is no contact or collision, the connected domain between the floating objects and the bridges, tunnels, and culverts obtained through image segmentation may be connected, resulting in inaccurate results in identifying whether flash flood floating objects have impacted the bridges, tunnels, and culverts. Furthermore, because the location where floating objects impact the bridges, tunnels, and culverts is not fixed, the damage caused by flash flood floating objects to the bridges, tunnels, and culverts can often only be assessed based on the number of identified impacts, failing to accurately identify the safety of the bridges, tunnels, and culverts. Due to these factors, the reliability of existing bridge, tunnel, and culvert disaster monitoring is relatively poor. Summary of the Invention

[0004] The purpose of the present invention is to provide a bridge, tunnel and culvert disaster monitoring method and system using image and video technology to solve the problem of poor reliability of existing bridge, tunnel and culvert disaster monitoring.

[0005] To solve the above technical problems, in a first aspect, the present invention provides a method for monitoring bridge, tunnel and culvert disasters using image and video technology, comprising the following steps:

[0006] Obtain a monitoring video of the bridge, tunnel, and culvert to be monitored, and perform bridge, tunnel, culvert, water area, and floating object identification on each video frame in the monitoring video to obtain the bridge, tunnel, culvert area and water area in each video frame in the monitoring video, as well as first continuous video frames corresponding to the same suspected floating object, wherein the first continuous video frames are composed of a first video frame, a second video frame, and a video frame between the first and second video frames in the monitoring video, the first video frame being a video frame in which the area corresponding to the suspected floating object is first identified, and the second video frame being a video frame in which the area corresponding to the suspected floating object is first identified to be in contact with the bridge, tunnel, and culvert area;

[0007] Determining the likelihood that the same suspected floating object corresponding to each of the first continuous video frames is a floating object based on a grayscale difference between an area corresponding to the suspected floating object in each of the first continuous video frames and the water area, as well as positional features and regional features of the area corresponding to the suspected floating object in the first continuous video frames, and screening out the real floating object based on the likelihood of being a floating object;

[0008] Obtaining a second continuous video frame corresponding to each real floating object, where the second continuous video frame is composed of a plurality of continuous video frames following the second video frame corresponding to the real floating object in the monitoring video, and determining a bridge, tunnel, and culvert collision damage index corresponding to each real floating object based on a difference in regional position of each real floating object in the corresponding first continuous video frame and second continuous video frame;

[0009] Based on the bridge, tunnel and culvert collision damage indicators, a safety assessment of the bridge, tunnel and culvert is performed.

[0010] In conjunction with the first aspect above, in some possible implementations, determining the likelihood that the first consecutive video frames correspond to the same suspected floating object is a floating object includes:

[0011] Determining a grayscale difference mean and a grayscale difference discrete value based on an average distribution and a discreteness of grayscale differences between the region of the suspected floating object and the water region corresponding to each video frame in the first continuous video frames;

[0012] determining an area discrete value according to a discreteness of the area of ​​the region of the suspected floating object corresponding to each video frame in the first continuous video frames;

[0013] determining a degree of displacement uniformity based on positional differences of regions corresponding to suspected floating objects in respective video frames in the first continuous video frames;

[0014] The possibility that the same suspected floating object corresponding to the first continuous video frames is a floating object is determined based on the grayscale difference mean, grayscale difference discrete value, area discrete value and displacement uniformity. The grayscale difference mean is positively correlated with the possibility of being a floating object, and the grayscale difference discrete value, area discrete value and displacement uniformity are negatively correlated with the possibility of being a floating object.

[0015] In conjunction with the first aspect above, in some possible implementations, the step of determining the grayscale difference between the area of ​​the suspected floating object corresponding to each video frame in the first continuous video frames and the water area includes:

[0016] Determining the grayscale mean of all pixels in the area of ​​each video frame corresponding to the suspected floating object in the first continuous video frames to obtain a first grayscale mean;

[0017] Determine the grayscale mean of all pixels in the water area in all video frames in the first continuous video frames to obtain a second grayscale mean;

[0018] Determine the absolute value of the difference between the first grayscale mean value corresponding to the suspected floating object and the second grayscale mean value corresponding to each video frame in the first continuous video frames, and obtain the grayscale difference between the area corresponding to the suspected floating object in each video frame in the first continuous video frames and the water area.

[0019] In conjunction with the first aspect above, in some possible implementations, determining the displacement uniformity includes:

[0020] determining a centroid position of an area corresponding to the suspected floating object in each video frame of the first continuous video frames;

[0021] Mapping all the centroid positions to the same image, and connecting the centroid positions of the areas corresponding to the suspected floating objects in the first and last two video frames of the first continuous video frames in the image to obtain a centroid position line;

[0022] determining, in the image, a distance between a centroid position of an area of ​​the suspected floating object corresponding to each video frame except the first and last two video frames in the first continuous video frames and a line connecting the centroid positions;

[0023] According to the average distribution of all the distances, a distance mean is obtained, and the distance mean is used as the displacement uniformity.

[0024] In conjunction with the first aspect above, in some possible implementations, screening out real floating objects based on the likelihood of the objects being floating objects includes:

[0025] It is determined whether the possibility of the floating object is greater than a set floating object possibility threshold, and the suspected floating object corresponding to the possibility of the floating object greater than or equal to the set floating object possibility threshold is regarded as a real floating object.

[0026] In conjunction with the first aspect above, in some possible implementations, determining the bridge, tunnel, and culvert collision damage index corresponding to each real floating object includes:

[0027] Determining the bridge, tunnel, and culvert collision possibility corresponding to each real floating object based on a difference in the regional position of each real floating object in the corresponding first continuous video frame and second continuous video frame;

[0028] The bridge, tunnel and culvert collision damage index corresponding to each real floating object is determined according to the bridge, tunnel and culvert collision possibility and the area change of each real floating object in the corresponding first continuous video frame and second continuous video frame.

[0029] In conjunction with the first aspect above, in some possible implementations, determining the bridge, tunnel, and culvert collision probability corresponding to each real floating object includes:

[0030] determining a moving direction of the first floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the first continuous video frames, and determining a speed of the first floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the first continuous video frames and a time interval between video frames in the monitoring video;

[0031] determining a centroid position of an area corresponding to a real floating object in each video frame of the second continuous video frames;

[0032] determining a moving direction of the second floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the second continuous video frames, and determining a speed of the second floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the second continuous video frames and a time interval between video frames in the monitoring video;

[0033] The bridge-tunnel-culvert collision probability corresponding to each of the real floating objects is determined based on the directional difference between the first floating object's movement direction and the second floating object's movement direction, and the speed difference between the first floating object's speed and the second floating object's speed, wherein both the directional difference and the speed difference are positively correlated with the bridge-tunnel-culvert collision probability.

[0034] In conjunction with the first aspect above, in some possible implementations, determining the bridge, tunnel, and culvert collision damage index corresponding to each real floating object includes:

[0035] Analyzing the regional position difference of the real floating object in the second video frame and the adjacent video frames in the first continuous video frames corresponding to each real floating object to determine a first instantaneous motion direction and a first instantaneous motion speed;

[0036] Analyzing the regional position difference of the real floating object in the second video frame of the first continuous video frames and the first set number of video frames of the second continuous video frames corresponding to each real floating object to determine a second instantaneous motion direction and a second instantaneous motion speed;

[0037] determining a collision parameter corresponding to each of the real floating objects according to a direction difference between the first instantaneous motion direction and the second instantaneous motion direction, and a speed difference between the first instantaneous motion speed and the second instantaneous motion speed;

[0038] The bridge-tunnel-culvert collision damage index corresponding to each real floating object is determined according to the bridge-tunnel-culvert collision possibility and collision parameters, and the bridge-tunnel-culvert collision possibility and collision parameters are both positively correlated with the bridge-tunnel-culvert collision damage index.

[0039] In conjunction with the first aspect above, in some possible implementations, performing a bridge, tunnel, or culvert safety assessment based on the bridge, tunnel, or culvert collision damage indicator includes:

[0040] determining whether the bridge-tunnel-culvert collision damage index is greater than or equal to a set bridge-tunnel-culvert collision damage index threshold; if the bridge-tunnel-culvert collision damage index is greater than or equal to the set bridge-tunnel-culvert collision damage index threshold, updating a current bridge-tunnel-culvert safety factor according to the bridge-tunnel-culvert collision damage index;

[0041] If the updated bridge, tunnel or culvert safety factor is less than the set safety factor threshold, a bridge, tunnel or culvert safety warning will be issued.

[0042] In order to solve the above technical problems, in the second aspect, the present invention also provides a bridge, tunnel and culvert disaster monitoring system using image and video technology, including a memory, a processor and an executable computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it executes the method in the above first aspect or any possible implementation of the first aspect.

[0043] In order to solve the above technical problems, in a third aspect, the present invention also provides a computer program product, which includes: computer program code, when the computer program code is run on a computer, enables the computer to execute the method in the above first aspect or any possible implementation of the first aspect.

[0044] In order to solve the above technical problems, in the fourth aspect, the present invention also provides a computer-readable storage medium, which stores computer program code. When the computer program code is run on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0045] The present invention has the following beneficial effects: by identifying the first continuous video frames corresponding to the same suspected floating object, and analyzing the grayscale difference between the area corresponding to the suspected floating object and the water area in each video frame of the first continuous video frames, as well as the positional features and regional features of the area corresponding to the suspected floating object in the first continuous video frames, the present invention determines the possibility that the suspected floating object is a floating object, thereby identifying the real floating object to avoid the misidentification phenomenon caused by image segmentation. At the same time, after the area of ​​the real floating object contacts the bridge, tunnel and culvert area, several continuous video frames are continuously acquired, and the difference in the regional position of the real floating object in the corresponding first continuous video frame and the second continuous video frame is analyzed to evaluate whether the real floating object and the bridge, tunnel and culvert have collided, and to determine the degree of damage caused by the collision to the bridge, tunnel and culvert, thereby obtaining a bridge, tunnel and culvert collision damage index corresponding to each real floating object, and making a bridge, tunnel and culvert safety judgment based on the bridge, tunnel and culvert collision damage index, effectively improving the reliability of bridge, tunnel and culvert disaster monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a flow chart of a bridge, tunnel and culvert disaster monitoring method using image and video technology according to an embodiment of the present invention;

[0048] Figure 2 The present invention is a schematic structural diagram of a bridge, tunnel and culvert disaster monitoring system using image and video technology. DETAILED DESCRIPTION

[0049] In order to clearly illustrate the technical features of this solution, the present invention is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0050] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0051] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0052] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0053] It should be noted that the concepts of "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0054] Although operations or steps are described in a particular order in the drawings in the embodiments of the present invention, this should not be understood as requiring that these operations or steps be performed in the particular order shown or in a serial order, or that all of the operations or steps shown be performed to obtain a desired result. In the embodiments of the present invention, these operations or steps may be performed serially; they may also be performed in parallel; or a portion of these operations or steps may be performed.

[0055] At the same time, it is understood that the data involved in the technical solutions of the present invention (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions. Unless otherwise defined, all technical and scientific terms used in this invention have the same meanings as those commonly understood by those skilled in the art to which this invention belongs, and all parameters or indicators in the formulas involved in this invention are normalized values ​​to eliminate the influence of dimensions.

[0056] In order to solve the problem that the existing results of identifying whether floating objects caused by mountain torrents collide with bridges, tunnels and culverts are not accurate enough, thereby affecting the accuracy of bridge, tunnel and culvert disaster monitoring, an embodiment of the present invention provides a bridge, tunnel and culvert disaster monitoring method and system using image and video technology. By analyzing the movement trajectory and movement state changes of the floating objects before and after the area of ​​the floating objects in the monitoring video is connected with the bridge, tunnel and culvert area, the real floating objects can be accurately identified, and it can be accurately evaluated whether the real floating objects collide with the bridge, tunnel and culvert, and the degree of damage caused by the collision to the bridge, tunnel and culvert can be determined, so as to obtain the bridge, tunnel and culvert collision damage index corresponding to each real floating object, and perform bridge, tunnel and culvert safety judgment, thereby effectively improving the reliability of bridge, tunnel and culvert disaster monitoring.

[0057] The following will describe in detail a bridge, tunnel and culvert disaster monitoring method and system using image and video technology provided by an embodiment of the present invention with reference to the accompanying drawings.

[0058] Figure 1 FIG. 1 shows a basic flow chart of a bridge, tunnel and culvert disaster monitoring method using image and video technology provided by an embodiment of the present invention. Figure 1 As shown, the method specifically includes the following steps:

[0059] Step S100: Obtain a monitoring video of the bridge, tunnel, and culvert to be monitored, identify the bridge, tunnel, culvert, water area, and floating objects in each video frame in the monitoring video, and obtain the bridge, tunnel, and culvert area and water area in each video frame in the monitoring video, as well as the first continuous video frames corresponding to the same suspected floating object. The first continuous video frames are composed of the first video frame, the second video frame, and the video frames between the first video frame and the second video frame in the monitoring video. The first video frame is the video frame in which the area corresponding to the suspected floating object is first identified, and the second video frame is the video frame in which the area corresponding to the suspected floating object is first identified to be in contact with the bridge, tunnel, and culvert area.

[0060] Water level gauges are installed at appropriate locations on the sides of the bridges, tunnels, and culverts to be monitored to monitor in real time whether the current water level exceeds safety standards. If the water level exceeds safety standards, some torrent debris may not be able to pass under the bridge, tunnel, or culvert, but may instead impact the sides, causing damage. Therefore, a camera is installed at an appropriate location on the side of the bridge, tunnel, or culvert to monitor the side when the water level exceeds safety standards, detecting in real time whether torrent debris could cause damaging impacts.

[0061] Since the camera only activates monitoring when the water level exceeds safety standards, a water level gauge is installed at a suitable location on the side of the bridge, tunnel, or culvert. The water level gauge signal output cable is connected to the relay input. The relay input is connected to a power source, and the output is connected to the camera. When the water level rises to a certain height, it is detected by the water level gauge, which generates a corresponding electrical signal and transmits it to the relay. The relay receives the signal and changes its contact switch state, powering the camera, which then begins monitoring the side of the bridge, tunnel, or culvert.

[0062] It should be noted that after the camera is installed, adjust the camera parameters, shooting angle and position so that the shooting image can cover the entire bridge, tunnel and culvert area and part of the water environment around the bridge, tunnel and culvert area. After the camera parameters, shooting angle and position are adjusted, keep the camera position, shooting angle and various parameters fixed, and mark the bridge, tunnel and culvert area in the shooting video frame in advance to monitor and shoot the bridge, tunnel and culvert.

[0063] When the water level meter detects that the current water level has exceeded the safety standard in real time, it sends an electrical signal to connect the relay to the power supply and the camera, turning on the camera to monitor the bridge, tunnel and culvert conditions. The monitoring process is as follows:

[0064] The monitoring video captured by the camera is continuously acquired, and each video frame in the monitoring video is segmented using the image segmentation method. Combined with the pre-marked bridge, tunnel and culvert area positions, the entire bridge, tunnel and culvert area is regarded as a connected domain; and the water area in each video frame is identified, as well as whether there are any suspected floating objects in the water area.

[0065] Because the grayscale values ​​of water areas in video frames are similar, they appear as a single connected domain after image segmentation. Floating objects are moved by the water's surface and appear in the monitoring video. Floating objects, defined as objects floating on the floodwater (water surface) and displaced by the current, have a significantly different grayscale from the water surface. Therefore, in the grayscale image of the video frame, the floating object area appears to be within the water area, with a grayscale value different from the connected domain of the water area. Therefore, image segmentation can be used to identify the floating object area.

[0066] When a connected domain with a grayscale value different from that of the water area is identified in a video frame of the surveillance video, it is considered a suspected floating object area. Considering that a connected domain with a grayscale value different from that of the water area may not necessarily be a floating object, for example, when the optical properties of the water body (such as refraction and scattering of light) change, the waves stirred up by turbulent water will appear different from the surrounding water color in the surveillance video frame. It will also appear as a connected domain with a grayscale value different from the water area. Therefore, it is not enough to judge whether it is a floating object based on this alone, and further judgment is required.

[0067] Considering the turbulent water flow during flash floods, floating objects are generally large pieces of mud, sand, and rocks washed down from the mountains. Their grayscale values ​​should differ significantly from those of the water area, and their color and shape should remain largely unchanged during the drift. In contrast, the grayscale value difference between waves and the water area is smaller, and the water morphology (appearance) and optical properties of waves change continuously from formation to identification, resulting in significant changes in their grayscale values. Therefore, the shape and grayscale value of the connected domain of suspected floating objects can be used to identify the true floating objects.

[0068] To this end, when an area with suspected floating objects is identified in a certain video frame, the certain video frame is used as the first video frame, and video frames after the first video frame are continuously obtained in the monitoring video until a video frame is identified in which the area with the suspected floating object is in contact with the bridge, tunnel or culvert area. The video frame in which the area with the suspected floating object is identified in contact with the bridge, tunnel or culvert area is used as the second video frame, and the first video frame, the second video frame and the continuous video frames between the first video frame and the second video frame in the monitoring video are used as the first continuous video frames of the suspected floating object. The first continuous video frame refers to the continuous video frames from the identification of the appearance of the suspected floating object to the identification of the suspected floating object in contact with the bridge, tunnel or culvert. It should be understood that if the suspected floating object is not identified in the monitoring video as being in contact with the bridge, tunnel or culvert area, the suspected floating object will not be processed. At the same time, when multiple areas of suspected floating objects appear in the same video frame, the same suspected floating object is tracked and identified. For example, whether it is the same suspected floating object can be determined based on the centroid distance between the areas of suspected floating objects in consecutive video frames, and the areas of suspected floating objects with the closest centroid distance in adjacent video frames are regarded as the same suspected floating object.

[0069] Through the above method, the first continuous video frames corresponding to the same suspected floating object, or the first continuous video frames corresponding to multiple suspected floating objects, can be determined. Below, taking the determination of the first continuous video frames corresponding to the same suspected floating object as an example, the steps for determining whether the suspected floating object is a real floating object that has actually collided with a bridge, tunnel, or culvert, and if so, whether it has collided with the bridge, tunnel, or culvert, and the extent of damage to the bridge, tunnel, or culvert in the event of a collision will be described in detail. When the first continuous video frames corresponding to multiple suspected floating objects can be determined, each suspected floating object can be judged in the same manner.

[0070] Step S200: Determine the possibility that the same suspected floating object corresponding to the first continuous video frames is a floating object based on the grayscale difference between the area corresponding to the suspected floating object in each video frame in the first continuous video frames and the water area, as well as the positional features and area features of the area corresponding to the suspected floating object in the first continuous video frames, and screen out the real floating objects based on the possibility of being a floating object.

[0071] When the first continuous video frames corresponding to the same suspected floating object are identified through the above steps, it is possible to judge whether it is a real floating object based on the shape and grayscale value of the connected domain of the suspected floating object, and the shape and grayscale value of the connected domain of the suspected floating object can be represented by the grayscale value, area and displacement uniformity of the centroid of the region of the suspected floating object in each video frame in the first continuous video frame.

[0072] In an embodiment of the present invention, determining the possibility that the first continuous video frames correspond to the same suspected floating object is a floating object is implemented by the following steps:

[0073] Step S201: determining a grayscale difference mean and a grayscale difference discrete value based on the average distribution and discreteness of grayscale differences between the region of the suspected floating object and the water region in each video frame in the first continuous video frames;

[0074] Step S202: determining an area discrete value according to a discreteness of the area of ​​the region of the suspected floating object corresponding to each video frame in the first continuous video frames;

[0075] Step S203: determining the displacement uniformity based on the positional differences of the regions of the suspected floating objects corresponding to the respective video frames in the first continuous video frames;

[0076] Step S204: Determine the possibility that the same suspected floating object corresponding to the first continuous video frames is a floating object based on the grayscale difference mean, grayscale difference discrete value, area discrete value and displacement uniformity, wherein the grayscale difference mean is positively correlated with the possibility of being a floating object, and the grayscale difference discrete value, area discrete value and displacement uniformity are negatively correlated with the possibility of being a floating object.

[0077] Regarding the above steps, in this embodiment of the present invention, for the first continuous video frames corresponding to the same suspected floating object, assuming that the first continuous video frames include N continuous video frames, taking the first video frame as an example, the following operations are performed:

[0078] Determine the grayscale mean of all pixels in the area of ​​the first video frame corresponding to the suspected floating object, obtain the first grayscale mean, and record it as g1; at the same time, determine the grayscale mean of all pixels in the water area of ​​the first video frame, obtain the average grayscale value, and record it as G1. Determine the area (number of pixels) of the area corresponding to the suspected floating object in the first video frame, and record it as S1. Since the camera position, shooting angle and various parameters are fixed, the first video frame is placed in the first quadrant of the rectangular coordinate system with the lower left corner of the first video frame as the coordinate origin, the length of the lower side of the image as the positive direction of the X axis, and the length of the left side of the image as the positive direction of the Y axis. Then, the centroid coordinates of the area of ​​the suspected floating object in the first video frame can be calculated using the centroid calculation method of irregular images, and recorded as Q1 = (x1, y1), where x1 and y1 are the horizontal and vertical coordinates of the centroid of the area of ​​the suspected floating object, respectively.

[0079] The above operation is performed on N consecutive video frames in the first consecutive video frames corresponding to the same suspected floating object, so that the first grayscale mean, average grayscale value, area, and centroid coordinates of each of the N consecutive video frames can be obtained. The first grayscale mean, average grayscale value, area, and centroid coordinates of the suspected floating object in the nth video frame are respectively recorded as g n , G n 、S n , Q n =(x n ,y n ). Determine the average grayscale value of the water area in N consecutive video frames in the first consecutive video frames corresponding to the same suspected floating object Get the second grayscale mean.

[0080] Since floating objects are mostly large pieces of mud, sand, rocks, etc. washed down from the mountains, the grayscale value difference between them and the water area should be large, and their grayscale values ​​are almost unchanged. Calculate the difference between the grayscale mean of the suspected floating objects and the grayscale value of the water area in N consecutive video frames, that is, calculate the absolute value of the difference between the first grayscale mean and the second grayscale mean of the suspected floating objects in each video frame in N consecutive video frames, and use the absolute value of the difference as the grayscale difference between the area of ​​the suspected floating objects and the water area in each video frame in N consecutive video frames. The grayscale difference between the area of ​​the suspected floating objects and the water area in the nth video frame is recorded as Further calculate the mean and variance of all grayscale differences, thereby obtaining the grayscale difference mean and grayscale difference discrete value, and record them as and

[0081] Since floating objects are generally solid, their shape is difficult to change, their area is almost unchanged, and their displacement with the water flow has a certain regularity. Calculate the variance of the area of ​​the suspected floating objects in N consecutive video frames, and use the variance as the area discrete value, and record it as According to the centroid coordinates of the suspected floating objects in each of the N consecutive video frames, the centroid positions of the areas corresponding to the suspected floating objects in the N consecutive video frames are mapped to the same image, and the centroid positions Q1 and Q2 of the areas corresponding to the suspected floating objects in the first and last two video frames are mapped to the same image. N The centroid position line l is obtained by connecting the two consecutive video frames. The distance from the centroid position of each area corresponding to the suspected floating object in N consecutive video frames, except for the first and last two video frames, to the centroid position line l is calculated. The average value of these distances is further calculated to obtain the mean distance value, which is used as the displacement uniformity and recorded as

[0082] Based on the grayscale difference mean determined above Grayscale difference discrete value Area discrete values and displacement uniformity Determine the probability P of the first continuous video frame corresponding to the same suspected floating object being a floating object. The corresponding calculation formula is:

[0083]

[0084] Among them, sigmoid represents the normalization function; exp represents the exponential function with the natural constant e as the base.

[0085] In the above calculation formula, The larger the value is, the greater the difference in grayscale value between the suspected floating object area and the water area is, and the more likely the suspected floating object is a real floating object. The smaller the value, the more evenly the area of ​​the suspected floating object changes along the route of water flow displacement, and the more likely the suspected floating object is to be a real floating object. and The smaller the value of The closer the value of is to 1, the smaller the variation in the grayscale value and area of ​​the suspected floating object region, and the more likely the suspected floating object is to be a real floating object. The more likely the suspected floating object is to be a real floating object, the greater the value of the probability P of being a floating object.

[0086] The above method determines the probability P of the same suspected floating object corresponding to the first consecutive video frames. This probability P comprehensively represents the likelihood that the suspected floating object is a real floating object. The larger the value, the more likely it is a floating object. A threshold value α = 0.8 is set. The probability is then determined to be greater than or equal to the threshold value α. If the probability P ≥ α, the same suspected floating object corresponding to the first consecutive video frames is determined to be a real floating object.

[0087] Step S300: Obtain a second continuous video frame corresponding to each of the real floating objects, where the second continuous video frames are composed of a number of continuous video frames following the second video frame corresponding to the real floating object in the monitoring video, and determine the bridge, tunnel and culvert collision damage index corresponding to each of the real floating objects based on the difference in the regional position of each of the real floating objects in the corresponding first continuous video frames and the second continuous video frames.

[0088] The above steps filter out areas that are actually floating objects by comparing the contact between the suspected floating object area and the bridge, tunnel, and culvert area, as well as their previous appearance. If a floating object collides with a bridge, tunnel, or culvert, the two areas will be connected (contacted). However, this connection does not necessarily mean that a collision has occurred. If a floating object simply passes by a bridge, tunnel, or culvert, and the two are close in distance, their areas may also appear connected during image segmentation. Therefore, the connection between the floating object and the bridge and culvert area is a necessary but not sufficient condition for a collision. After the two areas are connected, further determination of whether a collision has occurred is necessary.

[0089] Whether a floating object has collided with a bridge or culvert can be determined by the changes in its motion characteristics after the two areas are connected. If no collision occurs, the object's motion state will not change significantly compared to before the connection. However, if a collision occurs, the collision will exert an external force on the floating object, which will significantly change its motion state compared to before the collision. Therefore, the degree of change in the floating object's motion state, such as speed and displacement direction, before and after the two areas are connected can be used to determine whether a collision has occurred. The degree of change in the floating object's motion state can be determined by the change in the position of the floating object's area centroid in consecutive video frames.

[0090] The above steps have obtained the centroid position of the floating object area in the continuous video frames from the time the floating object area appears to the time it connects with the bridge, tunnel and culvert area. After the two areas connect (contact), several continuous video frame images are continuously obtained for a period of time, that is, several continuous video frames are continuously obtained after the second video frame corresponding to the real floating object in the monitoring video. Assuming that a total of M frames are obtained, this embodiment of the present invention sets M=5, and these M continuous video frames are referred to as second continuous video frames. By analyzing the difference in the regional position of each real floating object in the corresponding first continuous video frame and the second continuous video frame, the degree of change in the floating object's motion state is determined, thereby determining the possibility of collision between each real floating object and the bridge, tunnel and culvert, and the degree of damage caused by the floating object to the bridge and culvert in the event of a collision.

[0091] In an embodiment of the present invention, the bridge, tunnel and culvert collision damage index corresponding to each real floating object is determined, and the implementation steps include:

[0092] Step S301: determining the bridge, tunnel and culvert collision possibility corresponding to each real floating object based on the difference in the regional position of each real floating object in the corresponding first continuous video frame and the second continuous video frame;

[0093] Step S302: determining a bridge, tunnel, or culvert collision damage index corresponding to each real floating object based on the bridge, tunnel, or culvert collision possibility and the area change of each real floating object in the corresponding first and second continuous video frames.

[0094] Regarding the above steps, when a floating object collides with a bridge, tunnel, or culvert, the moving direction and speed of the floating object in the corresponding first continuous video frames will be significantly different from the moving direction and speed of the floating object in the corresponding second continuous video frames. This allows the possibility of each floating object colliding with the bridge, tunnel, or culvert to be determined.

[0095] In an embodiment of the present invention, determining the bridge-tunnel-culvert collision possibility corresponding to each real floating object includes the following steps:

[0096] determining a moving direction of the first floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the first continuous video frames, and determining a speed of the first floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the first continuous video frames and a time interval between video frames in the monitoring video;

[0097] determining a centroid position of an area corresponding to a real floating object in each video frame of the second continuous video frames;

[0098] determining a moving direction of the second floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the second continuous video frames, and determining a speed of the second floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the second continuous video frames and a time interval between video frames in the monitoring video;

[0099] The bridge-tunnel-culvert collision probability corresponding to each of the real floating objects is determined based on the directional difference between the first floating object's movement direction and the second floating object's movement direction, and the speed difference between the first floating object's speed and the second floating object's speed, wherein both the directional difference and the speed difference are positively correlated with the bridge-tunnel-culvert collision probability.

[0100] Regarding the above steps, in an embodiment of the present invention, the centroid position and centroid coordinates of the area corresponding to the suspected floating object in each video frame in the first continuous video frames can be obtained according to the same method as described above.

[0101] The speed of a floating object can be represented by the displacement distance per unit time. Therefore, the speed of a floating object can be determined by dividing the distance between the centroid positions of the area corresponding to the floating object in adjacent continuous video frames in the monitoring video by the video frame time interval. The video frame time interval refers to the time interval between two adjacent video frames in the monitoring video. For the area corresponding to the real floating object in N video frames in the first continuous video frames, determine the distance between the centroid coordinates of the area corresponding to the real floating object in each of the two adjacent video frames, and use this distance as the speed of the real floating object in the corresponding time period of the two adjacent video frames. Calculate the average value of all the moving speeds corresponding to the N video frames in the first continuous video frames to obtain the speed of the first floating object. v i In the same way, for the area of ​​the real floating object corresponding to the M video frames in the second continuous video frames, based on the centroid coordinates of the area of ​​the real floating object in each video frame, the average value of all the moving speeds corresponding to the M video frames in the second continuous video frames can be calculated to obtain the second floating object speed v′ i Denotes the i-th moving speed corresponding to the second continuous video frame. Determine the speed difference VC = |V1-V2| between the first floating object speed V1 and the second floating object speed V2. The first floating object speed V1 and the second floating object speed V2 both refer to the magnitude of the velocity and are scalars.

[0102] The direction of movement of the floating object can be determined by the movement trend of the centroid position of the area corresponding to the floating object in the video frames of the monitoring video. Using the least squares method, linear fitting is performed on the centroid coordinates of the area corresponding to the real floating object in N video frames of the first consecutive video frames and the centroid coordinates of the area corresponding to the real floating object in M ​​video frames of the second consecutive video frames, respectively. Fitted lines L1 and L2 are obtained accordingly. The extension direction of fitted line L1 pointing to the direction of movement of the real floating object is used as the first floating object movement direction, and the extension direction of fitted line L2 pointing to the direction of movement of the real floating object is used as the second floating object movement direction. The angle between the first floating object movement direction and the second floating object movement direction is determined, and this angle is used as the directional difference between the first and second floating object movement directions and is recorded as θ.

[0103] Based on the directional difference θ between the first floating object's moving direction and the second floating object's moving direction, and the speed difference VC between the first floating object's speed V1 and the second floating object's speed V2, the bridge-tunnel-culvert collision probability T corresponding to each real floating object can be determined. The corresponding calculation formula is:

[0104]

[0105] Wherein, cosθ represents the cosine value of the directional difference θ between the first floating object's moving direction and the second floating object's moving direction, ∈1 represents a correction parameter for preventing the denominator from being 0. In the embodiment of the present invention, ∈1 is set to 0.001.

[0106] In the above calculation formula, the greater the difference VC between the first floating object's velocity V1 and the second floating object's velocity V2, the greater the change in the floating object's velocity before and after the actual floating object's area connects with the bridge, tunnel, and culvert area. The greater the difference θ between the first and second floating object's motion directions, the greater the change in the floating object's displacement direction before and after the actual floating object's area connects with the bridge, tunnel, and culvert area, and the closer the value of 1+cosθ is to 0. The greater the change in the floating object's velocity and displacement direction, the greater the degree of change in the floating object's motion state, the more likely it is that the floating object has collided with the bridge, tunnel, and culvert, and the corresponding bridge, tunnel, and culvert collision probability T is greater.

[0107] When a floating object collides with a bridge, tunnel, or culvert, it is subjected to the impact force, causing its motion state to change momentarily. The more severe the collision, the more intense the instantaneous change in the floating object's motion, and the greater the damage to the bridge, tunnel, or culvert. Therefore, the degree of change in the floating object's instantaneous motion state can be used to further determine whether a collision has occurred and quantify the severity of the collision, thereby determining a corresponding bridge, tunnel, or culvert collision damage index for each actual floating object.

[0108] In an embodiment of the present invention, the bridge, tunnel and culvert collision damage index corresponding to each real floating object is determined, and the implementation steps include:

[0109] Analyzing the regional position difference of the real floating object in the second video frame and the adjacent video frames in the first continuous video frames corresponding to each real floating object to determine a first instantaneous motion direction and a first instantaneous motion speed;

[0110] Analyzing the regional position difference of the real floating object in the second video frame of the first continuous video frames and the first set number of video frames of the second continuous video frames corresponding to each real floating object to determine a second instantaneous motion direction and a second instantaneous motion speed;

[0111] determining a collision parameter corresponding to each of the real floating objects according to a direction difference between the first instantaneous motion direction and the second instantaneous motion direction, and a speed difference between the first instantaneous motion speed and the second instantaneous motion speed;

[0112] The bridge-tunnel-culvert collision damage index corresponding to each real floating object is determined according to the bridge-tunnel-culvert collision possibility and collision parameters, and the bridge-tunnel-culvert collision possibility and collision parameters are both positively correlated with the bridge-tunnel-culvert collision damage index.

[0113] In the above steps, in an embodiment of the present invention, the change in the instantaneous motion state of the floating object before and after the connection between the two areas is calculated based on the video frame at the moment when each floating object's area connects with the bridge, tunnel, or culvert area, as well as the preceding and following adjacent video frames. In implementation, the instantaneous motion speed of the floating object before the connection between the real floating object's area and the bridge, tunnel, or culvert area can be calculated based on the distance between the centroid coordinates of the real floating object's area in the second video frame and the preceding video frame in the first consecutive video frames corresponding to each real floating object, as well as the video frame time interval. Specifically, the distance between the two centroid coordinates is divided by the video frame time interval to obtain the first instantaneous motion speed V'1. Furthermore, the centroid coordinates of the real floating object's area in the second video frame and the preceding video frame in the first consecutive video frames corresponding to each real floating object are connected to form a line L3. The direction extending from the centroid coordinate of the real floating object's area in the preceding video frame to the centroid coordinate of the real floating object's area in the second video frame is defined as the first instantaneous motion direction. At the same time, based on the distance between the centroid coordinates of the region of the real floating object in the second video frame of the first consecutive video frames and the region of the real floating object in the first video frame of the second consecutive video frames, as well as the video frame time interval, the instantaneous velocity of the floating object after the region of the real floating object connects with the bridge, tunnel, and culvert region can be calculated. Specifically, the distance between the two centroid coordinates is divided by the video frame time interval to obtain the second instantaneous velocity V'2. Furthermore, the centroid coordinates of the region of the real floating object in the second video frame of the first consecutive video frames and the region of the real floating object in the first video frame of the second consecutive video frames corresponding to each real floating object are connected to obtain a line L4. The direction extending from the centroid coordinate of the region of the real floating object in the second video frame to the centroid coordinate of the region of the real floating object in the first video frame of the second consecutive video frames is defined as the second instantaneous direction of motion.

[0114] Determine the speed difference VC′=|V′1-V′2| between the first instantaneous motion speed V′1 and the second instantaneous motion speed V′2. The first instantaneous motion speed V′1 and the second instantaneous motion speed V′2 both refer to the magnitude of the rate and are scalar quantities. At the same time, determine the angle between the first instantaneous motion direction and the second instantaneous motion direction, and use the angle as the direction difference between the first instantaneous motion direction and the second instantaneous motion direction, and record it as τ. Based on the speed difference VC′ between the first instantaneous motion speed V′1 and the second instantaneous motion speed V′2 and the direction difference τ between the first instantaneous motion direction and the second instantaneous motion direction, make a correction to the possibility of collision between floating objects and bridges, tunnels and culverts, and use the corrected value to make collision judgments and quantify the severity of the collision, thereby obtaining the bridge, tunnel and culvert collision damage index TX. The corresponding calculation formula is:

[0115]

[0116] Wherein, cosτ represents the cosine value of the direction difference τ between the first instantaneous motion direction and the second instantaneous motion direction; sigmoid represents a normalization function; T represents the bridge-tunnel-culvert collision probability corresponding to each real floating object; ∈2 represents a correction parameter used to prevent the denominator from being zero. In the embodiment of the present invention, ∈2 is set to 0.001.

[0117] In the above calculation formula, VC′ and τ are the instantaneous velocity change and displacement angle of the floating object before and after the area of ​​the real floating object is connected to the area of ​​the bridge, tunnel and culvert, respectively. The collision parameter is determined based on VC′ and τ. When the values ​​of the two are larger, it means that the collision is more likely to occur and the severity of the collision is greater. Therefore, the collision parameters The bridge-tunnel-culvert collision possibility T is corrected and normalized using the sigmoid function, and finally the bridge-tunnel-culvert collision damage index TX is obtained.

[0118] Step S400: Based on the bridge, tunnel, and culvert collision damage index, a safety assessment of the bridge, tunnel, and culvert is performed.

[0119] Through the above steps, the bridge-tunnel-culvert collision damage index TX corresponding to each real floating object can be determined. A larger TX value indicates a greater likelihood of collision between the floating object and the bridge-tunnel-culvert, and a higher collision severity, i.e., greater damage to the bridge-tunnel-culvert. Therefore, a pre-set bridge-tunnel-culvert collision damage index threshold β = 0.7 is set. A determination is made as to whether the bridge-tunnel-culvert collision damage index TX corresponding to each real floating object is greater than or equal to the pre-set bridge-tunnel-culvert collision damage index threshold β. If the bridge-tunnel-culvert collision damage index TX ≥ β, it is assumed that the floating object has collided with the bridge-tunnel-culvert and caused a certain degree of damage to the bridge-tunnel-culvert.

[0120] If it is ultimately determined that the floating object has collided with the bridge or culvert and caused a certain degree of damage to the bridge or culvert, the bridge or culvert safety factor is updated based on the bridge or culvert collision damage index, and a determination is made as to whether the updated bridge or culvert safety factor is qualified, thereby issuing a bridge or culvert safety warning. During implementation, an initial value can be set for the bridge or culvert safety factor, which is then reduced based on the bridge or culvert collision damage index. A set safety factor threshold γ = 0.5 is also set. When the bridge or culvert safety factor is less than the set safety factor threshold γ, it is deemed unqualified and a bridge or culvert safety warning is issued. Based on the bridge or culvert safety warning, technicians can then plan subsequent repair and reinforcement operations for the bridge or culvert to prevent bridge or culvert disasters.

[0121] In this embodiment of the present invention, the initial value of the bridge-tunnel-culvert safety factor is set to 1. Three bridge-tunnel-culvert collision damage index ranges are set: [0.7, 0.8), [0.8, 0.9], and [0.9, 1]. Three collision safety factor update steps are set for each of the three bridge-tunnel-culvert collision damage index ranges: 1, 2, and 4. In addition, a collision safety factor update parameter k = 0.01 is set. Based on the initial value of the bridge-tunnel-culvert safety factor, whenever a real floating object corresponding to a bridge-tunnel-culvert collision damage index TX ≥ β is detected, the corresponding collision safety factor update step is determined based on the bridge-tunnel-culvert collision damage index range in which the bridge-tunnel-culvert collision damage index TX falls. The product of the collision safety factor update step and k is calculated to obtain a collision safety factor reduction. The collision safety factor reduction is then subtracted from the bridge-tunnel-culvert safety factor to obtain an updated bridge-tunnel-culvert safety factor. When the updated bridge-tunnel-culvert safety factor is less than the set safety factor threshold γ, a safety risk is considered to exist in the bridge-tunnel-culvert, and a bridge-tunnel-culvert safety warning is issued.

[0122] In addition, when a floating object is detected to have collided with a bridge culvert and caused a certain degree of damage to the bridge culvert, the following operations are required:

[0123] 1. Record the time when the collision occurred and record the video frames within the first set time period before the collision. The first set time period can be set by the user, such as video frames within five seconds.

[0124] 2. After the collision occurs, continue to record video frames within a second set time period. The second set time period can be set by the user, such as video frames within five seconds.

[0125] 3. The video frames recorded in the first set time period and the second set time period before and after the collision and the bridge-tunnel-culvert collision damage index TX calculated above are saved and uploaded to relevant personnel. Relevant technical personnel will make subsequent repair and reinforcement work plans for the bridge, tunnel and culvert based on the video frames and calculation results of each collision to prevent the occurrence of bridge, tunnel and culvert disasters.

[0126] Based on the same inventive concept, the embodiment of the present invention also provides a bridge, tunnel and culvert disaster monitoring system using image and video technology, such as Figure 2 As shown, the system includes: a memory 201, a processor 202, and a computer program 203 stored in the memory 201 and running on the processor 202, wherein when the processor 202 executes the computer program 203, the system can execute any of the bridge, tunnel and culvert disaster monitoring methods using image and video technology introduced above.

[0127] In embodiments of the present invention, the system can be divided into functional modules based on the above-described method examples. For example, these modules can correspond to individual functional modules, or two or more functions can be integrated into a single processing module. The integrated modules can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used.

[0128] Based on the same inventive concept, an embodiment of the present invention also provides a computer program product, which includes: computer program code, which, when running on a computer, enables the computer to execute any one of the bridge, tunnel and culvert disaster monitoring methods using image and video technology introduced above.

[0129] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, the computer executes any one of the bridge, tunnel and culvert disaster monitoring methods using image and video technology introduced above.

[0130] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A bridge, tunnel and culvert disaster monitoring method using image and video technology, characterized in that: The following steps are involved: Obtain a monitoring video of the bridge, tunnel, and culvert to be monitored, and perform bridge, tunnel, culvert, water area, and floating object identification on each video frame in the monitoring video to obtain the bridge, tunnel, culvert area and water area in each video frame in the monitoring video, as well as first continuous video frames corresponding to the same suspected floating object, wherein the first continuous video frames are composed of a first video frame, a second video frame, and a video frame between the first and second video frames in the monitoring video, the first video frame being a video frame in which the area corresponding to the suspected floating object is first identified, and the second video frame being a video frame in which the area corresponding to the suspected floating object is first identified to be in contact with the bridge, tunnel, and culvert area; Determining the likelihood that the same suspected floating object corresponding to each of the first continuous video frames is a floating object based on a grayscale difference between an area corresponding to the suspected floating object in each of the first continuous video frames and the water area, as well as positional features and regional features of the area corresponding to the suspected floating object in the first continuous video frames, and screening out the real floating object based on the likelihood of being a floating object; Obtaining a second continuous video frame corresponding to each real floating object, where the second continuous video frame is composed of a plurality of continuous video frames following the second video frame corresponding to the real floating object in the monitoring video, and determining a bridge, tunnel, and culvert collision damage index corresponding to each real floating object based on a difference in regional position of each real floating object in the corresponding first continuous video frame and second continuous video frame; Based on the bridge, tunnel and culvert collision damage indicators, a safety assessment of the bridge, tunnel and culvert is performed.

2. The bridge, tunnel and culvert disaster monitoring method using image and video technology according to claim 1 is characterized in that: Determining the possibility that the same suspected floating object corresponding to the first continuous video frames is a floating object includes: Determining a grayscale difference mean and a grayscale difference discrete value based on an average distribution and a discreteness of grayscale differences between the region of the suspected floating object and the water region corresponding to each video frame in the first continuous video frames; determining an area discrete value according to a discreteness of the area of ​​the region of the suspected floating object corresponding to each video frame in the first continuous video frames; determining a degree of displacement uniformity based on positional differences of regions corresponding to suspected floating objects in respective video frames in the first continuous video frames; The possibility that the same suspected floating object corresponding to the first continuous video frames is a floating object is determined based on the grayscale difference mean, grayscale difference discrete value, area discrete value and displacement uniformity. The grayscale difference mean is positively correlated with the possibility of being a floating object, and the grayscale difference discrete value, area discrete value and displacement uniformity are negatively correlated with the possibility of being a floating object.

3. The bridge, tunnel and culvert disaster monitoring method using image and video technology according to claim 2 is characterized in that: The step of determining the grayscale difference between the area of ​​the suspected floating object and the water area corresponding to each video frame in the first continuous video frames includes: Determining the grayscale mean of all pixels in the area of ​​each video frame corresponding to the suspected floating object in the first continuous video frames to obtain a first grayscale mean; Determine the grayscale mean of all pixels in the water area in all video frames in the first continuous video frames to obtain a second grayscale mean; Determine the absolute value of the difference between the first grayscale mean value corresponding to the suspected floating object and the second grayscale mean value corresponding to each video frame in the first continuous video frames, and obtain the grayscale difference between the area corresponding to the suspected floating object in each video frame in the first continuous video frames and the water area.

4. The bridge, tunnel and culvert disaster monitoring method using image and video technology according to claim 2 is characterized in that: Determine the degree of displacement uniformity, including: determining a centroid position of an area corresponding to the suspected floating object in each video frame of the first continuous video frames; Mapping all the centroid positions to the same image, and connecting the centroid positions of the areas corresponding to the suspected floating objects in the first and last two video frames of the first continuous video frames in the image to obtain a centroid position line; determining, in the image, a distance between a centroid position of an area of ​​the suspected floating object corresponding to each video frame except the first and last two video frames in the first continuous video frames and a line connecting the centroid positions; According to the average distribution of all the distances, a distance mean is obtained, and the distance mean is used as the displacement uniformity.

5. The bridge, tunnel and culvert disaster monitoring method using image and video technology according to claim 2 is characterized in that: According to the possibility of being floating objects, the real floating objects are screened out, including: It is determined whether the possibility of the floating object is greater than a set floating object possibility threshold, and the suspected floating object corresponding to the possibility of the floating object greater than or equal to the set floating object possibility threshold is regarded as a real floating object.

6. The method for monitoring bridge, tunnel and culvert disasters using image and video technology according to claim 4, characterized in that: Determining the bridge, tunnel, and culvert collision damage index corresponding to each of the real floating objects, including: Determining the bridge, tunnel, and culvert collision possibility corresponding to each real floating object based on a difference in the regional position of each real floating object in the corresponding first continuous video frame and second continuous video frame; The bridge, tunnel and culvert collision damage index corresponding to each real floating object is determined according to the bridge, tunnel and culvert collision possibility and the area change of each real floating object in the corresponding first continuous video frame and second continuous video frame.

7. The bridge, tunnel and culvert disaster monitoring method using image and video technology according to claim 6 is characterized in that: Determining the bridge, tunnel, and culvert collision probability corresponding to each of the real floating objects includes: determining a moving direction of the first floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the first continuous video frames, and determining a speed of the first floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the first continuous video frames and a time interval between video frames in the monitoring video; determining a centroid position of an area corresponding to a real floating object in each video frame of the second continuous video frames; determining a moving direction of the second floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the second continuous video frames, and determining a speed of the second floating object based on changes in the centroid positions of regions corresponding to the real floating object in each of the second continuous video frames and a time interval between video frames in the monitoring video; The bridge-tunnel-culvert collision probability corresponding to each of the real floating objects is determined based on the directional difference between the first floating object's movement direction and the second floating object's movement direction, and the speed difference between the first floating object's speed and the second floating object's speed, wherein both the directional difference and the speed difference are positively correlated with the bridge-tunnel-culvert collision probability.

8. The bridge, tunnel and culvert disaster monitoring method using image and video technology according to claim 7 is characterized in that: Determining the bridge, tunnel, and culvert collision damage index corresponding to each of the real floating objects, including: Analyzing the regional position difference of the real floating object in the second video frame and the adjacent video frames in the first continuous video frames corresponding to each real floating object to determine a first instantaneous motion direction and a first instantaneous motion speed; Analyzing the regional position difference of the real floating object in the second video frame of the first continuous video frames and the first set number of video frames of the second continuous video frames corresponding to each real floating object to determine a second instantaneous motion direction and a second instantaneous motion speed; determining a collision parameter corresponding to each of the real floating objects according to a direction difference between the first instantaneous motion direction and the second instantaneous motion direction, and a speed difference between the first instantaneous motion speed and the second instantaneous motion speed; The bridge-tunnel-culvert collision damage index corresponding to each real floating object is determined according to the bridge-tunnel-culvert collision possibility and collision parameters, and the bridge-tunnel-culvert collision possibility and collision parameters are both positively correlated with the bridge-tunnel-culvert collision damage index.

9. The method for monitoring bridge, tunnel and culvert disasters using image and video technology according to claim 8, characterized in that: Based on the bridge, tunnel and culvert collision damage indicators, a safety assessment of the bridge, tunnel and culvert is performed, including: determining whether the bridge-tunnel-culvert collision damage index is greater than or equal to a set bridge-tunnel-culvert collision damage index threshold; if the bridge-tunnel-culvert collision damage index is greater than or equal to the set bridge-tunnel-culvert collision damage index threshold, updating a current bridge-tunnel-culvert safety factor according to the bridge-tunnel-culvert collision damage index; If the updated bridge, tunnel or culvert safety factor is less than the set safety factor threshold, a bridge, tunnel or culvert safety warning will be issued.

10. A bridge, tunnel and culvert disaster monitoring system using image and video technology, characterized in that: The method comprises a memory, a processor and an executable computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the method for monitoring bridge, tunnel and culvert disasters using image and video technology as described in any one of claims 1 to 9 is executed.

Images