A real-time assessment method for pedestrian danger in urban floods based on video images

Abstract

The present invention discloses a method for real-time assessment of pedestrian danger in urban floods based on video images. The main idea of ​​the method is: construct a pedestrian and head detection model based on deep learning, detect pedestrians and heads from images in real time, and determine the pedestrian category; assume the height, head height and weight of pedestrians according to the pedestrian category, use the information such as the height, head height and the submerged part of pedestrians to estimate the real-time water depth at the pedestrian, use the optical flow method to calculate the optical flow size of all pixel positions in the pedestrian boundary box, and convert the average optical flow size into the real-time flow velocity at the pedestrian; according to the height, weight and real-time water depth at the pedestrian, use the starting flow velocity formula when the human body is unstable to calculate the starting flow velocity, and use the ratio of the real-time flow velocity at the pedestrian to the starting flow velocity as the pedestrian danger value, so as to realize the real-time assessment of pedestrian danger. The present invention can provide support for urban flood disaster relief and emergency rescue, and help improve flood response capabilities and emergency rescue levels.

Description

Technical Field

[0001] The present invention belongs to the technical field of video image processing, relates to urban flood disaster prevention and reduction, and specifically relates to a real-time evaluation method for pedestrian danger in urban floods based on video images. Background Art

[0002] Severe urban floods have a great impact on pedestrian travel. Pedestrians trapped in flood water may slip or fall, resulting in casualties. Pedestrian risk assessment in floods refers to the quantitative characterization of the possibility of pedestrians slipping or falling due to flood water.

[0003] The existing methods for assessing pedestrian hazards in floods mainly use the designed precipitation data to drive the hydrodynamic model to simulate water flow elements such as flood depth and flow velocity, and assume that there are pedestrians at all locations in the flood area. The pedestrian hazards at all locations in the flood area are calculated using water flow elements or a combination of water flow elements and human characteristics to obtain a spatial distribution map of pedestrian hazards, thereby providing decision support for the formulation of urban drainage planning and flood prevention plans. This method is only suitable for assessing pedestrian hazards in simulated flood events under specific designed precipitation conditions, but not for real-time assessment of pedestrian hazards in flood events that actually occur on site.

[0004] Real-time assessment of pedestrian hazards in urban floods refers to the quantitative calculation of the possibility of slipping or falling of specific pedestrians who are actually in the flood flow based on the real-time water flow elements and human characteristics of the on-site flood event. Real-time assessment of pedestrian hazards in floods helps relevant departments to timely grasp the real-time danger status of pedestrians who are actually in the flood flow, so as to take reasonable response measures, provide decision-making support for urban flood disaster reduction and relief and emergency rescue, improve urban flood response capabilities and emergency rescue levels, and minimize pedestrian casualties and other adverse effects on pedestrians. Therefore, it is of great significance to develop a real-time assessment method for pedestrian hazards in urban floods.

[0005] Urban flood video images contain rich information on water flow elements (such as water depth and flow velocity) and human characteristics (such as height and weight), which provide data conditions for real-time assessment of pedestrian hazards in urban floods. At present, there is still a lack of real-time assessment methods for pedestrian hazards in urban floods based on video images, which greatly limits the promotion and application of real-time assessment of pedestrian hazards, and is very unfavorable to reduce and mitigate the harm of urban floods to pedestrians. Summary of the invention

[0006] In view of the shortcomings of the existing methods, the present invention proposes a real-time assessment method for pedestrian danger in urban floods based on video images. By obtaining the water depth, flow velocity, height and weight of the pedestrians in real time from the video images, combined with the starting flow velocity formula when the human body is unstable, real-time assessment of pedestrian danger can be achieved.

[0007] A real-time assessment method for pedestrian danger in urban floods based on video images, the specific steps are as follows:

[0008] Step 1: Get your average body index

[0009] Obtain the average height, head height and weight of adults in the monitored area, as well as the average height, head height and weight of children. Determine the average height range of different body parts of children from the ground in an upright state, as well as the average height range of different body parts of adults from the ground. The body parts include soles to ankles, ankles to knees, knees to hips, hips to shoulders, and shoulders to top of the head.

[0010] Step 2: Build pedestrian and head detection models

[0011] Collect urban flood images containing pedestrians and pedestrian heads, and use the labelme tool to annotate pedestrian bounding boxes, pedestrian categories, and pedestrian head bounding boxes in flood images. The pedestrian categories are adults or children. Convert the annotated data into training data that meets the YOLOv10 format requirements, train the YOLOv10 pre-training model, and obtain the pedestrian and head detection model based on YOLOv10.

[0012] Step 3: Real-time detection of pedestrians and heads

[0013] The pedestrian and head detection model is used to obtain the pedestrian category, pedestrian bounding box and pedestrian head bounding box in the video image in real time, and the corresponding relationship between the pedestrian bounding box and the head bounding box is established.

[0014] Step 4: Assume the pedestrian's height, head height and weight

[0015] According to the pedestrian category, the average height, average head height and average weight obtained in step 1 are used as the height h of the pedestrian in the video image. body 、Head height h head and weight .h body and h head The unit is m, The unit is kg.

[0016] Step 5: Real-time estimation of water depth at pedestrian location

[0017] The pixel height of the pedestrian bounding box detected in step 3 is used as the pixel height of the pedestrian’s body above the water surface. , the pixel height of the head bounding box corresponding to the pedestrian bounding box is taken as the pixel height of the pedestrian head , estimate the pedestrian's body height above the water surface :

[0018] (1)

[0019] in, and The unit is px, The unit is m.

[0020] Calculate the initial value d of the real-time water depth at the pedestrian position:

[0021] (2)

[0022] Wherein, the unit of d is m.

[0023] Determine the part of the pedestrian's body that is submerged by the accumulated water, and correct d according to the average height range of the part to obtain the real-time water depth D at the pedestrian:

[0024] (3)

[0025] Where, the unit of D is m, , They are respectively the lower and upper limits of the average height range of pedestrian body parts submerged by water, in meters.

[0026] Step 6: Real-time estimation of flow velocity at pedestrian locations

[0027] Calculate the average optical flow size of all pixel positions in the pedestrian bounding box area , convert it into flow velocity, and get the real-time flow velocity V at the pedestrian:

[0028] (4)

[0029] The unit of V is m / s. The unit is px / s.

[0030] Step 7: Real-time pedestrian risk assessment

[0031] According to the real-time obtained pedestrian height h body ,weight and the real-time water depth D at the location, and calculate the starting flow velocity when the pedestrian slips and falls , :

[0032] (5)

[0033] (6)

[0034] in, , The unit is m / s.

[0035] Calculate the real-time pedestrian danger value RD in flood scenarios:

[0036] (7)

[0037] The larger the RD value is, the higher the danger level of the pedestrian is.

[0038] The present invention has the following beneficial effects:

[0039] 1. The present invention fully explores and utilizes the real-time water flow and pedestrian information contained in the video images, and combines it with the starting flow rate formula when the human body is unstable, thereby realizing real-time assessment of pedestrian danger in urban floods based on video images.

[0040] 2. The present invention can utilize the video surveillance probes already deployed in the city without the need to deploy other additional monitoring equipment, which greatly reduces the cost and threshold of real-time pedestrian risk assessment, and is beneficial to the large-scale promotion and application of real-time pedestrian risk assessment.

[0041] 3. The present invention helps relevant departments to timely understand the dangerous conditions of pedestrians, provide decision-making support for emergency rescue, etc., and reduce the harm of urban floods to pedestrians. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a method for real-time assessment of pedestrian danger in urban floods based on video images;

[0043] Figure 2 Flow chart for real-time estimation of water depth at pedestrian location;

[0044] Figure 3 Flowchart for real-time pedestrian hazard assessment. DETAILED DESCRIPTION

[0045] The present invention is further explained below with reference to the accompanying drawings:

[0046] like Figure 1 As shown, a real-time assessment method for pedestrian instability risk in urban floods based on video images specifically includes the following steps:

[0047] Step 1: Get your average body index

[0048] Obtain the average height of children from the local city's health and medical departments , Average head height and average weight ; and the average height of adults , Average head height and average weight The units of height and head height data are both in meters, and the units of weight data are both in kilograms.

[0049] The present invention defines the height range of a body part as a numerical interval consisting of the height of the lower position of the body part from the ground and the height of the upper position of the body part from the ground in an upright state. Take the body part of ankle to knee as an example, the ankle is its lower position and the knee is its upper position. Assuming that the height of the ankle of a pedestrian from the ground in an upright state is a1, and the height of the knee from the ground is a2, then the height range of the ankle to the knee of the pedestrian is [a1, a2], a1 is the lower limit of the height range, and a2 is the upper limit of the height range. In the present invention, the body parts include sole to ankle, ankle to knee, knee to buttocks, buttocks to shoulders, and shoulders to the top of the head. By consulting health and medical departments and field measurements, the average height range of different body parts of children from the ground in an upright state and the average height range of different body parts of adults from the ground in an upright state are determined. If the accumulated water submerges a certain body part, the average height range of the body part can be regarded as the water depth range. The units of the lower limit and upper limit of the average height range are both m.

[0050] Step 2: Build pedestrian and head detection models

[0051] Urban flood images containing pedestrians and pedestrian heads are collected, and the labelme tool is used to annotate the pedestrian bounding boxes, pedestrian categories (children or adults), and pedestrian head bounding boxes in the flood images. The annotated data is converted into training data that meets the YOLOv10 format requirements, and the YOLOv10 pre-trained model is trained to obtain the YOLOv10-based pedestrian and head detection model.

[0052] Step 3: Real-time detection of pedestrians and heads

[0053] s3.1. The urban flood video image is input into the pedestrian and head detection model. The model detects the pedestrian bounding box, pedestrian category and pedestrian head bounding box from the image.

[0054] s3.2. Establish the correspondence between the pedestrian bounding box and the head bounding box: For each pedestrian bounding box, if there is only one head bounding box in the pedestrian bounding box, then this head bounding box is used as the head bounding box corresponding to the pedestrian bounding box; if there are multiple head bounding boxes in the pedestrian bounding box, then the Euclidean distance between the top frame center point of each head bounding box and the top frame center point of the pedestrian bounding box is calculated, and the head bounding box corresponding to the minimum Euclidean distance is used as the head bounding box corresponding to the pedestrian bounding box.

[0055] Step 4: Assume the pedestrian's height, head height and weight

[0056] According to the detected pedestrian category, the average height, average head height and average weight obtained in step 1 are used as the height h of the pedestrian in the video image. body 、Head height h head and weight :

[0057] (1)

[0058] (2)

[0059] (3)

[0060] Among them, h body and h head The unit is m, The unit is kg.

[0061] Step 5: Real-time estimation of water depth at pedestrian location

[0062] The process of real-time estimation of water depth at pedestrians is as follows: Figure 2 shown.

[0063] s5.1. Use the y-axis coordinate of the lower right corner of the pedestrian bounding box minus the y-axis coordinate of the upper left corner to get the pixel height of the pedestrian bounding box, which is used as the pixel height of the pedestrian's body above the water surface. ;According to the correspondence between the pedestrian bounding box and the head bounding box, obtain the head bounding box corresponding to the pedestrian bounding box, and use the y-axis coordinate of the lower right corner of the head bounding box minus the y-axis coordinate of the upper left corner to obtain the pixel height of the head bounding box as the pixel height of the pedestrian head . and The unit is px.

[0064] s5.2, based on the pixel height of the pedestrian’s body above the water surface and the pedestrian's head height h head Height in pixels from the head The ratio of the pedestrian's body height above the water surface is estimated. :

[0065] (4)

[0066] in, The unit is m.

[0067] s5.3, according to the height of the pedestrian h body Height of pedestrian’s body above water , calculate the initial value d of the water depth at the pedestrian:

[0068] (5)

[0069] Wherein, the unit of d is m.

[0070] s5.4. Use the method recorded in the prior art CN115880627B to determine the part of the pedestrian's body that is submerged by the water, and use the average height range of the body part to correct d to obtain the real-time water depth D at the pedestrian:

[0071] (6)

[0072] Where, the unit of D is m, is the lower limit of the average height range of the body part, The unit is m, is the upper limit of the average height range of the body part, The unit is m.

[0073] Step 6: Real-time estimation of flow velocity at pedestrian locations

[0074] s6.1. Use the Farneback method to calculate the optical flow size of all pixel positions in the pedestrian bounding box area: Get the current frame and the previous frame images in the pedestrian bounding box area, and convert them into 8-bit single-channel images respectively. Then use these two 8-channel images as the input images of the Farneback method to calculate the optical flow vector (v) of each pixel position in the pedestrian bounding box area. x , v y ), where v x is the velocity value in the x-axis direction, v y is the velocity value in the y-axis direction, and the optical flow size v of each pixel position is obtained:

[0075] (7)

[0076] Among them, v x 、v y The units of and v are both px / s.

[0077] s6.2. Calculate the average optical flow size of all pixel positions , convert the average value into the real-time water velocity V at the pedestrian:

[0078] (8)

[0079] The unit of V is m / s. The unit is px / s.

[0080] Step 7: Real-time pedestrian risk assessment

[0081] The process of real-time pedestrian risk assessment is as follows: Figure 3Referring to the starting flow velocity formula and related parameters when a human body slips and falls and becomes unstable in the prior art "Evaluation and Improvement of Existing Human Instability Standards in Floods", the starting flow velocity when a pedestrian slips and falls and becomes unstable is calculated. , :

[0082] (9)

[0083] (10)

[0084] in, , The unit is m / s.

[0085] Calculate real-time pedestrian danger value in flood scenarios:

[0086] (11)

[0087] Among them, RD is the real-time pedestrian danger value in flood scenarios. The larger the RD value, the higher the danger level of pedestrians.

Claims

1. A real-time assessment method for pedestrian danger in urban floods based on video images, characterized by: The specific steps are as follows: Step 1, respectively obtain the average height, average head height and average weight of local children and adults; respectively determine the average height range of different body parts of children and adults in an upright state; Step 2: Build a pedestrian and pedestrian head detection model based on deep learning; Step 3: using the pedestrian and head detection model to obtain the pedestrian category, pedestrian bounding box and pedestrian head bounding box from the video image in real time, wherein the pedestrian category is a child or an adult; Establish the correspondence between the pedestrian bounding box and the head bounding box; Step 4: According to the pedestrian category, the average height, average head height and average weight obtained in step 1 are used as the height h of the pedestrian in the video image. body 、Head height h head and weight ; Step 5: Calculate the pixel height of the pedestrian bounding box And the pixel height of the head bounding box corresponding to the pedestrian bounding box ;according to , and h head Estimate the height of the pedestrian's body above the water , h body and The difference between the values ​​of d and d is used as the initial value d of the real-time water depth at the pedestrian; the part of the pedestrian's body submerged by the accumulated water is determined, and d is corrected using the average height range of the body part determined in step 1 to obtain the real-time water depth D at the pedestrian; Step 6: Calculate the average optical flow size of all pixel positions in the pedestrian bounding box area ,Will Convert it into flow velocity to get the real-time flow velocity V at the pedestrian location; Step 7: According to the pedestrian height h obtained in real time from the video image body ,weight and the real-time water depth D at the pedestrian, and calculate the starting flow velocity when the pedestrian slips and falls , , calculate the real-time pedestrian danger value RD: （1） The larger the RD value is, the higher the danger level of the pedestrian is.

2. A method for real-time assessment of pedestrian risk in urban floods based on video images as claimed in claim 1, characterized in that: The method for constructing a pedestrian and pedestrian head detection model based on deep learning is: collecting urban flood images containing pedestrians and pedestrian heads, using the labelme tool to annotate pedestrian bounding boxes, pedestrian categories and pedestrian head bounding boxes in the flood images, and training the YOLOv10 pre-trained model to obtain the pedestrian and pedestrian head detection model based on deep learning.

3. A method for real-time assessment of pedestrian risk in urban floods based on video images as claimed in claim 1, characterized in that: The method for establishing the correspondence between the pedestrian bounding box and the head bounding box is as follows: for each pedestrian, if there is only one head bounding box in the pedestrian bounding box, then the head bounding box is used as the head bounding box corresponding to the pedestrian bounding box; If there are multiple head bounding boxes in the pedestrian bounding box, the Euclidean distance between the top frame center point of each head bounding box and the top frame center point of the pedestrian bounding box is calculated, and the head bounding box corresponding to the minimum Euclidean distance is used as the head bounding box corresponding to the pedestrian bounding box.

4. The method for real-time assessment of pedestrian danger in urban floods based on video images as claimed in claim 1, characterized in that: Estimated height of the pedestrian above the water surface The method is: （2）。 5. The method for real-time assessment of pedestrian danger in urban floods based on video images as claimed in claim 1, characterized in that: The method for correcting the initial water depth d according to the average height range of the body part to obtain the real-time water depth D of the pedestrian is: （3） in, is the lower limit of the average height range of the body part, The upper limit of the average height range for this body part.

6. The method for real-time assessment of pedestrian danger in urban floods based on video images as claimed in claim 1, characterized in that: The calculation method is to average the optical flow values ​​of all pixels in the pedestrian bounding box area. The method to convert it into the real-time flow velocity V at the pedestrian is: use the Farneback method to calculate the optical flow size of all pixel positions in the pedestrian bounding box area, and calculate the average value of the optical flow size of all pixel positions , and convert it into the real-time flow velocity V at the pedestrian location: （4）。 7. The method for real-time assessment of pedestrian danger in urban floods based on video images as claimed in claim 1, characterized in that: The starting flow velocity when the pedestrian slips and falls , The calculation method is: （5） （6）。 8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images