A street quality evaluation method based on behavior recognition, tracking and analysis

Abstract

The application discloses a street quality evaluation method based on behavior recognition, tracking and analysis, which comprises the following steps: using target recognition and target tracking technology in computer vision technology to extract the behavior trajectories of various crowds from high-altitude shot street videos, and performing three-dimensional coordinate conversion to real distance, so that sub-meter dynamic behavior measurement of the high-altitude video is realized; through analysis of the whole video, the moving speed, moving direction and crowd size of each pedestrian are analyzed, and the behavior type of the individual is comprehensively judged; through comprehensive operation of the behavior database combined with position information, the environment use quality evaluation accurate to a street segment is obtained, and the recognition accuracy of the method can reach decimeter level. Moreover, the time investment of designers in site investigation and analysis in the initial stage of a project is significantly reduced, and the work efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of urban design and urban renewal, and specifically relates to a street quality evaluation method based on behavior recognition, tracking and analysis. Background Technology

[0002] As China's urbanization process shifts from incremental to stock-based development, urban renewal has entered a phase of "stock enhancement," with improving the quality of street spaces being a crucial element. Creating better public spaces for everyone is a persistent and core issue in public space planning and design practice, but its quality depends on how people use these spaces. Therefore, behavioral observation is a key method for measuring the quality of public spaces.

[0003] Previously, urban researchers typically assessed public spaces by calculating behavioral indicators through observational records and / or interviews. These methods have been applied to assist in redevelopment design and to evaluate the vitality, safety, and walkability of such designs. However, this approach is time-consuming, labor-intensive, has low accuracy, is difficult to replicate, and is not suitable for the general assessment of current urban streets. Therefore, there is a need to develop a new tool based on computer vision technology to automatically identify, track, and analyze activities in open public spaces, and to comprehensively evaluate the quality of public spaces. Summary of the Invention

[0004] To address the aforementioned issues, this invention discloses a street quality evaluation method based on behavior recognition, tracking, and analysis. This method automatically identifies, tracks, and analyzes activities in open public spaces with decimeter-level accuracy. This process significantly reduces the time designers spend conducting site surveys and analyses at the initial stages of a project, thereby improving work efficiency.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A street quality evaluation method based on behavior recognition, tracking, and analysis includes the following steps:

[0007] Step 1: Video Acquisition, Preprocessing, and Data Input

[0008] The selected streets are divided using satellite maps. Taking into account the physical environment of the street segments, appropriate locations and angles are selected. Videos of pedestrians, cyclists, and motor vehicles in the street segments are captured by tilting a high-altitude camera. In addition, the locations selected for the video are surveyed. Based on the site environment, four non-collinear markers are selected in the site. The distances between any five points between the four points are measured using an infrared rangefinder and then input into the data operation interface.

[0009] The captured video was preprocessed, and Adobe After Effects was used to reduce the frame rate and stabilize the video clips. The frame rate of the video clips was set to 10. For videos captured by drones, the "Stabilize Motion" function in the Tracker was used to track stationary objects protruding from the street in the video, thereby achieving image stabilization.

[0010] The program of this invention includes a data operation interface written in Python. This interface is divided into four quadrants: the upper left quadrant is the perspective transformation operation area, the lower left quadrant is the data input area, the upper right quadrant is the perspective transformation preview area, and the lower right quadrant is the file management and operation area. In use, first select the video segment to be preprocessed in the file management and operation area. The program will display the first frame of the video in the perspective transformation operation area. Hold down the middle mouse button and drag the image. Click the button for the points to be entered in the data input area and select four points corresponding to the surveying stage in a clockwise direction in the image. Then, enter the coordinates of the four points in the actual planar space in the data input area. Alternatively, you can enter the distances between the five points; the program will automatically convert the distances into coordinates. After clicking "Perform Perspective Transformation" in the file management and operation area, a transformation matrix is ​​calculated using these eight points. The perspective transformation preview area displays the image transformation under this matrix operation, allowing the user to evaluate the transformation and improve the transformation accuracy by reselecting points or correcting the surveying data. Once the transformation meets the requirements, click "Start Calculation" to complete the data input process. The program will then automatically perform subsequent calculations and generate analysis results.

[0011] Step 2: Basic Data Calculation

[0012] The core computational analysis program of this invention is written in Python and contains three modules. These modules will perform calculations sequentially after the data input stage. The functions and algorithms of the five modules will be explained in detail below.

[0013] 1. Identification and Tracking Module

[0014] The identification and tracking module uses the open-source YOLOv5 object detection algorithm and the ByteTrack multi-object tracking algorithm. This invention rewrites and tunes some functions in the YOLOv5 code to ensure that its computational parameters, model configuration, recognition resolution, and data format meet the requirements of subsequent calculations, facilitating the use of this code in subsequent modules. Using a dataset to train the recognition weight model, this module ultimately exports spatiotemporal location data of street targets, identifying pedestrians, cyclists, and motor vehicles.

[0015] YOLOv5 is a lightweight, fast, and mature object detection (OD) method in computer vision technology, used to detect desired objects from street backgrounds. In this invention, these are pedestrians, cyclists, and motor vehicles closely related to street space. Another important area of ​​computer vision technology is multi-object tracking (MOT). ByteTrack is used to track detected targets; this algorithm connects targets across different frames to form a coherent motion trajectory.

[0016] The detection weight model used in this invention was trained using a database containing 2000 images: 1400 were collected and processed in-house, 300 were from the CrowdHuman dataset, and 200 were from the Visdrone dataset. These three datasets respectively enhance the detection of non-motorized vehicle riders (primarily electric scooters), pedestrians occluding each other in crowded environments, cyclists from a bird's-eye view, and motor vehicles. Through manual analysis, the model achieved an accuracy rate exceeding 90%.

[0017] To accurately identify and track the same person across different video frames, this invention employs the ByteTrack tracking algorithm. ByteTrack uses Kalman filtering to predict the target's trajectory and performs target matching via Intersection over Union (IoU), avoiding complex appearance feature extraction processes and thus achieving efficient real-time tracking while maintaining high accuracy. Furthermore, by introducing a low-resolution detection box matching mechanism, ByteTrack effectively handles issues such as target occlusion, blurring, and transient disappearance, further improving tracking robustness. Combined with the YOLOv5 target detection algorithm, ByteTrack accurately tracks each moving target in street public spaces, providing reliable support for subsequent behavior analysis and data statistics.

[0018] 2. Perspective Transformation Module

[0019] The spatiotemporal trajectory data obtained by the recognition and tracking module is based on the perspective plane of the video viewpoint, and its coordinates are also pixel coordinates. The function of the perspective transformation module is to map the pixel coordinates onto the plane of the actual street space. This module uses the points and coordinates input from the data operation interface and the data file exported by the tracking module for calculation.

[0020] Perspective transformation, also known as projection mapping, involves projecting points from the original image onto a new coordinate system using a transformation matrix. The matrix operation equation has eight unknowns, so solving it requires finding four sets of known mapping points. These mapping points define a three-dimensional space. In this invention, these eight points are the four points obtained through surveying and the four points selected in the data manipulation interface.

[0021] In actual surveying, it is difficult to directly obtain the coordinates of four points. Therefore, this module also includes a surveying verification tool. The specific process involves inputting six distances (four sides and two diagonals) between the four points (A, B, C, D) of the mapped quadrilateral. Then, using reference edges and reference points, the point coordinates are obtained based on a nonlinear solution of a quadratic equation in two variables, and the mapped distances are converted into actual reference coordinates. According to plane geometry principles, the coordinates of the third point can be determined by the coordinates of two known points and their two distances to the third point. Here, a set of quadratic equations in two variables needs to be established, as shown in the following two formulas. x1, y1, x2, y2 are the coordinates of the two points, x0, y0 are the coordinates of the third point, and Dis1 and Dis2 are the distances.

[0022]

[0023] Nonlinear solutions, as a real-number solution method, can eliminate imaginary numbers in equations. When only the lengths of four sides and one diagonal of a quadrilateral are input (or only the lengths of three sides and two diagonals are input), the shape of the quadrilateral can be determined using the above method. When the lengths of four sides and two diagonals are input, the verification and optimization module will be invoked. The verification module first uses five segments (the lengths of four sides and one diagonal of the quadrilateral) to solve a quadratic equation in two variables nonlinearly to obtain initial coordinate results. The error is then compared between the initial coordinate results and the last diagonal, and the calculation formula is shown below. x B y B This is the calculated initial coordinate result of point B, x D y D This is the calculated initial coordinate result of point D, Dis. BD This refers to the measured length of BD. When the error value Ina is less than 0.3, the optimization module is executed directly. When the error value Ina is greater than 0.3 but less than 0.5, the optimized reference value is executed and a remeasurement is recommended. When the error value Ina is greater than 0.5, the error value is considered too large and a remeasurement is required.

[0024]

[0025] The optimization module operates on the principle of a constrained optimization algorithm, implemented in Python by calling the function `scipy.optimize.minimize`. The objective function is defined as the sum of squared errors between the actual side length and the given side length, as shown in the formula below: where d i d represents the actual side length or diagonal length. i ′ is a given value.

[0026]

[0027] The constraints are set as quadrilateral convexity constraints and side length triangle inequality constraints, as shown in the following formulas:

[0028]

[0029] The initial guess results are used to obtain initial coordinates from nonlinear equations. The optimization method employs three parallel optimization algorithms (BFGS, SLSQP, and Trust-const), selecting the result with the smallest error as the final coordinates. Projection mapping eliminates perspective distortion caused by the shooting angle. Therefore, when collecting data on the same space at different times, it is no longer necessary to control the shooting position.

[0030] 3. Classification and Indicator Calculation Module

[0031] The classification and index calculation module is used to calculate the basic motion and clustering of street targets, using spatiotemporal trajectory data transformed by the perspective transformation module. The program first separates the data for pedestrians, cyclists, and motor vehicles, and then calculates the indexes separately for each. This invention sets three basic indicators: two basic motion indicators (v, d) and one cluster number (N). For cyclists and motor vehicles, only v and d need to be calculated, which is basically similar to the pedestrian index algorithm. Therefore, the following explanation of the algorithm and parameter selection will only use the pedestrian data index calculation as an example.

[0032] Instantaneous velocity (v) is an indicator of pedestrian movement and can be determined by displacement between frames. Its calculation formula is as follows:

[0033]

[0034] Where Dis(t) represents the displacement of the object after time t from time i, x i y i These represent the pixel coordinates at different times. The time change Δt is determined by the frame rate; with the video frame rate set to 10, each frame in the data represents 0.1 seconds. F is an integer used to represent the average velocity calculated every F frames. Due to video jitter and the jitter of the recognized bounding box itself, when F is too small, the target's velocity will fluctuate greatly, making it difficult to reflect the true trend and increasing calculation time. Through experiments, setting F to 100 can better reflect the target's velocity change trend. In this case, Dis(t) represents the distance the object moves in 10 seconds, and Δt represents the average velocity per frame within 10 seconds.

[0035]

[0036] The instantaneous change in direction (d) measures the direction of a pedestrian's movement. It is a relative angle in radians and can be determined by the difference in displacement vectors between frames. The movement vector between each frame t is Eq. First, the sine and cosine of the angle between the two vectors are calculated. The absolute value of the angle is then obtained using the inverse cosine function. The sign of the angle can be determined by calculating the vector product or by calculating the arctangent function from the sine and cosine values. To reduce jitter and error, t is set to 10 in the instantaneous velocity calculation formula. Therefore, the instantaneous change in direction (d) represents the angle between the object's displacement in one second and the next, as shown in the following formula:

[0037]

[0038] The cluster number (N) measures whether pedestrian behavior falls within a larger group, and the size of that group, which is crucial for reflecting street vitality. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is used to calculate the cluster number. DBSCAN requires two parameters: the scan radius (eps) and the minimum number of points contained (minPts). The algorithm randomly selects an unvisited point, finds all nearby points within eps, and treats them as a cluster. The scan radius depends on the maximum social distance of pedestrians; experimentally, eps = 200 (2 meters) and minPts = 2 were set. The cluster number is calculated frame-by-frame; each object in each frame receives a cluster number representing the size of its cluster.

[0039] Because an object may pass through other groups while moving, even if it is always acting alone, the DBSCAN algorithm will consider it to be in another group for a period of time, thus increasing its cluster count. Some people in a group may be more than 2 meters away from others at certain times; in these cases, they will be judged as acting alone, resulting in a lower cluster count. The algorithm corrects for these fluctuations in cluster count. Using a piecewise calculation method, with a speed of t=10, the cluster count N is equal to the mode of the cluster size in each frame from time i to time i+t:

[0040] N i =Mode(n i ,n i+1 ...n i+t )

[0041] By introducing the time control parameter t, the amount of data is compressed during the calculation of v, d, and N, with each data point representing the motion state within one second. This calculation effectively reduces errors caused by object jitter while accurately reflecting the object's movement and clustering state.

[0042] Step 3: Judgment of Behavioral Goals

[0043] The pedestrian type judgment module uses the v and N data of pedestrians and calculates the behavioral patterns of pedestrians in the street space through a series of classification judgments. Finally, the program classifies the behaviors of pedestrians into 7 categories, establishing the transformation from basic pedestrian indicators to pedestrian behavior categories.

[0044] Based on literature analysis and experimental data, the present invention sets 7 basic behavior categories for pedestrians, namely single-person stay S1, single-person wandering S2, single-person walking S3, multi-person stay M1, multi-person walking M2, group gathering G1, and group movement G2. The above behavior categories are mainly related to two overall trend indicators: the overall movement trend Mt and the overall clustering trend Nt. These two trend values are respectively related to the real-time movement state Mi and the overall clustering trend Ni.

[0045] The calibration of the real-time movement state Mi requires a speed separation point s1. When the instantaneous speed v < s1, it is recorded as M1, that is, the object is considered stationary. When v > s1, it is recorded as M2, indicating that the object is in motion. The overall movement trend Mt is determined by the stay ratio Ps. The calculation formula for Ps is as follows: when the stay ratio Ps is less than K1, the object can be regarded as always remaining stationary, Mt = 1. When Ps is greater than K2, the object is always moving, Mt = 3. When it is between the two values, it can be determined that the object exhibits a wandering motion state, that is, walking and stopping, and at this time Mt = 2.

[0046]

[0047] The calibration of the overall clustering trend Ni requires two clustering number separation points n1 and n2. When N > n1, it can be considered that multiple people are walking together and is recorded as N2. When N > n2, it can be considered that a group has been formed and is recorded as N3. In other cases, it can be determined as single-person behavior and is recorded as N1. The overall clustering trend Nt is determined by the clustering ratios P1 and P2. When P1 is greater than K3, the object can be regarded as always being alone, Nt = 1. When P2 is greater than K4, the object can be regarded as always being in a group, Nt = 3. In the remaining other cases, Nt = 2, that is, multiple people are moving together. The calculation formulas for P1 and P2 are as follows:

[0048]

[0049] In the present invention, the above preset values are taken as s1 = 2, K1 = 0.25, K2 = 0.8, n1 = 1, n2 = 3, K3 = 0.6, and K4 = 0.5. Based on the obtained overall movement trend Mt and overall clustering trend Nt, the behavior category of an object is judged:

[0050] When Nt = 1, if Mt = 1, it is marked as "single stay" (S1).

[0051] When Nt = 1, if Mt = 2, it is marked as "single loitering" (S2).

[0052] When Nt = 1, if Mt = 3, it is marked as "single person walking" (S3).

[0053] When Nt = 2, if Mt = 1, it is marked as "multiple people staying" (M1).

[0054] When Nt = 2, if Mt = 2 or 3, then it is marked as "multi-person walking" (M2).

[0055] When Nt = 3, if Mt = 1 or 2, it is marked as "group stay" (G1).

[0056] When Nt = 3, if Mt = 3, it is marked as "group movement" (G2).

[0057] Step 4: Street Space Quality Calculation

[0058] The pedestrian behavior category data obtained from the classification and index calculation module is imported into the street space quality evaluation system to calculate spatial quality evaluation indicators for each dimension. This invention establishes six street space evaluation indicators from different perspectives, and generates the final street quality radar map through these indicators.

[0059] 1. Average stay percentage

[0060] Average dwell time is an indicator of attractiveness, calculated as follows: Where N... staying This represents the number of people who paused during a video clip. The total number of people pausing during a clip is determined using the pause determination method in the behavior type determination module. N represents this total number. total This represents the total number of pedestrians. The average stopping percentage reflects the degree to which people are blocked or attracted in a space.

[0061]

[0062] 2. Street traffic speed

[0063] Street traffic speed is an indicator of the smoothness of traffic flow for different objects on a street. It calculates the average speed of different objects in the street space and divides it by the maximum allowed speed for that type of object in the space. The higher the value, the smoother the traffic flow. P TS represents the average traffic speed for pedestrians. c TS represents the average traffic speed for pedestrians. w p represents the average traffic speed for pedestrians. i c iw i Let v be the number of points identified in the automatic identification process for pedestrians, non-motorized vehicles, and motorized vehicles, respectively. p0 v is the maximum walking speed of a pedestrian on the street. c0 Speed ​​limits for non-motorized vehicles on urban roads, v w0 This represents the maximum speed of motor vehicles on urban residential roads. This indicator reflects the smoothness of pedestrian and vehicle flow in street space.

[0064]

[0065] v p0 =6km / h, v c0 =20km / h, v w0 =40km / h

[0066] 3. Trajectory Fluctuation Indicator

[0067] Trajectory fluctuation is an evaluation index related to the change in orientation (d) of an object, used to represent changes in traffic flow and trajectory. The formula for calculating trajectory fluctuation is shown in the following equation, where TF is the average of the weighted average and weighted variance of the orientation changes, and d... i This represents the instantaneous change in direction of each object i at time t. This represents the average directional change of the target, with ω1 and ω2 representing weighted values ​​obtained through actual experimental testing. A higher trajectory fluctuation index indicates weaker traffic flow and greater traffic complexity in that space. Traffic congestion can explain traffic disorder and mixing, and also provides a basis for people gathering and staying in certain areas.

[0068]

[0069] 4. Street congestion level

[0070] Congestion degree is a behavioral characteristic-based indicator that reflects the degree of obstruction by different objects in traffic flow. It is a weighted sum of the loitering behaviors of various objects. Loitering behavior mainly reflects the unevenness of traffic speed changes. Apart from a few pedestrians who actively loiter in the street, most sudden speed changes of objects are caused by obstruction from vehicles or crowds. Therefore, congestion degree can effectively reflect the degree of conflict between different objects during passage. The formula for calculating congestion degree is shown below. Where α, β, and γ are traffic weight coefficients; S² represents the number of loitering pedestrians; N v ′ and M v ′ represents the number of cyclists and motor vehicles exhibiting similar loitering behavior, respectively; l, m, and n represent the total number of pedestrians, cyclists, and motor vehicles on the street, respectively.

[0071]

[0072] 5. Percentage of leisure activities

[0073] The proportion of leisure activities is related to the type of behavior and is an important indicator for measuring the vitality of leisure activities in public spaces. To better reflect the function of streets as public spaces, based on the results of previous research experiments, a weighted average was applied to the proportion of people engaging in leisure and social activities on the street. Simultaneously, the average proportion of people who did not pass through this area was also calculated. The specific calculation method for this indicator is as follows: The average proportion of people staying (S1), lingering (S2), walking in groups (M2), and moving in groups (G2) at different times within a certain period was calculated. Where t is the total time of that period, l i This represents the total number of people appearing in the image at time i. This indicator reflects the proportion of leisure activities taking place in the street space; the higher the value, the greater the leisure attribute of the street and the more suitable it is for people to engage in leisure activities.

[0074]

[0075] 6. Proportion of social activities

[0076] The proportion of social activity is related to the group size (N) and group behavior, and is used to reflect social activity in the design goals of residential streets. It refers to the frequency with which pedestrians meet, walk side-by-side, or interact on the street. This calculation considers group behavior or the movement of two or more people in a group, where their aggregation lasts for a certain period. In the expression, M1 and M2 represent multiple people stopping and multiple people walking, respectively; G1 and G2 represent group stopping and group walking, respectively. i This represents the total number of people appearing in the image at time i. This indicator reflects the proportion of social activities taking place in the street space.

[0077]

[0078] 7. Radar chart

[0079] By calculating the aforementioned street space indicators, these six indicators are visualized using a radar chart. DS represents average dwell time; TS represents street traffic speed; TF represents trajectory fluctuation index; BD represents street congestion level; LA represents the proportion of leisure activities; and SA represents the proportion of social activities. The radar chart formed by these six indicators can intuitively show the characteristics of the street segment. By comparing radar charts of different segments within the same street, the trends of different street segments can be analyzed. Furthermore, by periodically comparing radar charts of the same segment, changes in the spatial indicators of the street segment under different conditions can be monitored.

[0080] The beneficial effects of this invention are:

[0081] This invention relates to an urban spatial quality analysis technology. This technology utilizes drone aerial photography combined with procedural analysis methods to conduct large-scale identification, analysis, and evaluation of urban street spaces. Specifically, this technology comprehensively evaluates the vitality, mobility, and other spatial quality attributes of street spaces by observing and quantitatively analyzing the behavior of pedestrians, cyclists, and motor vehicles in street spaces over a certain period. This technology provides a highly efficient and convenient research tool for the field of urban street design. Designers can record street space videos and input the video data into the street space analysis program to quickly obtain relevant data on street space quality. This process significantly reduces the time designers spend on site surveys and analyses in the early stages of a project, thereby improving work efficiency. Furthermore, the spatial quality data obtained by this technology can provide a scientific basis for urban street renewal. Designers can use this data to scientifically and objectively identify and improve problems in street spaces, providing strong empirical support for design schemes. Through this method, designers can more accurately optimize street spaces during the design phase to improve the quality of urban spaces. Attached Figure Description

[0082] Figure 1 This is the street space quality assessment roadmap described in this invention.

[0083] Figure 2 This is the multi-target recognition and tracking diagram described in this invention.

[0084] Figure 3 This is a schematic diagram illustrating the perspective transformation principle described in this invention.

[0085] Figure 4 This is a diagram of the perspective transformation operation interface described in this invention.

[0086] Figure 5 This is the street target behavior classification diagram described in this invention.

[0087] Figure 6 This is a diagram illustrating the public space quality assessment and analysis of multiple street segments as described in this invention. Detailed Implementation

[0088] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0089] As shown in the figure, the street quality evaluation method based on behavior recognition, tracking, and analysis according to the present invention includes the following steps:

[0090] Step 1: Video Acquisition, Preprocessing, and Data Input

[0091] The selected streets are divided using satellite maps, with each street segment approximately 10m-50m long. Taking into account the physical environment of each street segment (trees, tall buildings), suitable locations and angles are selected, and aerial cameras (drones or city surveillance cameras, etc.) are used to obliquely capture pedestrian trajectories within the street segments, with each video segment lasting 5 minutes. Additionally, mapping is required at the selected locations in the video. Based on the site environment, four non-collinear landmarks are selected, and the distances between any five points are measured using an infrared rangefinder. These measurements are then input into the data processing interface.

[0092] The captured video was preprocessed, and Adobe After Effects was used to reduce the frame rate and stabilize the image in the video clips:

[0093] Set the frame rate of the video clip to 10. For videos captured by drones, use the "Stabilize Motion" function in the "Tracker" to track stationary objects that stand out on the street in the video, thereby achieving image stabilization.

[0094] The program of this invention includes a data operation interface written in Python. This interface is divided into four quadrants: the upper left quadrant is the perspective transformation operation area, the lower left quadrant is the data input area, the upper right quadrant is the perspective transformation preview area, and the lower right quadrant is the file management and operation area. In use, first select the video segment to be preprocessed in the file management and operation area. The program will display the first frame of the video in the perspective transformation operation area. Hold down the middle mouse button and drag the image. By clicking the button for the points to be entered in the data input area and selecting them in the operation area, the user selects four points in the image clockwise corresponding to the surveying stage. Then, the user needs to enter the coordinates of the four points in the actual planar space in the data input area. Alternatively, the user can enter the distance between the five points, and the program will automatically convert the distances into coordinates. After clicking "Perform Perspective Transformation" in the file management and operation area, a transformation matrix is ​​calculated using these eight points. The perspective transformation preview area displays the image transformation under this matrix operation, allowing the user to evaluate the transformation and improve the transformation accuracy by reselecting points or correcting the surveying data. Once the transformation meets the requirements, click "Start Calculation" to complete the data input process. The program will then automatically perform subsequent calculations and generate analysis results.

[0095] Step 2: Basic Data Calculation

[0096] The core computational analysis program of this invention is written in Python and contains three modules. These modules will perform calculations sequentially after the data input stage. The functions and algorithms of the five modules will be explained in detail below.

[0097] 1. Identification and Tracking Module

[0098] The identification and tracking module uses the open-source YOLOv5 object detection algorithm and the ByteTrack multi-object tracking algorithm. This invention has partially rewritten and tuned the parameters of the algorithm, and trained the identification weight model using a self-developed dataset. This module will eventually export the spatiotemporal location data of street spatial objects, and can identify objects including pedestrians, cyclists and motor vehicles.

[0099] YOLOv5 is a lightweight, fast, and mature object detection (OD) method in computer vision technology, used to detect desired objects from street backgrounds; in this invention, these are pedestrians, cyclists, and motor vehicles closely related to street space. Another important area of ​​computer vision technology is multi-object tracking (MOT). This uses improved algorithms to track detected targets, connecting and mapping targets across different frames to form a coherent motion trajectory.

[0100] The detection weight model used in this invention was trained using a self-developed database containing 2000 images, of which 1400 were collected and processed in-house, 300 were from the CrowdHuman dataset, and 200 were from the Visdrone dataset, to respectively enhance the detection of cyclists, pedestrians occluding each other in crowded environments, and motor vehicles from a bird's-eye view. Through manual analysis, the model achieved an accuracy rate of over 90%.

[0101] To accurately identify and track the same person across different video frames, this study employed the ByteTrack tracking algorithm. ByteTrack uses Kalman filtering to predict the target's trajectory and performs target matching via Intersection over Union (IoU), avoiding complex appearance feature extraction processes and thus achieving efficient real-time tracking while maintaining high accuracy. Furthermore, by introducing a low-resolution bounding box matching mechanism, ByteTrack effectively handles issues such as target occlusion, blurring, and transient disappearance, further improving tracking robustness. Combined with the YOLOv5 target detection algorithm, ByteTrack accurately tracks each moving target in street public spaces, providing reliable support for subsequent behavior analysis and data statistics.

[0102] 2. Perspective Transformation Module

[0103] The spatiotemporal trajectory data obtained by the recognition and tracking module is based on the perspective plane of the video viewpoint, and its coordinates are also pixel coordinates. The function of the perspective transformation module is to map the pixel coordinates onto the plane of the actual street space. This module uses the points and coordinates input from the data operation interface and the data file exported by the tracking module for calculation.

[0104] Perspective transformation, also known as projection mapping, involves projecting points from the original image onto a new coordinate system using a transformation matrix. The matrix operation equation has eight unknowns, so solving it requires finding four sets of known mapping points. These mapping points define a three-dimensional space. In this invention, these eight points are the four points obtained through surveying and the four points selected in the data manipulation interface.

[0105] In actual surveying, it is difficult to directly obtain the coordinates of four points. Therefore, this module also includes a surveying verification tool. The specific process involves inputting six distances (four sides and two diagonals) between the four points (A, B, C, D) of the mapped quadrilateral. Then, using reference edges and reference points, the point coordinates are obtained based on a nonlinear solution of a quadratic equation in two variables, and the mapped distances are converted into actual reference coordinates. According to plane geometry principles, the coordinates of the third point can be determined by the coordinates of two known points and their two distances to the third point. Here, a set of quadratic equations in two variables needs to be established, as shown in the following two formulas. x1, y1, x2, y2 are the coordinates of the two points, x0, y0 are the coordinates of the third point, and Dis1 and Dis2 are the distances.

[0106]

[0107] Nonlinear solutions, as a real-number solution method, can eliminate imaginary numbers in equations. When only the lengths of four sides and one diagonal of a quadrilateral are input (or only the lengths of three sides and two diagonals are input), the shape of the quadrilateral can be determined using the above method. When the lengths of four sides and two diagonals are input, the verification and optimization module will be invoked. The verification module first uses five segments (the lengths of four sides and one diagonal of the quadrilateral) to solve a quadratic equation in two variables nonlinearly to obtain initial coordinate results. The error is then compared between the initial coordinate results and the last diagonal, and the calculation formula is shown below. x B y B This is the calculated initial coordinate result of point B, x D y D This is the calculated initial coordinate result of point D, Dis. BD This refers to the measured length of BD. When the error value Ina is less than 0.3, the optimization module is executed directly. When the error value Ina is greater than 0.3 but less than 0.5, the optimized reference value is executed and a remeasurement is recommended. When the error value Ina is greater than 0.5, the error value is considered too large and a remeasurement is required.

[0108]

[0109] The optimization module operates on the principle of a constrained optimization algorithm, implemented in Python by calling the function `scipy.optimize.minimize`. The objective function is defined as the sum of squared errors between the actual side length and the given side length, as shown in the formula below: where d i d represents the actual side length or diagonal length. i ′ is a given value.

[0110]

[0111] The constraints are set as quadrilateral convexity constraints and side length triangle inequality constraints, as shown in the following formulas:

[0112]

[0113] The initial guess results are used to obtain initial coordinates from nonlinear equations. The optimization method employs three parallel optimization algorithms (BFGS, SLSQP, and Trust-const), selecting the result with the smallest error as the final coordinates. Projection mapping eliminates perspective distortion caused by the shooting angle. Therefore, when collecting data on the same space at different times, it is no longer necessary to control the shooting position.

[0114] 3. Classification and Indicator Calculation Module

[0115] The classification and index calculation module is used to calculate the basic motion and clustering of street targets, using spatiotemporal trajectory data transformed by the perspective transformation module. The program first separates the data for pedestrians, cyclists, and motor vehicles, and then calculates the indexes separately for each. This invention sets three basic indicators: two basic motion indicators (v, d) and one cluster number (N). For cyclists and motor vehicles, only v and d need to be calculated, which is basically similar to the pedestrian index algorithm. Therefore, the following explanation of the algorithm and parameter selection will only use the pedestrian data index calculation as an example.

[0116] Instantaneous velocity (v) is an indicator of pedestrian movement and can be determined by displacement between frames. Its calculation formula is as follows:

[0117]

[0118] Where Dis(t) represents the displacement of the object after time t from time i, x i y iThese represent the pixel coordinates at different times. The time change Δt is determined by the frame rate; with the video frame rate set to 10, each frame in the data represents 0.1 seconds. F is an integer used to represent the average velocity calculated every F frames. Due to video jitter and the jitter of the recognized bounding box itself, when F is too small, the target's velocity will fluctuate greatly, making it difficult to reflect the true trend and increasing calculation time. Through experiments, setting F to 100 can better reflect the target's velocity change trend. In this case, Dis(t) represents the distance the object moves in 10 seconds, and Δt represents the average velocity per frame within 10 seconds.

[0119]

[0120] The instantaneous change in direction (d) measures the direction of a pedestrian's movement. It is a relative angle in radians and can be determined by the difference in displacement vectors between frames. The movement vector between each frame t is Eq. First, the sine and cosine of the angle between the two vectors are calculated. The absolute value of the angle is then obtained using the inverse cosine function. The sign of the angle can be determined by calculating the vector product or by calculating the arctangent function from the sine and cosine values. To reduce jitter and error, t is set to 10 in the instantaneous velocity calculation formula. Therefore, the instantaneous change in direction (d) represents the angle between the object's displacement in one second and the next, as shown in the following formula:

[0121]

[0122] The cluster number (N) measures whether pedestrian behavior falls within a larger group, and the size of that group, which is crucial for reflecting street vitality. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is used to calculate the cluster number. DBSCAN requires two parameters: the scan radius (eps) and the minimum number of points contained (minPts). The algorithm randomly selects an unvisited point, finds all nearby points within eps, and treats them as a cluster. The scan radius depends on the maximum social distance of pedestrians; eps = 200 (2 meters), and minPts = 2. The cluster number is calculated frame-by-frame; each object in each frame receives a cluster number representing the size of its cluster.

[0123] Since a certain object will pass by other groups when moving, although it always acts alone, the DBSCAN algorithm will consider it to be in another group for a period of time, resulting in a larger clustering number. Some people in the group will be more than 2m away from others at certain moments, and at this time, they will be judged to act alone, making the clustering number smaller. The fluctuation of the clustering number is corrected by the algorithm. Using the segmented calculation method, with the velocity value t = 10, the clustering number N is equal to the mode of the clustering size of each frame during the period from time i to time i + t:

[0124] N i = Mode(n i ,n i+1 ...n i+t )

[0125] By introducing the time control parameter t, in the calculation process of v, d, and N, the data volume is compressed, and each data represents the motion state within one second. Such calculation can effectively reduce the error caused by object jitter and accurately reflect the moving and clustering states of the object.

[0126] Step 3: Judgment of behavior target

[0127] The pedestrian type judgment module uses the v and N data of pedestrians and calculates the behavior patterns of pedestrians in the street space through a series of classification judgments. The program finally classifies the behaviors of pedestrians into 7 categories, establishing the conversion from the basic indicators of pedestrians to the behavior categories of pedestrians.

[0128] Based on literature analysis and experimental data, the present invention sets 7 basic behavior categories for pedestrians, namely single-person stay S1, single-person wandering S2, single-person walking S3, multi-person stay M1, multi-person walking M2, group gathering G1, and group movement G2. The above behavior categories are mainly related to two overall trend indicators: the overall movement trend Mt and the overall clustering trend Nt. These two trend values are respectively related to the real-time motion state Mi and the overall clustering trend Ni.

[0129] The calibration of the real-time motion state Mi requires a speed separation point s1. When the instantaneous speed v < s1, it is recorded as M1, that is, the object is considered to be stationary. When v > s1, it is recorded as M2, indicating that the object is moving. The overall movement trend Mt is determined by the stay ratio Ps. The calculation formula of Ps is: when the stay ratio Ps is less than K1, the object can be regarded as always remaining stationary, Mt = 1. When Ps is greater than K2, the object is always moving, Mt = 3. When it is between the two values, it can be determined that the object shows a wandering motion state, that is, walking and stopping, and at this time Mt = 2.

[0130]

[0131] The overall clustering tendency Ni requires two cluster separation points n1 and n2. When N > n1, it can be considered that multiple people are moving together, denoted as N2. When N > n2, it can be considered that a group has been formed, denoted as N3. In other cases, it can be considered as individual behavior, denoted as N1. The overall movement tendency Nt is determined by the cluster ratios P1 and P2. When P1 is greater than K3, the object can be considered to always be moving alone, Nt = 1. When P2 is greater than K4, the object can be considered to always be in a group, Nt = 3. In the remaining cases, Nt = 2, that is, multiple people are moving together. The formulas for calculating P1 and P2 are as follows:

[0132]

[0133] In this invention, the preset values ​​are s1 = 2, K1 = 0.25, K2 = 0.8, n1 = 1, n2 = 3, K3 = 0.6, and K4 = 0.5. Based on the obtained overall motion trend Mt and overall clustering trend Nt, the behavior type of a certain object is determined:

[0134] When Nt = 1, if Mt = 1, it is marked as "single stay" (S1).

[0135] When Nt = 1, if Mt = 2, it is marked as "single loitering" (S2).

[0136] When Nt = 1, if Mt = 3, it is marked as "single person walking" (S3).

[0137] When Nt = 2, if Mt = 1, it is marked as "multiple people staying" (M1).

[0138] When Nt = 2, if Mt = 2 or 3, then it is marked as "multi-person walking" (M2).

[0139] When Nt = 3, if Mt = 1 or 2, it is marked as "group stay" (G1).

[0140] When Nt = 3, if Mt = 3, it is marked as "group movement" (G2).

[0141] Step 4: Street Space Quality Calculation

[0142] The pedestrian behavior category data obtained from the classification and index calculation module is imported into the public space quality evaluation system to calculate the space evaluation indicators for each dimension.

[0143] Six public space evaluation indicators were established from different perspectives, and the final street quality radar map was generated using these indicators.

[0144] 1. Average stay percentage

[0145] Average dwell time is an indicator of attractiveness, calculated as follows: Where N... staying This represents the number of people who paused during a video clip. The total number of people pausing during a clip is determined using the pause determination method in the behavior type determination module. N represents this total number. total This represents the total number of pedestrians. The average stopping percentage reflects the degree to which people are blocked or attracted in a space.

[0146]

[0147] 2. Street traffic speed

[0148] Street traffic speed is an indicator of the smoothness of traffic flow for different objects on a street. It calculates the average speed of different objects in the street space and divides it by the maximum allowed speed for that type of object in the space. The higher the value, the smoother the traffic flow. P TS represents the average traffic speed for pedestrians. c TS represents the average traffic speed for pedestrians. w p represents the average traffic speed for pedestrians. i c i w i Let v be the number of points identified in the automatic identification process for pedestrians, non-motorized vehicles, and motorized vehicles, respectively. p0 v is the maximum walking speed of a pedestrian on the street. c0 Speed ​​limits for non-motorized vehicles on urban roads, v w0 This represents the maximum speed of motor vehicles on urban residential roads. This indicator reflects the smoothness of pedestrian and vehicle flow in street space.

[0149]

[0150] v p0 =6km / h, v c0 =20km / h, v w0 =40km / h

[0151] 3. Trajectory Fluctuation Indicator

[0152] Trajectory fluctuation is an evaluation metric related to the change in orientation (d) of an object, used to represent changes in traffic flow and trajectory. The formula for calculating trajectory fluctuation is shown below, where TF is the weighted average of the changes in orientation and the average of the weighted variances, where d... i This represents the instantaneous change in direction of each object i at time t. This represents the average directional change of the target, with ω1 and ω2 representing weighted values ​​obtained through actual experimental testing. A higher trajectory fluctuation index indicates weaker traffic flow and greater traffic complexity in that space. Traffic congestion can explain traffic disorder and mixing, and also provides a basis for people gathering and staying in certain areas.

[0153]

[0154] 4. Street congestion level

[0155] Congestion degree is a behavioral characteristic-based indicator that reflects the degree of obstruction by different objects in traffic flow. It is a weighted sum of the loitering behaviors of various objects. Loitering behavior mainly reflects the unevenness of traffic speed changes. Apart from a few pedestrians who actively loiter in the street, most sudden speed changes of objects are caused by obstruction from vehicles or crowds. Therefore, congestion degree can effectively reflect the degree of conflict between different objects during passage. The formula for calculating congestion degree is shown below. Where α, β, and γ are traffic weight coefficients; S² represents the number of loitering pedestrians; N v ′ and M v ′ represents the number of cyclists and motor vehicles exhibiting similar loitering behavior, respectively; l, m, and n represent the total number of pedestrians, cyclists, and motor vehicles on the street, respectively.

[0156]

[0157] 5. Percentage of leisure activities

[0158] The proportion of leisure activities is related to the type of behavior and is an important indicator for measuring the vitality of leisure activities in public spaces. To better reflect the function of streets as public spaces, based on the results of previous research experiments, a weighted average was applied to the proportion of people engaging in leisure and social activities on the street. Simultaneously, the average proportion of people who did not pass through this area was also calculated. The specific calculation method for this indicator is as follows: The average proportion of people staying (S1), lingering (S2), walking in groups (M2), and moving in groups (G2) at different times within a certain period was calculated. Where t is the total time of that period, l i This represents the total number of people appearing in the image at time i. This indicator reflects the proportion of leisure activities taking place in the street space; the higher the value, the greater the leisure attribute of the street and the more suitable it is for people to engage in leisure activities.

[0159]

[0160] 6. Proportion of social activities

[0161] The proportion of social activity is related to the group size (N) and group behavior, and is used to reflect social activity in the design goals of residential streets. It refers to the frequency with which pedestrians meet, walk side-by-side, or interact on the street. This calculation considers group behavior or the movement of two or more people in a group, where their aggregation lasts for a certain period. In the expression, M1 and M2 represent multiple people stopping and multiple people walking, respectively; G1 and G2 represent group stopping and group walking, respectively. i This represents the total number of people appearing in the image at time i. This indicator reflects the proportion of social activities taking place in the street space.

[0162]

[0163] 7. Radar chart

[0164] By calculating the aforementioned street space indicators, these six indicators are visualized using a radar chart. DS represents average dwell time; TS represents street traffic speed; TF represents trajectory fluctuation index; BD represents street congestion level; LA represents the proportion of leisure activities; and SA represents the proportion of social activities. The radar chart formed by these six indicators can intuitively show the characteristics of the street segment. By comparing radar charts of different segments within the same street, the trends of different street segments can be analyzed. Furthermore, by periodically comparing radar charts of the same segment, changes in the spatial indicators of the street segment under different conditions can be monitored.

[0165] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A street quality evaluation method based on behavior recognition, tracking, and analysis, characterized in that: The steps include the following: Step 1: Video acquisition, preprocessing, and data input; Step 2: Basic data calculation; The computational analysis program is written in Python and consists of three modules. These modules perform calculations sequentially after the data input stage. The three modules are: Identification and tracking module; Perspective transformation module; Classification and index calculation module; Step 3: Determining the behavioral goal; Step 4: Calculation of street space quality; The acquired pedestrian behavior category data is imported into the public space quality evaluation system, and six public space evaluation indicators are calculated. These indicators are used to generate the final street quality radar map. The six public space evaluation indicators are: average dwell time, street traffic speed, trajectory fluctuation index, street congestion level, proportion of leisure activities, and proportion of social activities. Specifically as follows: The street space calculation module imports the acquired pedestrian behavior category data into the public space quality evaluation system to calculate space evaluation indicators for various dimensions. Seven public space evaluation indicators were established from different perspectives, and the final street quality radar map was generated using these indicators. A1. Average stay percentage Average dwell time is an indicator of attractiveness, calculated as follows; where N staying This represents the number of people who paused during a video clip. The total number of people pausing during a clip is determined using the pause determination method in the behavior type determination module. N represents this total number. total This indicates the total number of pedestrians; the average stopping percentage reflects the degree to which people are blocked or attracted in a space. ； A2. Street traffic speed Street traffic speed is an indicator that measures the smoothness of traffic flow for different objects on a street; it calculates the average speed of different objects in the street space and divides it by the maximum allowed speed for that type of object in the space; the higher the value, the smoother the traffic flow. P TS represents the average traffic speed for pedestrians. c TS represents the average traffic speed for pedestrians. w p represents the average traffic speed of pedestrians. i c i w i The points identified in the automatic identification process for pedestrians, non-motorized vehicles, and motorized vehicles are l, m, and n, respectively; where v p0 v is the maximum walking speed of a pedestrian on the street. c0 Speed ​​limits for non-motorized vehicles on urban roads, v w0 This refers to the maximum speed of motor vehicles on urban residential roads; this indicator reflects the smoothness of pedestrian and vehicle flow in street space. ； A3. Trajectory Fluctuation Indicator Trajectory fluctuation is an evaluation index related to the change of orientation (d) of an object, used to represent changes in traffic flow and trajectory; the formula for calculating trajectory fluctuation is as follows, where TF is the average of the weighted average and weighted variance of the orientation change, and d... i This represents the instantaneous change in direction of each object i at time t. The average directional change of the target is represented by ω1 and ω2, which are weighted values ​​obtained through actual experimental testing. The higher the value of the trajectory fluctuation index, the weaker the traffic flow in that space and the higher the complexity of the traffic. ； A4. Street congestion level Congestion degree is a behavioral characteristic-based indicator that reflects the degree of obstruction by different objects in traffic flow. It is a weighted sum of the loitering behaviors of various objects. Loitering behavior mainly reflects the unevenness of traffic speed changes. Apart from a few pedestrians who actively loiter in the street, most sudden speed changes of objects are caused by obstruction from vehicles or crowds. Therefore, congestion degree can reflect the degree of conflict between different objects in the passage process relatively well. The formula for calculating congestion degree is as follows; where α, β, and γ are traffic weight coefficients; S² represents the number of loitering pedestrians; N v ′ and M v ′ represents the number of cyclists and motor vehicles exhibiting similar loitering behavior, respectively; l, m, and n represent the total number of pedestrians, cyclists, and motor vehicles on the street, respectively; ； A5. Percentage of leisure activities The proportion of leisure activities is related to the type of behavior and is an important indicator for measuring the vitality of leisure activities in public spaces. To better reflect the function of streets as public spaces, based on the results of previous research experiments, the proportion of people engaging in leisure and social activities in the streets was weighted. Simultaneously, the average proportion of people who did not pass through this area was also calculated. The specific calculation method for this indicator is as follows: The average proportion of people staying (S1), lingering (S2), walking in groups (M2), and moving in groups (G2) at different times within a certain period was calculated; where t is the total time of that period, l i This represents the total number of people appearing in the image at time i. This indicator reflects the proportion of leisure activities taking place in the street space. The higher the value, the higher the leisure attributes of the street, and the more suitable it is for people to engage in leisure activities in the street space. ； A6. Proportion of Social Activities The proportion of social activity is related to the group size (N) and group behavior, and is used to reflect social activity in the design goals of residential streets; it refers to the frequency with which pedestrians meet, walk side by side, or interact on the street; here, group behavior or multi-person walking involving two or more people, and their gathering behavior lasting for a certain period of time, is calculated; in the expression, M1 and M2 represent multi-person loitering and multi-person walking, respectively; G1 and G2 represent group loitering and group walking, respectively; i This represents the total number of people appearing in the image at time i; this indicator reflects the proportion of social activities taking place in the street space. ； A7. Radar Chart By calculating the aforementioned street space indicators, these six indicators are visualized using a radar chart. DS represents average dwell time; TS represents street traffic speed; TF represents trajectory fluctuation index; BD represents street congestion level; LA represents the proportion of leisure activities; and SA represents the proportion of social activities. The radar chart formed by these six indicators can intuitively represent the characteristics of the street segment. By comparing radar charts of different segments within the same street, the trends of different street segments can be analyzed. Furthermore, by periodically comparing radar charts of the same segment, changes in the spatial indicators of the street segment under different conditions can be monitored.

2. The street quality evaluation method based on behavior recognition, tracking, and analysis according to claim 1, characterized in that: Step one is as follows: The selected streets are divided using satellite maps. Taking into account the physical environment of the street segments, appropriate locations and angles are selected. A high-altitude camera is used to capture pedestrian trajectories in the street segments at an angle to collect video. In addition, the locations selected in the video are surveyed. Based on the site environment, four non-collinear markers are selected in the site. An infrared rangefinder is used to measure any five distances between the four points and then input them into the data operation interface. The captured video was preprocessed, and Adobe After Effects was used to reduce the frame rate and stabilize the video clips: Set the frame rate of the video clip to 10. For videos captured by drones, use the "Stabilize Motion" function in the "Tracker" to track stationary objects that stand out on the street in the video, thereby achieving image stabilization. The program includes a data manipulation interface written in Python. This interface is divided into four quadrants: the upper left quadrant is the perspective transformation operation area, the lower left quadrant is the data input area, the upper right quadrant is the perspective transformation preview area, and the lower right quadrant is the file management and operation area. When using it, first select the video segment that needs to be preprocessed in the file management and operation area. The program will display the first frame of the video in the perspective transformation operation area. Hold down the middle of the mouse and drag the image. By clicking the button for the point to be entered in the data input area and making selections in the operation area, four points corresponding to the surveying stage can be selected clockwise in the image; Next, you need to enter the coordinates of four points or the distances of five points in the actual planar space in the data input area. The program will automatically convert the point distances into point coordinates. After clicking "Perspective Transformation" in the file management and operation area, a transformation matrix will be calculated using these eight points. The perspective transformation preview area will display the image transformation under the matrix operation, allowing users to evaluate the transformation status and improve the transformation accuracy by reselecting points or correcting the survey data. When the transformation status meets the requirements, click "Start Calculation" to end the data input process. The program will then automatically perform subsequent calculations and generate analysis results.

3. The street quality evaluation method based on behavior recognition, tracking, and analysis according to claim 1, characterized in that: Step two is as follows: The computational analysis program is written in Python and consists of five modules. These modules perform calculations sequentially after the data input stage. The functions and algorithms of the five modules are explained in detail below: (1) Identification and Tracking Module The identification and tracking module uses the open-source YOLOv5 object detection algorithm and ByteTrack multi-object tracking algorithm; and uses a dataset to train an identification weight model. This module will eventually export the spatiotemporal location data of street space objects and can identify objects including pedestrians, cyclists and motor vehicles. YOLOv5 is used to detect desired objects from street backgrounds, namely pedestrians, cyclists and motor vehicles closely related to street space; multi-object tracking (MOT) is used to connect and correspond targets between different frames to form a coherent motion trajectory. The detection weight model was trained using a database containing 2,000 images, of which 1,400 were collected and processed by the researchers themselves, 300 were from the CrowdHuman dataset, and 200 were from the Visdrone dataset, to enhance the detection of cyclists, pedestrians occluding each other in crowded environments, and motor vehicles from a bird's-eye view, respectively. To accurately identify and track the same person in different video frames, the ByteTrack tracking algorithm was adopted. By introducing a low-resolution detection box matching mechanism, the problems of target occlusion, blurring, and temporary disappearance can be effectively handled, further improving the robustness of tracking. Combined with the YOLOv5 target detection algorithm, ByteTrack is used to accurately track each moving target in the street public space, providing reliable support for subsequent behavior analysis and data statistics. (2). Perspective Transformation Module The spatiotemporal trajectory data obtained by the recognition and tracking module is based on the perspective plane of the video viewpoint, and its coordinates are also pixel coordinates. The function of the perspective transformation module is to map the pixel coordinates to the plane of the actual street space. This module uses the points input by the data operation interface and the data file exported by the coordinate tracking module for calculation. Perspective transformation, also known as projection mapping, requires the points in the original image to be projected onto a new coordinate system using a transformation matrix. The matrix operation equation has eight unknowns, so solving it requires finding four sets of known mapping points. These mapping points define a three-dimensional space. In operation, these eight points are the four points obtained from the survey and the four points selected in the data operation interface. In actual surveying, it is difficult to directly obtain the coordinates of four points. Therefore, this module also includes a surveying verification tool: The specific process involves inputting six distances (four sides and two diagonals) between the four points (A, B, C, D) of the mapped quadrilateral. Then, using reference edges and reference points, the point coordinates are obtained by solving a quadratic equation in two variables using nonlinearity. The mapped distances are then converted into actual reference coordinates. According to plane geometry principles, the coordinates of the third point are determined by the coordinates of two already defined points and their two distances to the third point. Therefore, a set of quadratic equations in two variables is established, as shown in the following two formulas: x1, y1, x2, y2 are the coordinates of the two points, x0, y0 are the coordinates of the third point, and Dis1 and Dis2 are the distances. ； ； Nonlinear solutions, as a real-number solution method, can eliminate imaginary numbers in the equation. When only the lengths of four sides and one diagonal of a quadrilateral are input, the shape of the quadrilateral is determined according to the above method. When the lengths of four sides and two diagonals are input, the verification and optimization module will be called based on this. The specific process of the verification module is to first obtain the initial coordinate result by nonlinearly solving a quadratic equation with two variables based on the lengths of four sides and one diagonal of the quadrilateral. The error is then compared with the last diagonal, and the calculation formula is as follows; x B y B This is the calculated initial coordinate result of point B, x D y D This is the calculated initial coordinate result of point D, Dis. BD It is the measured length of BD; when the error value Ina is less than 0.3, the optimization module is executed directly; when the error value Ina is greater than 0.3 but less than 0.5, the optimized reference value is executed and it is recommended to remeasure; when the error value Ina is greater than 0.5, the error value is considered too large and remeasurement is required. ； The optimization module operates on the principle of a constrained optimization algorithm, implemented in Python by calling the function `scipy.optimize.minimize`. The objective function is defined as the sum of squared errors between the actual side length and the given side length, as shown in the formula below: where d i d represents the actual side length or diagonal length. i ′ is a given value; ； The constraints are set as quadrilateral convexity constraints and side length triangle inequality constraints, as shown in the following formulas: ； ； The initial guess results are used to obtain the initial coordinates by calling nonlinear equations. The optimization method uses three optimization algorithms to solve in parallel: BFGS, SLSQP, and Trust-const. The result with the smallest error is selected as the final coordinates. Through projection mapping, perspective distortion caused by the shooting angle is eliminated. Therefore, when collecting data on the same space at different times, it is no longer necessary to control the shooting position. (3). Classification and index calculation module The classification and index calculation module is used to calculate the basic motion and clustering of street targets, using spatiotemporal trajectory data transformed by the perspective transformation module. The program first separates the data of pedestrians, cyclists and motor vehicles, and then calculates the indexes separately. A total of three basic indexes are set, namely two basic motion indexes (v, d) and one cluster number (N). For cyclists and motor vehicles, only v and d need to be calculated. Therefore, the algorithm and parameter selection are explained using the index calculation of pedestrian data as an example. Instantaneous velocity (v) is an indicator of pedestrian movement, determined by displacement between frames; its calculation formula is as follows: ； Where Dis(t) represents the displacement of the object after time t from time i. , These represent the pixel coordinates at different times. The time change Δt is determined by the frame rate. With the video frame rate set to 10, each frame in the data represents 0.1 seconds. F is an integer used to represent the average velocity calculated every F frames. Setting F to 100, Dis(t) represents the distance the object moves in 10 seconds, thus representing the average velocity per frame over 10 seconds. ； The instantaneous direction change (d) measures the moving direction of pedestrians. It is a relative angle in radians, determined by the displacement vector difference between frames. The movement vector between every t frames is Eq. First, calculate the sine and cosine values of the angle between two vectors, and use the arccosine function to obtain the absolute value of the angle. Determine the positive or negative of the angle by calculating the cross product of the vectors, or by calculating the arctangent function once using the sine and cosine values. To reduce jitter and error, t uses the value 10 in the instantaneous velocity calculation formula. Therefore, the instantaneous direction change (d) at this time represents the angle between the displacements of an object in one second and the next second, as shown in the following formula: ； ； The number of clusters (N) measures whether pedestrians' behaviors are in a larger group and the size of this group, which is crucial for reflecting street vitality. Use DBSCAN to calculate the number of clusters. DBSCAN requires two parameters: the scanning radius (eps) and the minimum number of points included (minPts). The algorithm randomly selects an unvisited point, finds all nearby points within the distance of eps from it, and regards them as a cluster. The scanning radius depends on the maximum social distance of pedestrians. Set eps = 200, that is, 2 meters, and minPts = 2. The number of clusters is calculated frame by frame. Each object in each frame will get a number of clusters, indicating the size of the cluster it belongs to. Correct the fluctuation of the number of clusters through the algorithm. Use the piecewise calculation method and continue to use the velocity value t = 10. The number of clusters N is equal to the mode of the cluster sizes of each frame during the period from time i to time i + t: ； By introducing the time control parameter t, the data volume is compressed during the calculation of v, d, and N. Each data represents the motion state within one second.

4. The street quality evaluation method based on behavior recognition, tracking, and analysis according to claim 1, characterized in that: Step three is as follows: The pedestrian type determination module uses the v and N data of pedestrians. Through classification judgment, calculate the behavior patterns of pedestrians in the street space. The program finally classifies the behaviors of pedestrians into 7 categories, establishing the conversion from the basic indicators of pedestrians to the pedestrian behavior categories. Set 7 basic behavior types for pedestrians, namely single-person stay S1, single-person wandering S2, single-person walking S3, multi-person stay M1, multi-person walking M2, group gathering G1, and group movement G2. These behavior types are related to two overall trend indicators: the overall movement trend Mt and the overall clustering trend Nt. These two trend values are respectively related to the real-time movement state Mi and the overall clustering trend Ni. Calibrating the real-time movement state Mi requires a speed separation point s1. When the instantaneous velocity v < s1, it is recorded as M1, that is, the object is considered stationary. When v > s1, it is recorded as M2, and the object is considered to be moving. The overall movement trend Mt is determined by the stay ratio Ps. The calculation formula of Ps is: when the stay ratio Ps is less than K1, the object is regarded as always remaining stationary, Mt = 1. When Ps is greater than K2, the object is always moving, Mt = 3. When it is between the two values, it is determined that the object shows a wandering motion state, that is, walking and stopping, and at this time Mt = 2; ； The overall clustering tendency Ni requires two cluster separators n1 and n2. When N > n1, it is considered that multiple people are moving together, denoted as N2. When N > n2, it is considered that a group has been formed, denoted as N3. All other cases are considered as individual behavior, denoted as N1. The overall movement tendency Nt is determined by the cluster ratios P1 and P2. When P1 is greater than K3, the object is considered to always be moving alone, Nt = 1. When P2 is greater than K4, the object is considered to always be in a group, Nt = 3. For the remaining cases, Nt = 2, that is, multiple people are moving together. The formulas for calculating P1 and P2 are as follows: ； ； The preset values ​​are s1=2, K1=0.25, K2=0.8, n1=1, n2=3, K3=0.6, K4=0.5; based on the overall motion trend Mt and the overall clustering trend Nt obtained above, the behavior type of a certain object is determined: When Nt=1, if Mt=1, it is marked as "single stay" (S1). When Nt=1, if Mt=2, it is marked as "single wandering" (S2). When Nt=1, if Mt=3, it is marked as "single person walking" (S3). When Nt=2, if Mt=1, it is marked as "multiple people staying" (M1). When Nt=2, if Mt=2 or 3, it is marked as "multi-person walking" (M2). When Nt=3, if Mt=1 or 2, it is marked as "group stay" (G1). When Nt=3, if Mt=3, it is marked as "group movement" (G2).

Images