Blind person street crossing prompter based on video analysis

Abstract

The invention relates to the technical field of intelligent traffic assistance, in particular to a blind person street-crossing prompter based on video analysis, which is characterized in that a video data acquisition module acquires a road monitoring video, extracts a real-time picture of a zebra crossing area, identifies pedestrians in the picture, screens out a blind person target individual carrying a blind guide stick or wearing a specific wearable device, and transmits the screened blind person target individual to a video processing module; and calculating a displacement track of the blind target individual among the video frames to obtain blind street crossing data. According to the invention, video data acquisition is expanded to precise monitoring of a zebra crossing area, a specific target is focused, pedestrians carrying blind guide sticks or wearing specific equipment can be screened out, and effective identification of blind individuals is realized. The moving path is calculated in combination with the displacement track, on the basis of stride frequency, step length and deviation angle analysis, the action stability is judged through an inertial sensor, the transverse deviation or stagnation situation in the moving process of the blind person can be dynamically evaluated, and the target recognition precision is improved.

Description

Technical Field

[0001] The invention relates to the technical field of intelligent traffic assistance, and in particular to a blind person crossing street reminder based on video analysis. Background Art

[0002] The field of intelligent traffic assistance technology includes the use of various information technologies, video analysis technologies, and sensor equipment to optimize traffic management, improve traffic safety, and improve travel conditions for special groups of people. The core content of this technology field includes traffic signal control, automatic identification of pedestrian and vehicle status, and intelligent prompt systems. Among them, the auxiliary technology for pedestrian crossing mainly relies on sensors, cameras, and signal processing equipment to monitor the status of pedestrians, and provides corresponding prompt information in combination with the traffic signal system. The overall technical system covers target detection, environmental perception, signal control, and information interaction to improve the intelligence level of the transportation system and meet the travel needs of different groups of people.

[0003] Among them, a blind crossing reminder based on video analysis refers to a device that uses video images to identify whether a blind person needs to cross the street and provides corresponding voice prompts accordingly. The device mainly collects video images of intersections by installing a high-definition camera near the pedestrian signal light, and uses target detection technology to identify the blind stick used by the blind to determine the position of the blind and the direction of movement. When it is detected that the blind person needs to cross the street, the device further analyzes the status of the signal light in the corresponding direction, and uses power signal detection technology to obtain traffic light change information, and finally provides the blind person with a voice prompt that matches the direction of his or her crossing.

[0004] The blind crossing reminder device based on video analysis relies only on the detection of the blind stick for identification. There are cases of misidentification or missed identification due to the occlusion of the blind stick and angle deviation, and it is impossible to effectively distinguish between the blind and ordinary pedestrians. The target recognition method based only on the camera is difficult to dynamically track the path of the blind, and fails to fully consider the cadence, step length and travel deviation, resulting in a rough judgment of the blind's travel status, unable to accurately identify potential deviations or abnormal stagnation, affecting the stability of the auxiliary system. The obstacle detection range is based only on the information of traffic light changes, and cannot distinguish between moving or stationary obstacles, and cannot judge the impact of obstacles on the passage of the blind, which is easy to cause inaccurate passage prompts. The voice prompt system gives prompts based on the status of the traffic light, without combining the specific travel situation of the blind, which may cause the prompt content to not match the actual needs and increase the cognitive burden of the blind. The existing technology lacks the cadence adjustment function and fails to dynamically adjust the blind's travel rhythm according to the remaining time of the green light, which may cause the blind to misjudge the passing time, thereby increasing the probability of detention or dangerous situations. In general, the assistance methods of existing technologies are relatively simple and fail to achieve a comprehensive closed loop from target screening to traffic assistance, making it difficult to further improve street crossing safety and passing efficiency. Summary of the invention

[0005] In order to solve the technical problems existing in the prior art, an embodiment of the present invention provides a blind person crossing street reminder based on video analysis. The technical solution is as follows: A blind person crossing street reminder based on video analysis, the reminder comprising: The video data acquisition module obtains surveillance videos, extracts images of zebra crossing areas, identifies blind people carrying guide sticks or specific equipment, calculates their trajectories, and establishes data on blind people crossing the street; The pedestrian crossing state recognition module extracts the step frequency, step length, and deviation angle based on the blind person crossing data, calculates the deviation, calls the inertial sensor data to evaluate the stability, determines the deviation or stagnation, and generates the blind person's walking state data; The obstacle detection and obstacle avoidance assessment module uses the foreground segmentation information based on the blind person crossing the street data, screens obstacles, calculates the relative distance and speed, and combines the stability data to assess the passage risk and generate obstacle avoidance risk data; The signal light linkage step frequency adjustment module obtains the road signal light status information, extracts the remaining time of the green light, calculates the street crossing time required by the current step frequency according to the blind person's walking status data, compares the countdown time to determine whether to adjust the step frequency, and generates step frequency adjustment control data; The voice prompt output module is based on the obstacle avoidance risk data and the step frequency adjustment control data, classified by risk level, screened the prompt type, and generated a street crossing broadcast instruction.

[0006] The present invention has the following improvements: the blind person crossing the street data includes a target individual gait sequence, a displacement vector between video frames, and a trajectory coordinate point set; the blind person's moving status data includes a cadence parameter, a step length parameter, a moving offset, and a trajectory curvature sequence; the obstacle avoidance risk data includes an obstacle relative distance, an approach rate, and a traffic risk assessment value; the cadence adjustment control data includes a cadence adjustment amplitude, a target cadence value, and a green light countdown; and the broadcast instructions include a traffic risk level classification, a prompt type, and a voice prompt template.

[0007] The present invention is improved in that the video data acquisition module comprises: The video monitoring submodule obtains the road monitoring video data and extracts the video frames, calls the image segmentation parameters to obtain the zebra crossing area, calculates the effective picture of the zebra crossing according to the pixel distribution boundary of the zebra crossing area, calls the sequence information of the video frames, and obtains the frame sequence of the zebra crossing area; The target recognition submodule uses the target detection parameters to screen pedestrians based on the zebra crossing area frame sequence, calculates gait features according to the outline, detects guide sticks and wearable devices, screens target individuals and numbers them, and obtains individual blind person mapping data; The trajectory calculation submodule obtains the displacement of the target individual between frames based on the blind individual mapping data, calls the coordinate points to calculate the motion trajectory, screens out abnormal points according to the coordinate changes, calls the time series to calculate the step frequency, screens the stable step frequency target, and obtains the displacement trajectory of the blind target; The data generation submodule obtains the duration of stay in the zebra crossing area based on the displacement trajectory of the blind target, calculates the crossing speed according to the path, calls the time information to count the number of crossings, filters the complete trajectory data, and obtains the blind crossing data.

[0008] The present invention is improved in that the gait characteristics of the blind target individual between video frames are calculated. , using the formula: ; Where T is the gait period, is the coordinate of the key joint point at time t, To avoid the minimum value of division by zero error.

[0009] The present invention is improved in that the pedestrian crossing state recognition module comprises: The gait parameter extraction submodule calls the displacement trajectory information based on the blind person crossing the street data, calculates the step frequency and step length, selects the stable gait trajectory, and obtains the gait parameter data; The travel offset calculation submodule analyzes the lateral position of the target individual based on the gait parameter data, calls the inter-frame coordinate points to obtain the direction angle, calculates the offset between the target individual and the central axis of the zebra crossing according to the angle change, screens outliers with sudden changes in the angle of the target individual's travel direction, calls the time series data to construct a deviation trend sequence, obtains the target individual's deviation rate according to the step length and angle of the blind person's travel direction, and obtains the travel offset data; The walking state evaluation submodule analyzes the gait stability based on the walking deviation data, calls the trajectory curvature sequence to calculate the lateral deviation value, compares the lateral deviation value with the preset lateral deviation value threshold, filters the abnormal stagnant frame data, and obtains the walking state data of the blind person.

[0010] The present invention has the following improvements: for calculating the angle change of the blind person's traveling direction , using the formula: ; in, is the angle change between frames, is the angle mean, is the position coordinate of the target individual in the previous frame, is the position coordinate of the target individual in the current frame.

[0011] The present invention is improved in that for calculating the lateral offset value , using the formula: ; in, is the standard deviation of curvature, is the median curvature, Indicates the target's x-direction moving speed, Indicates the target's y-direction moving speed, represents the acceleration of the target in the x direction, Indicates the acceleration of the target in the y direction.

[0012] The present invention is improved in that the obstacle detection and obstacle avoidance assessment module comprises: The obstacle screening submodule extracts the target in the zebra crossing area based on the blind person crossing data by calling the video foreground segmentation information, screens the moving and stationary obstacles, calls the coordinate information to mark the obstacles, and obtains the obstacle recognition data; The relative distance calculation submodule obtains the distance between the target individual and the obstacle based on the obstacle recognition data, calls the time series information to obtain the approach speed between the target individual and the obstacle, sets the reference approach speed according to the average walking speed of blind people in the zebra crossing area, screens out targets with approach speeds higher than the preset reference approach speed, calculates the contact probability between the target individual and the obstacle, screens out targets with contact probability lower than the preset contact probability threshold, calls the step length and approach speed to analyze the collision risk coefficient, and obtains the relative position information of the obstacle; The obstacle avoidance risk assessment submodule uses the blind person's walking stability data to calculate the target individual's deviation amplitude based on the obstacle relative position information, uses the gait stability data to analyze the obstacle avoidance response capability, screens the target individuals with obstacles, and obtains obstacle avoidance risk data.

[0013] The present invention is improved in that the signal light linkage step frequency adjustment module comprises: The signal light information extraction submodule obtains the road signal light status information based on the blind person's travel status data, extracts the remaining time of the green light, calls the time series data to calculate the countdown accuracy and filter the signal data, and obtains the remaining time data of the green light; The cadence calculation submodule extracts the target individual cadence based on the green light remaining time data by calling the blind walking stability data, and calculates the required crossing time under the current cadence by calling the step length and zebra crossing width, obtains the cadence margin according to the green light countdown time, screens individuals whose cadence margin does not exceed the preset cadence margin threshold, constructs a cadence comparison sequence based on the time series data, obtains the cadence adjustment demand intensity, and obtains cadence analysis data; The cadence adjustment determination submodule calculates the cadence adjustment value of the target individual based on the cadence analysis data, determines whether to perform cadence adjustment according to the adjustment value, screens the target individual for cadence adjustment, and obtains cadence adjustment control data.

[0014] The present invention is improved in that the voice prompt output module comprises: The risk level classification submodule calls the obstacle approach rate and gait stability data based on the obstacle avoidance risk data and the cadence adjustment control data, sets the risk assessment standard according to the traffic safety threshold, calculates the risk coefficient of the blind target individual, screens the targets whose risk coefficient exceeds the traffic safety threshold, calls the time series data to construct the risk level sequence, and obtains the traffic risk level data; The prompt type screening submodule screens dynamic prompt information based on the traffic risk level data and the cadence adjustment state, matches the preset prompt type data with the corresponding prompt category, obtains the obstacle avoidance prompt priority according to the obstacle relative position information, screens the category with the prompt priority higher than the prompt trigger threshold, constructs a prompt information matching sequence, obtains the optimal prompt type matching the cadence adjustment control data with the obstacle avoidance state, and obtains prompt type matching data; The announcement instruction generation submodule matches the data based on the prompt type, calls the voice prompt library to match the current scene announcement template, sets the announcement timing according to the time series data, filters the announcement sentences and generates the announcement instructions.

[0015] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: In the present invention, video data collection is extended to the precise monitoring of zebra crossing areas, focusing on specific targets, and can screen out pedestrians carrying guide sticks or wearing specific equipment, so as to achieve effective identification of blind individuals. The travel path is calculated in combination with the displacement trajectory. On the basis of the analysis of the step frequency, step length and offset angle, the inertial sensor is used to judge the stability of the action, and the lateral deviation or stagnation of the blind during the process of travel can be dynamically evaluated to improve the accuracy of target recognition. The screening of obstacles not only depends on the foreground segmentation information, but also calculates the relative distance and approach rate between people and obstacles to achieve effective prediction of the risk of passage. Based on the remaining time of the green light and the time required for the step frequency calculation, the travel rhythm adjustment is more accurate, which can improve the efficiency of passage and reduce the risk of failing to pass the intersection in time. The voice prompt method is classified according to the different levels of passage risk, which helps to reduce unnecessary information interference and make the prompt more intuitive and easy to understand. The overall solution integrates target screening, trajectory analysis, obstacle detection, passage rhythm control and dynamic voice broadcasting to achieve a complete closed loop from blind identification to passage assistance, and improve the safety and convenience of blind people crossing the street. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 is a system flow chart of the present invention; Figure 2 It is a submodule flow chart of the present invention; Figure 3 It is a flow chart of the video data acquisition module of the present invention; Figure 4 This is a flow chart of a pedestrian crossing status recognition module of the present invention; Figure 5 It is a flow chart of the obstacle detection and obstacle avoidance assessment module of the present invention; Figure 6 This is a flow chart of the signal light linkage step frequency adjustment module of the present invention; Figure 7 This is a flow chart of the voice prompt output module of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0019] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0020] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0021] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0022] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0023] See also Figure 1 The present invention provides a technical solution: a blind person crossing street reminder based on video analysis, the reminder comprising: The video data acquisition module obtains surveillance videos, extracts images of zebra crossing areas, identifies blind people carrying guide sticks or specific equipment, calculates their trajectories, and establishes data on blind people crossing the street; The pedestrian crossing state recognition module extracts the step frequency, step length, and deviation angle based on the blind person crossing data, calculates the deviation, calls the inertial sensor data to evaluate the stability, determines the deviation or stagnation, and generates the blind person's walking state data; The obstacle detection and obstacle avoidance assessment module uses the foreground segmentation information based on the blind person crossing the street data, screens obstacles, calculates the relative distance and speed, evaluates the passage risk in combination with the stability data, and generates obstacle avoidance risk data; The signal light linkage step frequency adjustment module obtains the road signal light status information, extracts the remaining time of the green light, calculates the street crossing time required by the current step frequency according to the blind person's walking status data, compares the countdown time to determine whether to adjust the step frequency, and generates step frequency adjustment control data; The voice prompt output module is based on the obstacle avoidance risk data and the step frequency adjustment control data, classified by risk level, screened the prompt type, and generated a street crossing broadcast instruction.

[0024] The blind people's street crossing data includes the target individual gait sequence, the displacement vector between video frames, and the trajectory coordinate point set. The blind people's moving status data includes the cadence parameters, step length parameters, moving offset, and trajectory curvature sequence. The obstacle avoidance risk data includes the relative distance of obstacles, approach rate, and traffic risk assessment value. The cadence adjustment control data includes the cadence adjustment amplitude, target cadence value, and green light countdown. The broadcast instructions include traffic risk level classification, prompt type, and voice prompt template.

[0025] See also Figure 3 and Figure 2 , the video data acquisition module includes: The video monitoring submodule obtains the road monitoring video data and extracts the video frames, calls the image segmentation parameters to obtain the zebra crossing area, calculates the effective picture of the zebra crossing according to the pixel distribution boundary of the zebra crossing area, calls the sequence information of the video frames, and obtains the frame sequence of the zebra crossing area; The video monitoring submodule obtains data from the road monitoring video and extracts the video frame. It first reads the shooting parameters of the monitoring device, including resolution R, frame rate F, exposure time E, etc., to ensure that the quality of the video processed later meets the requirements. The interval time of video frame extraction Depending on the frame rate setting, usually select , to ensure that the data covers a comprehensive time range, each frame of the image is denoised and histogram equalized to make subsequent analysis more stable; when segmenting the image, the system calls the established binary threshold , segment the pixel brightness and preliminarily determine the zebra crossing area. The threshold is generally set at This range is based on the statistical analysis of the brightness histogram of 10,000 road monitoring images, taking the average pixel brightness Add the brightness standard deviation The multiple of is calculated as follows: ; Among them, k is used as a "weight coefficient" to control the segmentation sensitivity. Its value range is 1.5≤k≤2.0. Its value varies with the ambient light conditions. In low light conditions (nighttime), k takes a higher value (such as 2.0). When the light is uniform during the day, k takes 1.5 to reduce false detection. For example, in a certain video frame, the average pixel brightness , standard deviation , the ambient light is uniform during the day, take k=1.5, and calculate: ; Then the binarization threshold of the frame is 165, ensuring the stability of the zebra crossing area segmentation. Then, morphological operations are used to remove isolated noise points, and edge detection technology is used to lock the zebra crossing boundary, and finally the complete zebra crossing area is obtained. In the boundary calculation process, edge pixels are extracted based on the connected domain analysis method, the area range is calculated based on the x and y coordinates of each pixel point, and the minimum bounding rectangle method is used to ensure the integrity of the zebra crossing in the picture, avoiding the influence of tilt or occlusion on the judgment, and the obtained boundary data is stored as a polygon coordinate set. , the boundary area A of the effective zebra crossing area is calculated as follows: ; Assuming that the boundary of the zebra crossing area consists of four vertices (100, 200), (300, 200), (300, 400), (100, 400), calculate the area: ; Then the area of ​​the zebra crossing area A = 20,000 pixels squared.

[0026] The target recognition submodule uses the target detection parameters to select pedestrians based on the zebra crossing area frame sequence, calculates gait features based on the outline, detects guide sticks and wearable devices, selects target individuals and numbers them, and obtains individual blind person mapping data; The target recognition submodule screens pedestrian targets for each frame based on the zebra crossing area frame sequence. After setting the grayscale and color space standards, it separates the moving targets through the background modeling method, extracts the outer contours of the candidate targets, and calculates their area. , aspect ratio Calculate and remove pixels with an area less than 500 pixels or an aspect ratio exceeding The goal is to set the pedestrian contour features within the 99% confidence interval in the real road monitoring video data to ensure that the false screening rate is less than 1%; set the gait features The calculation method is as follows: ; For example, if a target's gait changes as follows: , ,and =1, we get: ; The trajectory calculation submodule obtains the displacement of the target individual between frames based on the blind individual mapping data, calls the coordinate points to calculate the motion trajectory, screens out abnormal points according to the coordinate changes, calls the time series to calculate the step frequency, screens the stable step frequency target, and obtains the displacement trajectory of the blind target; The trajectory calculation submodule analyzes the displacement of the target individual between frames based on the blind individual mapping data. First, the coordinates of the center point of the target in each frame are extracted. , calculate the displacement between frames : ; If the coordinates of the target in the two frames are (150,250) and (160,270), then: .

[0027] The data generation submodule obtains the duration of stay in the zebra crossing area based on the blind person's target displacement trajectory, calculates the crossing speed according to the path, calls the time information to count the number of crossings, and filters the complete trajectory data to obtain the blind person's crossing data; The data generation submodule calculates the duration of stay in the zebra crossing area based on the displacement trajectory of the blind target. First, the time series data of the target in the zebra crossing area is extracted to calculate the entry time. and departure time , get the duration of stay : ; like , ,but: ; If the threshold value exceeds 10s, a long-stay event is recorded. Finally, the complete trajectory data is filtered to obtain the blind crossing data.

[0028] See also Figure 4 and Figure 2 , the pedestrian crossing status recognition module includes: The gait parameter extraction submodule uses the displacement trajectory information based on the blind person's street crossing data, calculates the step frequency and step length, and selects stable gait trajectories to obtain gait parameter data; The gait parameter extraction submodule is based on the blind person crossing street data. After calling the displacement trajectory information, the gait sequence of the target individual is first parsed, the coordinates of the key points of the foot in each frame are extracted from the trajectory data, and the inter-frame step length is calculated. By calculating the displacement change between two consecutive gait cycles, the complete step frequency and step length data are obtained. The number of frames in each gait cycle is determined by detecting the trend of foot position changes in adjacent frames. The improved step frequency calculation is as follows: ; Where F is the video frame rate, is the number of frames in the gait cycle, is the inter-frame variation of the step length, is the historical mean of the step length, the inter-frame change of the step length Need to set threshold , to determine whether the gait fluctuation exceeds the normal range, set , which is derived from the 95% confidence interval of a large number of pedestrian gait data. If the step length variation of a target δd>0.05m, it is determined that its step frequency is affected by external factors, such as obstacles or sudden stops, and additional marking is required. Assuming that the frame rate of a target is F=30fps, a gait cycle is completed after 20 frames, the step length variation between frames δd=0.03m, and the step length mean =0.68m, then the step frequency is calculated as follows: ; Calculation step length When , the improved calculation formula is as follows: ; in, is the step size standard deviation, is the median step length, set the step length fluctuation threshold ≤0.1m, this value is set according to the gait data distribution. >0.1m, indicating that the target's gait changes greatly, and there may be abnormal gait, and the trajectory needs to be re-marked. Assume that the step length sequence of a target is 0.65m, 0.70m, 0.68m, 0.72m, 0.66m, and the step length standard deviation =0.025m, median step length =0.68m, then: .

[0029] The travel offset calculation submodule analyzes the lateral position of the target individual based on the gait parameter data, calls the inter-frame coordinate points to obtain the direction angle, calculates the offset between the target individual and the zebra crossing center axis according to the angle change, screens outliers where the angle of the target individual's travel direction changes suddenly, calls the time series data to construct the offset trend sequence, obtains the target individual's offset rate according to the step length and angle of the blind person's travel direction, and obtains the travel offset data; The travel offset calculation submodule extracts the lateral position sequence of the target individual based on the gait parameter data, reads the coordinate points of adjacent frames from the trajectory data, calculates the angle change of the travel direction, and improves the direction angle calculation as follows: ; in, is the angle change between frames, is the angle mean, and sets the angle mutation threshold ,like , the target direction changes too much, which may be caused by obstacles or emergencies. The frame data needs to be removed. Assume that the coordinates of a target at two consecutive moments are (100, 200) and (120, 240). The angle changes between frames , angle mean , then calculate the angle: .

[0030] The walking state evaluation submodule analyzes the gait stability based on the walking deviation data, calls the trajectory curvature sequence to calculate the lateral deviation value, compares the lateral deviation value with the preset lateral deviation value threshold, filters the abnormal stagnant frame data, and obtains the walking state data of the blind person; The walking state evaluation submodule extracts the individual's gait stability based on the walking deviation data, calls the trajectory curvature sequence to calculate the lateral deviation value, and improves the curvature calculation as follows: ; in, is the standard deviation of curvature, is the median curvature, setting the lateral offset curvature threshold ,like , then it is determined that the target has an abnormal deviation. Assuming that the trajectory data of a target is =2, =1, =3, =-1, standard deviation of curvature , the median curvature , then calculate the curvature:

[0031] like , then the target is marked as offset, and finally the blind person’s moving status data is obtained.

[0032] See also Figure 5 and Figure 2 , the obstacle detection and avoidance assessment module includes: The obstacle screening submodule uses the video foreground segmentation information to extract targets in the zebra crossing area based on the blind crossing data, screens moving and stationary obstacles, and uses the coordinate information to mark the obstacles to obtain obstacle recognition data; The obstacle screening submodule is based on the blind crossing data. After extracting the target in the zebra crossing area by calling the video foreground segmentation information, it first screens the moving and stationary obstacles, and calculates the displacement between frames by extracting the center coordinate points of the targets in consecutive frames. , set the motion determination threshold In order to distinguish between moving obstacles and stationary obstacles, the improved movement determination formula is as follows: ; in, , is the center coordinate of the target in the current frame, , is the center coordinate of the target in the previous frame, is the displacement standard deviation, is the median displacement, set the movement judgment threshold Pixels, this value comes from the target frame displacement distribution calculated for 3000 sets of video frame data, where the displacement of the moving target within the 95% confidence interval is greater than 2 pixels. If the coordinates of the target in the previous frame are , the coordinates of the current frame are , displacement standard deviation Pixels, median Pixels, then calculate:

[0033] because , then the target is determined to be a moving obstacle. Call the coordinate information to mark the obstacle and set the obstacle boundary box , store the coordinates of all obstacles in the zebra crossing area and set the bounding box overlap rate threshold , calculate the overlap rate between the target and the marked obstacles : ; in, is the overlapping area between the target and the existing obstacles, is the target area, is the standard deviation of obstacle area, is the median obstacle area, set , which is derived from the overlap rate statistics of 5000 groups of obstacle areas, ensuring that the misjudgment rate is less than 5%. If the obstacle area of ​​a target is Pixels, overlapping area Pixels, obstacle area standard deviation Pixels, median Pixels, then calculate: ; because , merge obstacle information, and finally obtain obstacle recognition data.

[0034] The relative distance calculation submodule obtains the distance between the target individual and the obstacle based on the obstacle recognition data, calls the time series information to obtain the approach speed between the target individual and the obstacle, sets the benchmark approach speed according to the average walking speed of blind people in the zebra crossing area, screens out targets with approach speeds higher than the preset benchmark approach speed, calculates the contact probability between the target individual and the obstacle, screens out targets with contact probability lower than the preset contact probability threshold, calls the step length and approach speed to analyze the collision risk coefficient, and obtains the relative position information of the obstacle; The relative distance calculation submodule obtains the distance between the target individual and the obstacle based on the obstacle recognition data, and calculates the target center point between frames. and obstacle center The Euclidean distance between , the improved distance calculation formula is as follows: ; Set the minimum safe distance threshold =1.5m, which is derived from the data of 2000 blind people crossing the street. The minimum safe obstacle avoidance radius of the blind within the 95% confidence interval is 1.5m. If the target coordinates are (100, 200) and the obstacle coordinates are (120, 230), the distance standard deviation =0.5m, median =2.5m, then calculate: .

[0035] The obstacle avoidance risk assessment submodule uses the blind person's walking stability data to calculate the target individual's deviation amplitude based on the relative position information of the obstacle, and uses the gait stability data to analyze the obstacle avoidance response capability, screens the target individuals with obstacles, and obtains the obstacle avoidance risk data; The obstacle avoidance risk assessment submodule uses the blind person's walking stability data to calculate the target individual deviation amplitude and deviation angle based on the relative position information of the obstacle. , the improved offset angle calculation is as follows: ; Set the maximum allowable deviation angle threshold , which is derived from the analysis of real pedestrian trajectories. The normal pedestrian travel angle fluctuation does not exceed 20° within the 98% confidence interval. If the coordinates of adjacent frames are (100, 200) and (110, 220), the standard deviation of the offset angle is =2 degrees, median =15 degrees, then calculate: ; Finally, the obstacle avoidance risk data is obtained.

[0036] See also Figure 6 and Figure 2 , the signal light linkage step frequency adjustment module includes: The signal light information extraction submodule obtains the road signal light status information based on the blind person's travel status data, extracts the remaining time of the green light, calls the time series data to calculate the countdown accuracy and filter the signal data to obtain the remaining time data of the green light; After the signal light information extraction submodule obtains the road signal light status information based on the blind person's travel status data, it first extracts the remaining time of the current green light, obtains the green light countdown data from the signal control system, and performs time series analysis on the sampling interval, calculates the countdown error between consecutive frames, and screens out the error that meets the set threshold. signal data to ensure the accuracy of countdown data and set the time series error threshold , which is calculated by the standard deviation of 5000 sets of traffic signal data to ensure that the error does not exceed the 95% confidence interval. The error calculation formula is as follows: ; in, Countdown for the measured green light, For the countdown based on time series regression prediction, assuming that the current measured green light time is =18.2s, calculate the predicted time =17.9s, then calculate: ; because , the signal data meets the screening conditions, if , then the data is eliminated and does not participate in the calculation of the remaining green light time, and the remaining green light time data is finally obtained.

[0037] The cadence calculation submodule extracts the target individual cadence based on the green light remaining time data and the blind walking stability data, and uses the step length and zebra crossing width to calculate the required crossing time at the current cadence. The cadence margin is obtained according to the green light countdown time, and individuals whose cadence margin does not exceed the preset cadence margin threshold are screened. A cadence comparison sequence is constructed based on the time series data to obtain the cadence adjustment demand intensity and obtain cadence analysis data. The cadence calculation submodule uses the data of remaining green light time and blind walking stability to extract the target individual cadence, and uses the step length and zebra crossing width to calculate the required crossing time at the current cadence. The cadence calculation formula is as follows: ; in, is the width of the zebra crossing, is the target individual cadence, Set the stride margin threshold for the target individual stride length =3s, which is obtained by measuring the street crossing time of 2000 blind individuals and calculating the 95% confidence interval to ensure that the cadence margin is reasonable under the actual pedestrian flow. The cadence margin M is calculated as: ; Assuming target cadence =1.5Hz, step size =0.7m, zebra crossing width =8m, the remaining time of the current green light =12s, then calculate: ; M = 12-7.62-4.38s; because , the target cadence margin is sufficient and no adjustment is required. , the target is filtered to enter the cadence adjustment sequence, and finally the cadence analysis data is obtained.

[0038] The cadence adjustment determination submodule calculates the cadence adjustment value of the target individual based on the cadence analysis data, determines whether to adjust the cadence according to the adjustment value, selects the target individual for cadence adjustment, and obtains the cadence adjustment control data; The cadence adjustment determination submodule calculates the target individual cadence adjustment value based on the cadence analysis data. The cadence adjustment value is calculated as follows: ; Setting the Cadence Adjustment Threshold =0.2Hz, which is obtained by analyzing 3000 sets of cadence adjustment experimental data to ensure that the cadence adjustment does not exceed the walking physiological limit of the target individual. , then the target needs to adjust the step frequency, otherwise keep the current step frequency, assuming , , , , then calculate: ; because , filter the target to adjust the cadence, and finally obtain the cadence adjustment control data.

[0039] Table 1 Calculation of green light remaining time error

[0040] As shown in Table 1, the countdown errors of some traffic lights exceed the threshold, and data screening is required to ensure that the countdown accuracy meets the set requirements.

[0041] See also Figure 7 and Figure 2 , the voice prompt output module includes: The risk level classification submodule uses obstacle avoidance risk data and cadence adjustment control data, calls obstacle approach rate and gait stability data, sets risk assessment criteria according to the traffic safety threshold, calculates the risk coefficient of blind target individuals, screens targets whose risk coefficient exceeds the traffic safety threshold, calls time series data to construct risk level sequences, and obtains traffic risk level data; The risk level classification submodule is based on the obstacle avoidance risk data and the cadence adjustment control data. After calling the obstacle approach rate and gait stability data, it first calculates the risk coefficient of the blind target individual. , which comprehensively considers the approach rate of the target individual , gait stability , obstacle distance attenuation factor and traffic safety threshold , the optimized risk coefficient calculation formula is as follows: ; in: and They are the speed weight coefficient and the stability weight coefficient, which are used to balance the influence of speed and stability. The setting basis is the statistical analysis of 10,000 sets of target marching data, so that the influence ratio of step frequency and speed change is reasonable; =1.5m, which is the minimum safe distance under the 95% confidence interval. The calculation is based on the average obstacle avoidance distance of the target individual in different scenarios, ensuring that 95% of the targets can safely avoid obstacles within this range; =0.3 is the obstacle attenuation coefficient, which is used to correct the influence of obstacles within a short distance. The setting basis is the velocity distribution curve of the target individual approaching the obstacle, so that the short-distance obstacles have a greater impact on the calculation; =0.2 is the gait stability influence coefficient, which ensures that the gait stability target will not be judged as high risk due to minor disturbances. The setting basis is the 90% confidence interval of the gait variation coefficient to filter out the influence of normal gait fluctuations.

[0042] Assumptions: .

[0043] Then calculate: ; Set traffic safety threshold , which is calculated through linear regression analysis of 10,000 sets of target movement data, so that more than 90% of low-risk targets will not be misjudged as high-risk. , then the target has a higher risk, and finally the traffic risk level data is obtained.

[0044] The prompt type screening submodule screens dynamic prompt information based on the traffic risk level data and the cadence adjustment status, matches the preset prompt type data with the corresponding prompt category, obtains the obstacle avoidance prompt priority according to the obstacle relative position information, screens the category with a prompt priority higher than the prompt trigger threshold, constructs a prompt information matching sequence, obtains the optimal prompt type matching the cadence adjustment control data and the obstacle avoidance status, and obtains the prompt type matching data; The prompt type screening submodule filters dynamic prompt information based on the traffic risk level data and the cadence adjustment status. First, the cadence adjustment status and prompt type are matched to construct a prompt information weight matrix. , according to the degree of adjustment of the target individual's cadence , obstacle avoidance status and the coefficient of variation of the time series step frequency Calculating prompt priority : ; in: and They are the cadence adjustment weight coefficient and the obstacle avoidance weight coefficient, which are set based on the weight analysis of 5000 sets of cadence adjustment and obstacle avoidance experimental data to ensure that cadence adjustment plays a leading role in prompt selection; The adjustment factor for the coefficient of cadence variation is calculated based on the standard deviation of cadence fluctuations, ensuring that targets with high cadence variation receive higher priority. It is a nonlinear adjustment factor for the obstacle avoidance state, which increases the priority of targets with high obstacle avoidance requirements. The setting basis is calculated based on the average deviation angle of the target individual obstacle avoidance path. is the maximum obstacle avoidance state value, is the current cadence.

[0045] Assumptions: .

[0046] Then calculate: ; Set the reminder trigger threshold =0.35, which is based on the 90% confidence interval of the prompt information feedback experiment. , a high priority prompt is triggered, and finally the prompt type matching data is obtained.

[0047] The broadcast instruction generation submodule matches the data based on the prompt type, calls the voice prompt library to match the current scene broadcast template, sets the broadcast timing according to the time series data, filters the broadcast sentences and generates broadcast instructions; The broadcast instruction generation submodule matches the data based on the prompt type, calls the voice prompt library to match the current scene broadcast template, and sets the broadcast timing. To ensure that the broadcast information is completed before the target individual adjusts its cadence and avoids obstacles, calculate the broadcast advance : ; in: In order to optimize the broadcast time coefficient and ensure that the broadcast will not be invalid due to a short remaining green light time, the setting is based on the analysis of 2000 sets of broadcast response data.

[0048] Assumption: The green light time is , the pedestrian action time is .

[0049] Then calculate: ; Set the broadcast advance threshold =3.5s, which is determined by statistical analysis of blind people's street crossing behavior data. , then the prompt is broadcast immediately, otherwise the broadcast order is adjusted and the broadcast instruction is finally generated.

[0050] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A blind person crossing street reminder based on video analysis, characterized in that: The prompter comprises: The video data acquisition module obtains surveillance videos, extracts images of zebra crossing areas, identifies blind people carrying guide sticks or specific equipment, calculates their trajectories, and establishes data on blind people crossing the street; The pedestrian crossing state recognition module extracts the step frequency, step length, and deviation angle based on the blind person crossing data, calculates the deviation, calls the inertial sensor data to evaluate the stability, determines the deviation or stagnation, and generates the blind person's walking state data; The obstacle detection and obstacle avoidance assessment module uses the foreground segmentation information based on the blind person crossing the street data, screens obstacles, calculates the relative distance and speed, evaluates the passage risk in combination with the stability data, and generates obstacle avoidance risk data; The signal light linkage step frequency adjustment module obtains the road signal light status information, extracts the remaining time of the green light, calculates the street crossing time required by the current step frequency according to the blind person's walking status data, compares the countdown time to determine whether to adjust the step frequency, and generates step frequency adjustment control data; The voice prompt output module is based on the obstacle avoidance risk data and the step frequency adjustment control data, classified by risk level, screened the prompt type, and generated a street crossing broadcast instruction.

2. The blind person crossing street reminder device based on video analysis according to claim 1, characterized in that: The blind person crossing the street data includes the target individual gait sequence, the displacement vector between video frames, and the trajectory coordinate point set; the blind person moving status data includes the cadence parameter, the step length parameter, the moving offset, and the trajectory curvature sequence; the obstacle avoidance risk data includes the relative distance of the obstacle, the approach rate, and the traffic risk assessment value; the cadence adjustment control data includes the cadence adjustment amplitude, the target cadence value, and the green light countdown; the broadcast instructions include the traffic risk level classification, the prompt type, and the voice prompt template.

3. The blind person crossing street reminder device based on video analysis according to claim 1, characterized in that: The video data acquisition module comprises: The video monitoring submodule obtains the road monitoring video data and extracts the video frames, calls the image segmentation parameters to obtain the zebra crossing area, calculates the effective picture of the zebra crossing according to the pixel distribution boundary of the zebra crossing area, calls the sequence information of the video frames, and obtains the frame sequence of the zebra crossing area; The target recognition submodule uses the target detection parameters to select pedestrians based on the zebra crossing area frame sequence, calculates gait features according to the outline, detects guide sticks and wearable devices, selects target individuals and numbers them, and obtains blind individual mapping data; The trajectory calculation submodule obtains the displacement of the target individual between frames based on the blind individual mapping data, calls the coordinate points to calculate the motion trajectory, screens out abnormal points according to the coordinate changes, calls the time series to calculate the step frequency, screens the stable step frequency target, and obtains the displacement trajectory of the blind target; The data generation submodule obtains the duration of stay in the zebra crossing area based on the displacement trajectory of the blind target, calculates the crossing speed according to the path, calls the time information to count the number of crossings, filters the complete trajectory data, and obtains the blind crossing data.

4. The blind person crossing street reminder device based on video analysis according to claim 3, characterized in that: For calculating the gait characteristics of blind target individuals between video frames , using the formula: ； Where T is the gait period, is the coordinate of the key joint point at time t, To avoid the minimum value of division by zero error.

5. The blind person crossing street reminder device based on video analysis according to claim 1, characterized in that: The pedestrian crossing state recognition module comprises: The gait parameter extraction submodule calls the displacement trajectory information based on the blind person crossing the street data, calculates the step frequency and step length, selects the stable gait trajectory, and obtains the gait parameter data; The travel offset calculation submodule analyzes the lateral position of the target individual based on the gait parameter data, calls the inter-frame coordinate points to obtain the direction angle, calculates the offset between the target individual and the central axis of the zebra crossing according to the angle change, screens outliers with sudden changes in the angle of the target individual's travel direction, calls the time series data to construct a deviation trend sequence, obtains the target individual's deviation rate according to the step length and angle of the blind person's travel direction, and obtains the travel offset data; The walking state evaluation submodule analyzes the gait stability based on the walking deviation data, calls the trajectory curvature sequence to calculate the lateral deviation value, compares the lateral deviation value with the preset lateral deviation value threshold, filters the abnormal stagnant frame data, and obtains the walking state data of the blind person.

6. The blind person crossing street reminder device based on video analysis according to claim 5, characterized in that: To calculate the angle change of the blind person's direction of travel , using the formula: ； in, is the angle change between frames, is the angle mean, is the position coordinate of the target individual in the previous frame, is the position coordinate of the target individual in the current frame.

7. The blind person crossing street reminder device based on video analysis according to claim 5, characterized in that: For calculating the lateral offset , using the formula: ； in, is the standard deviation of curvature, is the median curvature, Indicates the target's x-direction moving speed, Indicates the target's y-direction moving speed, represents the acceleration of the target in the x direction, Indicates the acceleration of the target in the y direction.

8. The blind person crossing street reminder device based on video analysis according to claim 1, characterized in that: The obstacle detection and avoidance assessment module includes: The obstacle screening submodule extracts the target in the zebra crossing area based on the blind person crossing data by calling the video foreground segmentation information, screens the moving and stationary obstacles, calls the coordinate information to mark the obstacles, and obtains the obstacle recognition data; The relative distance calculation submodule obtains the distance between the target individual and the obstacle based on the obstacle recognition data, calls the time series information to obtain the approach speed between the target individual and the obstacle, sets the reference approach speed according to the average walking speed of blind people in the zebra crossing area, screens out targets with approach speeds higher than the preset reference approach speed, calculates the contact probability between the target individual and the obstacle, screens out targets with contact probability lower than the preset contact probability threshold, calls the step length and approach speed to analyze the collision risk coefficient, and obtains the relative position information of the obstacle; The obstacle avoidance risk assessment submodule uses the blind person's walking stability data to calculate the target individual's deviation amplitude based on the obstacle relative position information, uses the gait stability data to analyze the obstacle avoidance response capability, screens the target individuals with obstacles, and obtains obstacle avoidance risk data.

9. The blind person crossing street reminder device based on video analysis according to claim 1, characterized in that: The signal light linkage step frequency adjustment module includes: The signal light information extraction submodule obtains the road signal light status information based on the blind person's travel status data, extracts the remaining time of the green light, calls the time series data to calculate the countdown accuracy and filter the signal data, and obtains the remaining time data of the green light; The cadence calculation submodule extracts the target individual cadence based on the green light remaining time data by calling the blind walking stability data, and calculates the required crossing time under the current cadence by calling the step length and zebra crossing width, obtains the cadence margin according to the green light countdown time, screens individuals whose cadence margin does not exceed the preset cadence margin threshold, constructs a cadence comparison sequence based on the time series data, obtains the cadence adjustment demand intensity, and obtains cadence analysis data; The cadence adjustment determination submodule calculates the cadence adjustment value of the target individual based on the cadence analysis data, determines whether to perform cadence adjustment according to the adjustment value, screens the target individual for cadence adjustment, and obtains cadence adjustment control data.

10. The blind person crossing street reminder device based on video analysis according to claim 1, characterized in that: The voice prompt output module comprises: The risk level classification submodule calls the obstacle approach rate and gait stability data based on the obstacle avoidance risk data and the cadence adjustment control data, sets the risk assessment standard according to the traffic safety threshold, calculates the risk coefficient of the blind target individual, screens the targets whose risk coefficient exceeds the traffic safety threshold, calls the time series data to construct the risk level sequence, and obtains the traffic risk level data; The prompt type screening submodule screens dynamic prompt information based on the traffic risk level data and the cadence adjustment state, matches the preset prompt type data with the corresponding prompt category, obtains the obstacle avoidance prompt priority according to the obstacle relative position information, screens the category with the prompt priority higher than the prompt trigger threshold, constructs a prompt information matching sequence, obtains the optimal prompt type matching the cadence adjustment control data with the obstacle avoidance state, and obtains prompt type matching data; The announcement instruction generation submodule matches the data based on the prompt type, calls the voice prompt library to match the current scene announcement template, sets the announcement timing according to the time series data, filters the announcement sentences and generates the announcement instructions.

Images