Blind area pedestrian early warning method and system based on V2X and machine vision

Abstract

The invention relates to the technical field of road early warning, and discloses a blind area pedestrian early warning method and system based on V2X and machine vision, and the key points of the technical scheme are that a road side end RSU carries out edge calculation, and starts to detect road pedestrian information; if the RSU at the roadside end detects that the pedestrian appears on the current road, tracking the pedestrian based on machine vision, and calculating the accurate position of the current pedestrian target; the road side end RSU broadcasts and sends blind area pedestrian information to external traffic participants, and when a driving vehicle enters a V2I communication range, a vehicle-mounted OBU receives pedestrian information of a front road; when the vehicle-mounted OBU receives the information of the pedestrian ahead, collision early warning calculation is carried out, and the possibility of collision is judged; the road pedestrian shielding detection problem can be solved in the roadside system, meanwhile, the vehicle-mounted equipment receives the pedestrian information from the blind area through V2I communication, the front road pedestrian information is obtained through the beyond-visual-range sensing capacity, the driving safety is improved, and the protection measures for the road pedestrians are perfected.

Description

Technical Field

[0001] The present invention relates to the field of road warning technology, and more specifically, to a blind spot pedestrian warning method and system based on V2X and machine vision. Background Art

[0002] In actual road traffic operation, the existing pedestrian warning roadside system often encounters special conditions such as pedestrians being obscured, pedestrian targets being small and far away from the roadside equipment, and severe weather conditions such as rain, snow, and fog. As a result, traditional roadside technology is prone to false detection and missed detection of pedestrian targets due to the loss of target features, which greatly interferes with the operation of the traffic system and is prone to traffic accidents.

[0003] Therefore, the present invention provides a blind spot pedestrian warning method and system based on V2X and machine vision, which improves the above-mentioned technical problems. Summary of the invention

[0004] The disclosed embodiments aim to address the deficiencies of the prior art and provide a blind spot pedestrian warning method and system based on V2X and machine vision. The warning system of the present invention can solve the detection problem of road pedestrian obstruction in the roadside system. At the same time, the on-board equipment receives pedestrian information from the blind spot through V2I communication, and uses the beyond-visual-range perception capability to obtain pedestrian information on the road ahead, thereby improving driving safety and improving the protection measures for road pedestrians.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: A blind spot pedestrian warning method and system based on V2X and machine vision, comprising the following steps:

[0006] S1. Start the V2X communication equipment, which includes: a roadside RSU equipment and an onboard OBU equipment;

[0007] S2: The roadside RSU performs edge computing and starts detecting pedestrian information on the road;

[0008] S3. If the roadside RSU detects a pedestrian on the current road, it tracks the pedestrian based on machine vision and calculates the exact position of the current pedestrian target based on the fusion positioning of radar and image information;

[0009] S4, the roadside RSU broadcasts the pedestrian information in the blind spot to external traffic participants. When the moving vehicle enters the V2I communication range, the on-board OBU receives the pedestrian information on the road ahead;

[0010] S5. When the vehicle-mounted OBU receives information about the pedestrian ahead, it calculates the collision warning based on the current vehicle's driving direction, speed, and acceleration, and determines the possibility of a collision. If no collision will occur, the vehicle continues to drive;

[0011] S6. If the vehicle-mounted OBU determines that the warning conditions are met, the warning level is divided based on the Mazda collision model, and the front pedestrian collision warning prompt is displayed on the UI tablet connected to the vehicle-mounted OBU, and a voice broadcast reminds the driver to slow down.

[0012] As a preferred technical solution of the present invention, the pedestrian detection model in machine vision and the multi-sensor fusion positioning technology are deployed in the edge computing device. The edge computing device is connected to the camera and the radar through the serial port to obtain data information of the road environment and analyze it to obtain pedestrian information as the input of the RSU device;

[0013] The edge computing process is: data-level fusion of the image detection results and the radar detection results, which involves three parts: pedestrian detection, pedestrian tracking, and pedestrian positioning;

[0014] The process of detecting road pedestrian information is as follows: pedestrian detection adopts the YOLOv7 single-stage target detection algorithm based on machine learning, which can process video streams faster and more accurately and quickly detect and locate pedestrian targets in the video, and at the same time can optimize the detection effect of road-occluded pedestrians and small-target pedestrians.

[0015] As a preferred technical solution of the present invention, the process of tracking pedestrians based on machine vision is: Kalman filtering prediction is performed on the marking frame of the target detection result to obtain the predicted frame of the pedestrian target in the next frame, and then cascade matching is performed with the target frame after the actual pedestrian moves in the next frame. If the match is successful, it means that they are the same target, and the tracking of the pedestrian target can be completed.

[0016] As a preferred technical solution of the present invention, the process of calculating the accurate position of the current pedestrian target based on the fusion positioning of radar and image information is as follows: the image information is subjected to pedestrian detection and tracking through machine vision technology, and then the radar information is subjected to noise filtering and data correction, and then the two information are fused at the sensor level, feature level and decision level, and finally the distance of the pedestrian is obtained, and the position in the spatial coordinate system is calculated. The specific position of the pedestrian is represented by longitude and latitude coordinates.

[0017] As a preferred technical solution of the present invention, the process of calculating the collision warning according to the current driving direction, speed and acceleration of the vehicle is: calculating the distance d between the longitude and latitude coordinates of HV and P by the longitude and latitude coordinates of HV and P; as follows:

[0018]

[0019] Among them, formula (1) is the Haversine formula, which can be used to calculate the distance d between two points by the longitude and latitude; where (lon1, lat1) is the driving coordinate, (lon2, lat2) is the pedestrian coordinate, sin and cos are trigonometric functions, arcsin is the inverse sine function, and R is the radius of the earth.

[0020] As a preferred technical solution of the present invention, the process of dividing the warning level based on the Mazda collision model is: according to the Mazda collision model strategy, the warning level is divided using the safety distance strategy; as shown in the following formula:

[0021]

[0022] Among them, formula (2) is the Mazda model formula, where D is the safety distance, which is equal to the braking distance plus the stopping reaction distance, v and a m is the current vehicle speed and maximum braking deceleration, ν r and a n are the speed and maximum braking deceleration of the target object, and t1 and t2 are the reaction times of the two objects.

[0023] As a preferred technical solution of the present invention, the process of displaying a front pedestrian collision warning prompt on a UI tablet connected to the vehicle-mounted OBU, and a voice broadcast to remind the driver to slow down is: select a V2X Android application on an Android tablet as a terminal display, use WIFI to wirelessly connect the vehicle-mounted OBU device, receive the vehicle BSM information and pedestrian warning information transmitted by the DVIN module through the TCP communication protocol, and display this information on the Android app;

[0024] The terminal display interface is used to animate the vehicle's driving status information and received warning information, and to use voice broadcasts to remind the driver to slow down or stop in emergency warning situations.

[0025] A blind spot pedestrian warning system based on V2X and machine vision, the system comprising a roadside system and a vehicle-mounted system;

[0026] The roadside system includes: edge computing equipment, roadside sensors, and roadside RSU equipment;

[0027] The vehicle-mounted system includes: a vehicle-mounted OBU device, a vehicle-mounted controller, and a UI interaction device.

[0028] As a preferred technical solution of the present invention,

[0029] The edge computing device is used to detect and record the movement trajectory and location information of pedestrians;

[0030] The roadside sensor is used to obtain the perception information of the surrounding environment, and filter and fuse the perception information to transmit to the edge computing device;

[0031] The roadside RSU device is used to transmit the movement trajectory, speed, location and other information of the pedestrian to other traffic participants in a broadcast form;

[0032] The vehicle-mounted OBU device is used to receive roadside safety information transmitted by the roadside RSU device, and obtain the vehicle speed, position and other status information according to the vehicle interface service module; perform collision warning calculations for vulnerable traffic participants according to the driving speed, position and pedestrian information, and divide the warning level;

[0033] The on-board controller is used to adopt a corresponding warning strategy according to the collision warning level calculated by the on-board OBU device;

[0034] The UI interaction device obtains the human-computer interaction interface information of the vehicle-mounted OBU device through WLAN, and is used to convert the GNSS information of the pedestrians and vehicles ahead into map positions and display them dynamically in real time.

[0035] In summary, the present invention has the following beneficial effects:

[0036] First, the present invention detects whether there are pedestrians in the road blind spot through the roadside equipment environment perception system, and uses machine learning methods to fuse the image and radar information, optimizes the roadside system's detection performance for obstructed pedestrians and long-distance pedestrians, and realizes the functions of identifying, tracking and locating road pedestrians on the edge computing device, calculates the RSM information such as the position, speed, and moving direction of the target pedestrian, and wirelessly transmits the pedestrian information to external traffic participants through the RSU device. When the main vehicle enters the communication range, the OBU device receives the roadside system information, and determines whether it meets the VRUCW scenario based on the BSM information such as the vehicle's position, speed, and heading angle. If the collision conditions are met, the collision level is calculated, and finally the warning interface is displayed on the tablet device, and the driver is reminded by voice that there are pedestrians in the blind spot of the road ahead and that deceleration is required.

[0037] Secondly, the present invention can solve the problem of detecting pedestrian obstruction in the roadside system. At the same time, the vehicle-mounted equipment receives pedestrian information from blind spots through V2I communication, and uses the beyond-visual-range perception capability to obtain pedestrian information on the road ahead, thereby improving driving safety and improving the protection measures for pedestrians on the road. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A diagram of a warning implementation scenario provided by an embodiment of the present invention;

[0039] Figure 2An architecture diagram of a blind spot pedestrian warning system based on V2X and machine vision provided in an embodiment of the present invention;

[0040] Figure 3 An improved YOLOv7 pedestrian detection model diagram provided by an embodiment of the present invention;

[0041] Figure 4 A flow chart of a blind spot pedestrian warning method based on V2X and machine vision provided in an embodiment of the present invention;

[0042] Figure 5 A flowchart of a pedestrian detection and tracking algorithm provided by an embodiment of the present invention;

[0043] Figure 6 A VRUCW level determination flow chart provided in an embodiment of the present invention;

[0044] Figure 7 A V2X application display interface provided by an embodiment of the present invention;

[0045] Figure 8 This is a V2X warning display interface provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present application is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements can also be made without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. In addition, the words "first", "second", "third", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0049] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in this specification and in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0050] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0051] The embodiment of the present disclosure aims to solve the problem that roadside technology easily causes pedestrian target misdetection and missed detection due to target feature loss. In view of this, the embodiment of the present disclosure proposes to provide a blind spot pedestrian warning method and system based on V2X and machine vision, such as Figure 1 As shown in the figure, the warning implementation scenario diagram of the present invention includes driving vehicles, pedestrians, parked vehicles (obstacles) and roadside equipment. The warning system can solve the detection problem of pedestrian occlusion on the road in the roadside system. At the same time, the vehicle-mounted equipment receives pedestrian information from the blind spot through V2I communication, and uses the beyond-visual-range perception capability to obtain pedestrian information on the road ahead, thereby improving driving safety and improving the protection measures for pedestrians on the road.

[0052] Please refer to Figure 2 , Figure 2 The following is an architecture diagram of a blind spot pedestrian warning system based on V2X and machine vision according to an embodiment of the present disclosure. The overall architecture of the system is mainly divided into two parts: a roadside system and a vehicle-mounted system.

[0053] Specifically, the roadside system includes: edge computing equipment, cameras, radar sensors, and roadside RSU equipment. The vehicle-mounted system includes: vehicle-mounted OBU equipment, vehicle-mounted controller, and UI interaction equipment.

[0054] The early warning system of the present invention first obtains real-time image information of the road through a camera, and obtains the movement trajectory, speed, and location information of surrounding pedestrians through a radar sensor, and transmits the perception information to an edge computing device through V2X communication technology;

[0055] Furthermore, the edge computing device connects the camera and the radar through the serial port to obtain data information of the road environment and analyze it to obtain more accurate pedestrian information as the input of the RSU device. Edge computing performs data-level fusion of the image detection results and the radar detection results, which involves three parts: pedestrian detection, pedestrian tracking, and pedestrian positioning. Pedestrian detection uses the YOLOv7 single-stage target detection algorithm based on machine learning, which can process video streams faster and more accurately and quickly detect and locate pedestrian targets in the video. At the same time, it can optimize the detection effect of pedestrians obscured by the road and small-target pedestrians. The present invention improves the structure of the feature extraction network and the feature fusion network of the original YOLOv7 model, and trains it under the WiderPerson and CityPersons data sets to improve better pedestrian detection effects. The above-mentioned improved YOLOv7 pedestrian detection model, such as Figure 3 shown.

[0056] After target detection, pedestrian tracking uses the DeepSORT target tracking algorithm, which can track the same target in multiple consecutive video frames, and the target ID will not jump when the pedestrian on the road is temporarily occluded. The specific implementation is: first, the marked frame of the target detection result is predicted by Kalman filtering to obtain the predicted frame of the pedestrian target in the next frame, and then cascade matching is performed with the target frame of the actual pedestrian after moving in the next frame. If the match is successful, it means that they are the same target, and the pedestrian target can be tracked.

[0057] Pedestrian positioning uses image information and radar information to fuse and then determine the specific location of pedestrians. The specific implementation is: the image information is used for pedestrian detection and tracking through machine vision technology, and then the radar information is subjected to noise filtering and data correction, and then the two information are fused at the sensor level, feature level and decision level, and finally the distance of the pedestrian is obtained, and the position in the spatial coordinate system is calculated. The specific location of the pedestrian is expressed in longitude and latitude coordinates.

[0058] The roadside RSU device transmits the movement trajectory, speed, position and other information of pedestrians to other traffic participants in the form of broadcast through the wireless message service module; when the moving vehicle enters the V2I warning area, the on-board OBU device can receive the roadside safety information transmitted by the roadside RSU device, and the roadside safety information includes: the movement trajectory, speed and position information of the pedestrians; and connects to the CAN bus through the vehicle interface service module to obtain the vehicle speed, position and other status information; calculates the collision warning of vulnerable traffic participants according to the driving speed, position and pedestrian information, and divides the warning level; the on-board UI device obtains the human-computer interaction interface information of the OBU through WLAN, converts the GNSS information of the pedestrians and vehicles in front into the map position and displays it dynamically in real time. According to the collision warning calculation results, if the collision warning conditions are triggered, the map status bar displays a warning and the voice prompts the driver to slow down or stop.

[0059] The on-board controller is used to take corresponding warning strategies according to the collision warning level calculated by the on-board OBU device. When the warning level is low, the driver is reminded of the potential risk through the UI interactive device in the form of sound, light, vibration, etc.; when the warning level is high, in addition to reminding the driver, the vehicle is also controlled through the vehicle bus interface to perform active safety operations, such as automatic deceleration, emergency braking, etc., until the collision risk is eliminated.

[0060] Please refer to Figure 4 , Figure 4 The flowchart of the blind spot pedestrian warning based on V2X and machine vision described in the embodiment of the present disclosure is shown. The overall process mainly includes the following 6 steps:

[0061] S1: First, start the roadside RSU device and the vehicle-mounted OBU device to prepare for the later V2X communication. The vehicle-to-infrastructure communication (V2I) here uses LTE wireless communication technology with a communication range of several hundred meters to 1 kilometer. When the vehicle enters the road within the communication range of the roadside unit, it can wait to receive information from the roadside system;

[0062] S2: The roadside starts the edge computing device and starts detecting pedestrian information on the road. The machine vision pedestrian detection model and multi-sensor fusion positioning technology are deployed in the edge computing device. The camera is connected to obtain real-time images, and the millimeter-wave radar is connected to obtain external environment information. The improved YOLOv7 pedestrian detection algorithm can quickly determine whether there are pedestrians in the detection area, including detecting pedestrian targets blocked by obstacles;

[0063] S3: If a pedestrian is detected on the current road, the pedestrian is tracked through machine vision technology, and the accurate position of the current pedestrian target is calculated based on the fusion positioning of radar and image information, specifically the longitude and latitude coordinates of the pedestrian in the map coordinate system;

[0064] S4: The roadside RSU device broadcasts the pedestrian information in the blind spot to external traffic participants. When the moving vehicle enters the V2I communication range, the on-board OBU device receives the pedestrian information on the road ahead, including the pedestrian's movement trajectory and speed, as well as the longitude and latitude coordinates at the current moment.

[0065] S5: The vehicle-mounted OBU receives information about the pedestrian ahead and performs collision warning calculations based on the current vehicle direction, speed, and acceleration, thereby determining the possibility of a collision. If no collision will occur, the vehicle continues to drive safely through the main road;

[0066] S6: If the vehicle system determines that the warning conditions are met, the vehicle system divides the warning level according to the Mazda collision model, and displays the front pedestrian collision warning prompt on the UI tablet connected to the OBU device, and voice broadcasts to remind the driver to slow down;

[0067] Specifically, Figure 5 As shown in the figure, the target detection and tracking algorithm process is:

[0068] 1) First, input the video image to be detected, use the proposed improved YOLOv7 detection model as the target detector, detect the pedestrians in the video and mark the target box;

[0069] 2) Use the Kalman filter algorithm to estimate the state and trajectory of the pedestrian target bounding box, and cascade match the target detection result of the next frame with the current tracker. If it matches, continue tracking, otherwise proceed to the next step;

[0070] 3) According to the Hungarian algorithm, the unmatched target is matched with the unmatched tracker by IoU. If the match is successful, the tracking is maintained. The unmatched tracker is judged by the video disappearance frame threshold. If it is greater than 30 frames, the target is considered to have disappeared and the tracker is deleted. If it is less than 30 frames, the tracking state is maintained;

[0071] 4) The unmatched detection frame indicates the appearance of a new target, and then a new Kalman filter is initialized to continue tracking the next frame;

[0072] 5) Loop step (2) to step (4) to process all video images.

[0073] In this embodiment, in order to avoid interference from environmental conditions, the V2I communication range is set within 300m. When a vehicle enters the current area, the OBU device can receive the RSM information sent by the roadside system, and compare the pedestrian information with the driving information in the BSM to make a VRUCW collision warning judgment, and estimate the collision risk according to the Mazda collision model, and divide the warning level. Finally, an Android tablet device is selected as the UI device, and the front pedestrian warning is displayed on the tablet interface, and the voice broadcast reminds the driver to slow down, which improves driving safety more intuitively and efficiently.

[0074] More specifically, VRUCW refers to dangerous collision scenarios between the host vehicle (HV) and surrounding pedestrians (Pedestrians, P), as well as vulnerable traffic participants such as bicycles, electric vehicles, and animals.

[0075] Furthermore, the TA module of the OBU system conducts danger arbitration on VRUCW based on the input traffic participant information and determines the warning level of the collision. The distance d between the longitude and latitude coordinates of HV and P is calculated, and the safety distance strategy is used to divide the warning level according to the Mazda collision model strategy.

[0076]

[0077]

[0078] Formula (1) is the Haversine formula, which can be used to calculate the distance d between two points by using their longitude and latitude. (lon1, lat1) is the vehicle coordinate, (lon2, lat2) is the pedestrian coordinate, sin and cos are trigonometric functions, arcsin is the inverse sine function, and R is the radius of the earth. Note that the longitude and latitude in this formula must be converted into radians for calculation.

[0079] Formula (2) is the Mazda model formula, where D is the safety distance, which is equal to the braking distance plus the stopping reaction distance, v and a m is the current vehicle speed and maximum braking deceleration, ν r and a n are the speed and maximum braking deceleration of the target object, and t1 and t2 are the reaction times of the two objects. It is known that the maximum speed limit for urban road traffic is 30km / h (≈8.33m / s), the maximum braking deceleration of the car is 5m / s2, the moving speed of pedestrian targets is ignored, and the driver's reaction time is 1.5s, then it can be calculated. Therefore, this system sets the safety distance D=20m. When the distance between the vehicle and the pedestrian d>20m, the safety level is 2, there is no risk of collision, and only normal driving vehicles are prompted; when 10m<d<20m, the safety level is 1, there is a risk of collision, and the driver is reminded to slow down; when d<10m, the safety level is 0, the risk of collision is extremely high, and the driver is reminded to brake the vehicle. The above warning level judgment process is as follows Figure 6 shown.

[0080] Furthermore, the V2X Android application is selected as the terminal display on the Android tablet. The principle is to use WIFI to wirelessly connect to the on-board OBU device, receive the vehicle BSM information and pedestrian warning information transmitted by the DVIN module through the TCP communication protocol, and display this information on the Android app.

[0081] The display interface is mainly responsible for displaying the vehicle's driving status information and received warning information through animation, and in the case of emergency warnings, it uses voice broadcasts to remind the driver to slow down or stop. Figure 7As shown, the map information of the area where the vehicle is located is first displayed in the interface, and the map changes are displayed in real time with the current vehicle location as the center, marked with a green arrow, and the direction of the arrow is the direction of the vehicle. Adjacent vehicles are marked with orange cars, the current vehicle status information is in the middle, the warning voice broadcast switch is on the left, and the warning status bar and user configuration options are on the right, which are used to configure the IP connection on-board OBU and download offline maps of different regions to ensure real-time changes in the map during driving.

[0082] When a moving vehicle enters the communication range, the on-board OBU device obtains RSM wireless information in real time and performs collision warning calculations. First, the latitude and longitude information of the vehicle and the longitude and longitude information of the pedestrian are used to calculate the distance between the two. The calculation principle is based on the Haversine formula; then the TA module is used to perform VRUCW warning danger arbitration, and the warning strategy is based on the Mazda model; finally, the DVIN module is called to transmit the position of the pedestrian in the blind spot and the warning prompt to the tablet device for display. The warning display interface is as follows: Figure 8 As shown, the upper right corner of the APP not only warns of the location of dangerous pedestrians with a red sign, but also triggers a voice broadcast "Watch out for pedestrians 8m ahead", reminding the driver to slow down.

[0083] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A blind spot pedestrian warning method based on V2X and machine vision, characterized in that: The method comprises the following steps: S1. Start the V2X communication equipment, which includes: a roadside RSU equipment and an onboard OBU equipment; S2: The roadside RSU performs edge computing and starts detecting pedestrian information on the road; S3. If the roadside RSU detects a pedestrian on the current road, it tracks the pedestrian based on machine vision and calculates the exact position of the current pedestrian target based on the fusion positioning of radar and image information; S4, the roadside RSU broadcasts the pedestrian information in the blind spot to external traffic participants. When the moving vehicle enters the V2I communication range, the on-board OBU receives the pedestrian information on the road ahead; S5. When the vehicle-mounted OBU receives information about the pedestrian ahead, it calculates the collision warning based on the current vehicle's driving direction, speed, and acceleration, and determines the possibility of a collision. If no collision will occur, the vehicle continues to drive; S6. If the vehicle-mounted OBU determines that the warning conditions are met, the warning level is divided based on the Mazda collision model, and the front pedestrian collision warning prompt is displayed on the UI tablet connected to the vehicle-mounted OBU, and a voice broadcast reminds the driver to slow down.

2. According to claim 1, a blind spot pedestrian warning method based on V2X and machine vision is characterized in that: The pedestrian detection model in machine vision and the multi-sensor fusion positioning technology are deployed in the edge computing device. The edge computing device is connected to the camera and the radar through the serial port to obtain data information of the road environment and analyze it to obtain pedestrian information as the input of the RSU device; The edge computing process is: data-level fusion of the image detection results and the radar detection results, which involves three parts: pedestrian detection, pedestrian tracking, and pedestrian positioning; The process of detecting road pedestrian information is as follows: pedestrian detection adopts the YOLOv7 single-stage target detection algorithm based on machine learning, which can process video streams faster and more accurately and quickly detect and locate pedestrian targets in the video, and at the same time can optimize the detection effect of road-occluded pedestrians and small-target pedestrians.

3. The blind spot pedestrian warning method based on V2X and machine vision according to claim 1 is characterized in that: The process of tracking pedestrians based on machine vision is as follows: Kalman filter prediction is performed on the marked frame of the target detection result to obtain the predicted frame of the pedestrian target in the next frame, and then cascade matching is performed with the target frame after the actual pedestrian moves in the next frame. If the match is successful, it means that they are the same target, and the pedestrian target can be tracked.

4. The blind spot pedestrian warning method based on V2X and machine vision according to claim 1 is characterized in that: Based on the fusion positioning of radar and image information, the process of calculating the accurate position of the current pedestrian target is as follows: the image information is used for pedestrian detection and tracking through machine vision technology, and then the radar information is subjected to noise filtering and data correction. The two information are then fused at the sensor level, feature level and decision level to finally obtain the distance of the pedestrian and calculate the position in the spatial coordinate system. The specific position of the pedestrian is represented by longitude and latitude coordinates.

5. The blind spot pedestrian warning method based on V2X and machine vision according to claim 1 is characterized in that: The process of calculating the collision warning according to the current vehicle's driving direction, speed, and acceleration is as follows: the distance d between the longitude and latitude coordinates of HV and P is calculated by the longitude and latitude coordinates of HV and P; as follows: Among them, formula (1) is the Haversine formula, which can be used to calculate the distance d between two points by the longitude and latitude; where (lon1, lat1) is the driving coordinate, (lon2, lat2) is the pedestrian coordinate, sin and cos are trigonometric functions, arcsin is the inverse sine function, and R is the radius of the earth.

6. The blind spot pedestrian warning method based on V2X and machine vision according to claim 1 is characterized in that: The process of dividing the warning level based on the Mazda collision model is as follows: According to the Mazda collision model strategy, the warning level is divided using the safety distance strategy; as shown in the following formula: Among them, formula (2) is the Mazda model formula, where D is the safety distance, which is equal to the braking distance plus the stopping reaction distance, v and a m is the current vehicle speed and maximum braking deceleration, ν r and a n are the speed and maximum braking deceleration of the target object, and t1 and t2 are the reaction times of the two objects.

7. The blind spot pedestrian warning method based on V2X and machine vision according to claim 1 is characterized in that: The process of displaying the front pedestrian collision warning prompt on the UI tablet connected to the vehicle OBU and the voice broadcast reminding the driver to slow down is as follows: select the V2X Android application on the Android tablet as the terminal display, use WIFI to wirelessly connect the vehicle OBU device, receive the vehicle BSM information and pedestrian warning information transmitted by the DVIN module through the TCP communication protocol, and display this information on the Android app; The terminal display interface is used to animate the vehicle's driving status information and received warning information, and to use voice broadcasts to remind the driver to slow down or stop in emergency warning situations.

8. A blind spot pedestrian warning system based on V2X and machine vision, characterized in that: The system is used to implement a blind spot pedestrian warning method based on V2X and machine vision as described in any one of claims 1 to 7, and the system includes a roadside system and a vehicle-mounted system; The roadside system includes: edge computing equipment, roadside sensors, and roadside RSU equipment; The vehicle-mounted system includes: a vehicle-mounted OBU device, a vehicle-mounted controller, and a UI interaction device.

9. The blind spot pedestrian warning system based on V2X and machine vision according to claim 8, characterized in that: The edge computing device is used to detect and record the movement trajectory and location information of pedestrians; The roadside sensor is used to obtain the perception information of the surrounding environment, and filter and fuse the perception information to transmit to the edge computing device; The roadside RSU device is used to transmit the movement trajectory, speed, location and other information of the pedestrian to other traffic participants in a broadcast form; The vehicle-mounted OBU device is used to receive roadside safety information transmitted by the roadside RSU device, and obtain the vehicle speed, position and other status information according to the vehicle interface service module; perform collision warning calculations for vulnerable traffic participants according to the driving speed, position and pedestrian information, and divide the warning level; The on-board controller is used to adopt a corresponding warning strategy according to the collision warning level calculated by the on-board OBU device; The UI interaction device obtains the human-computer interaction interface information of the vehicle-mounted OBU device through WLAN, and is used to convert the GNSS information of the pedestrians and vehicles ahead into map positions and display them dynamically in real time.

Images