A bus blind area pedestrian detection device with risk avoidance function

Abstract

A pedestrian detection device for blind spots on buses with risk avoidance function includes a powered telescopic frame, a pedestrian recognition module, an audible and visual alarm, an environmental recognition module, a geographic location recognition module, an emergency avoidance module, and an embedded microcontroller. The powered telescopic frame is installed on the front roof of the bus. The pedestrian recognition module, environmental recognition module, and emergency avoidance module are all mounted on the powered telescopic frame. The audible and visual alarm and the embedded microcontroller are both installed on the interior ceiling of the bus, directly below the powered telescopic frame. The geographic location recognition module is mounted on the embedded microcontroller. The embedded microcontroller is connected to the powered telescopic frame, pedestrian recognition module, audible and visual alarm, environmental recognition module, geographic location recognition module, and emergency avoidance module via signals. This invention can effectively identify pedestrians in the blind spots of buses, alert the driver to driving safety through the audible and visual alarm to avoid traffic accidents, and also has risk avoidance and emergency avoidance functions.

Description

Technical Field

[0001] This invention relates to the field of driver assistance technology, and more specifically, to a pedestrian detection device for blind spots on buses with risk avoidance function. Background Technology

[0002] Buses, as large vehicles operating on city streets, have significant blind spots and are often obstructed by other vehicles. When starting in the driving lane, they cannot clearly see the movement of pedestrians in both lanes, making it difficult to brake in time and potentially leading to accidents. Due to the complex road traffic conditions in cities, an auxiliary device attached to the front of the bus is needed. This device would assist the driver in judging the situation of pedestrians in blind spots while also helping to avoid collisions and damage to the auxiliary device itself.

[0003] Application number "202210489436.6" discloses a pedestrian recognition method based on YOLOv5. This method uses a camera installed on a passenger vehicle to collect road images, identifies the images of the vehicle's blind spots through an in-vehicle terminal in the driver's cab, and tracks the pedestrian's location information. When a pedestrian is in a risk area of ​​the blind spot, an alarm message is issued to remind the driver to pay attention to avoid the pedestrian and prevent traffic accidents.

[0004] Application number "201822174302.7" discloses a vehicle blind spot detection alarm system. The system uses a pedestrian recognition sensor installed at the front of a large vehicle to identify the location of pedestrians in the blind spot. When the vehicle stops moving, the pedestrian detection device starts to work to detect whether there are pedestrians in the blind spot. When a pedestrian is detected, the daytime running lights on both sides of the vehicle flash to remind the driver that there are pedestrians in the blind spot, thereby avoiding the occurrence of safety accidents.

[0005] Based on existing patents, most patents that assist driving by detecting pedestrians in blind spots are only applicable to passenger cars. They do not take into account the need for additional devices to expand the camera's shooting range and cover the driver's blind spot for large vehicles. Although application number "201822174302.7" is applied to large vehicles, it relies on pedestrian recognition sensors for pedestrian detection in the blind spot of large vehicles. This method is prone to missed detections and cannot achieve detection during the movement of large vehicles. Summary of the Invention

[0006] The purpose of this invention is to provide a pedestrian detection device for blind spots on buses with risk avoidance function. This invention can effectively identify pedestrians in blind spots on buses, remind drivers to pay attention to driving safety through sound and light alarms to avoid traffic accidents, and also has risk avoidance and emergency avoidance functions.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A pedestrian detection device for blind spots on buses with risk avoidance function includes a power-operated telescopic frame, a pedestrian recognition module, an audible and visual alarm, an environmental recognition module, a geographic location recognition module, an emergency avoidance module, and an embedded microcontroller. The power-operated telescopic frame is installed on the outer roof of the front of the bus. The pedestrian recognition module, environmental recognition module, and emergency avoidance module are all mounted on the power-operated telescopic frame. The audible and visual alarm and the embedded microcontroller are both installed on the ceiling inside the bus and located directly below the power-operated telescopic frame. The geographic location recognition module is mounted on the embedded microcontroller. The embedded microcontroller is connected to the power-operated telescopic frame, the pedestrian recognition module, the audible and visual alarm, the environmental recognition module, the geographic location recognition module, and the emergency avoidance module via signals.

[0009] The power telescopic frame includes a front and rear telescopic mechanism and a lifting mechanism. The front and rear telescopic mechanism is set above the front roof of the bus in the front-rear direction. The rear end of the front and rear telescopic mechanism is rotatably mounted on the roof of the bus. The lifting mechanism is mounted on the front roof of the bus and located below the front and rear telescopic mechanism. The upper end of the lifting mechanism is connected to the lower part of the front and rear telescopic mechanism and drives the front and rear telescopic mechanism to lift.

[0010] The telescopic mechanism includes a telescopic base, a mover seat, a mover, and a telescopic rod. The telescopic base is positioned above the front roof of the bus along the front direction. Two bearing seats are fixedly installed on the front roof of the bus, spaced apart on the left and right sides of the rear end of the telescopic base. Horizontal rotating shafts are respectively installed on the left and right sides of the rear end of the telescopic base, and the two horizontal rotating shafts are respectively installed on the aforementioned two bearing seats. A guide groove is formed in the middle of the upper part of the telescopic base along the front direction, and two linear guide rails are also provided on the upper surface of the telescopic base along the front direction. Two linear guide rails are symmetrically arranged on the left and right sides of the guide groove, spaced apart. The mover seat is slidably mounted on the two linear guide rails. The mover is fixedly mounted on the lower side of the mover seat and slidably embedded in the guide groove. Several stators, which are rectangular permanent magnets, are fixedly mounted at equal intervals at the bottom of the guide groove. The stators are covered with a cover plate. The mover has a coil inside. The mover seat is provided with a wire interface for connecting to the mover wire. The wire interface is connected to an external wire to provide power to the mover. A telescopic base is fixedly mounted on the upper middle part of the front side. A front baffle is positioned at the front end of the guide chute, and a rear baffle is positioned at the upper middle part of the rear side of the telescopic base, positioned at the rear end of the guide chute. The upper sides of both the front and rear baffles are higher than the upper side of the linear guide rail. A guide hole with front and rear penetration is provided in the middle of the front baffle, and the telescopic rod is installed through the guide hole in the front-rear direction. The rear end of the telescopic rod is fixedly installed in the middle of the front side of the moving base, and a camera bracket located in front of the front baffle is fixedly installed at the front end of the telescopic rod. Anti-collision devices located within the guide chute are fixedly installed on the left and right sides of the rear side of the front baffle and the middle of the front side of the rear baffle. The device, the anti-collision device is a cylindrical pad made of rubber or sponge. Two limiters with front and rear spacing are fixedly installed on the upper left edge of the telescopic base. The front limiter is located in front of the moving base and close to the front baffle, and the rear limiter is located in rear of the moving base and close to the rear baffle. The height of the limiters is lower than the bottom of the moving base. The contact plate of the limiter is located on the top of the limiter. When the moving base slides on the linear guide rail and passes over the limiter, the bottom of the moving base presses the contact plate of the limiter and closes the contact plate of the limiter. The embedded microcontroller is connected to the moving base and the limiter signals respectively.

[0011] The lifting mechanism includes a base, a top seat, two first connecting rods, two second connecting rods, a rotary reduction motor, a lead screw, and a nut. Both the base and top seat are rectangular frame structures, positioned vertically below the telescopic base. The top seat is fixedly installed on the lower surface of the telescopic base, and the base is fixedly installed on the roof of the bus's front end. The two first connecting rods are spaced horizontally and inclined downwards from the front to the rear between the base and the top seat. Similarly, the two second connecting rods are also spaced horizontally and inclined downwards from the front to the rear between the base and the top seat. The first and second connecting rods are staggered horizontally. Two bottom guide rails, spaced horizontally and arranged along the front-rear direction, are fixedly installed on the base, each corresponding vertically to one of the two first connecting rods. Two top guide rails, spaced horizontally and arranged along the front-rear direction, are fixedly installed on the top seat, each corresponding vertically to one of the two second connecting rods. The front ends of the two first connecting rods are hinged to the lower front side of the top seat, and the rear ends of the two first connecting rods are rotatably connected. There are two first rotating wheels, which are respectively rolled and connected to two bottom guide rails. The front ends of two second connecting rods are hinged to the upper front part of the base, and the rear ends of the two second connecting rods are rotatably connected to second rotating wheels. The two second rotating wheels are respectively rolled and connected to two top guide rails. A horizontal support plate is fixedly connected to the front part of the base. Ear plate supports located between the two bottom guide rails are fixedly installed on the rear part of the horizontal support plate and the rear side frame of the base. The two ear plate supports are corresponding to each other. A rotary gear motor is fixedly installed on the horizontal support plate and located in front of the front ear plate support. A lead screw is set along the front-back direction and its two ends are respectively rotatably installed on the two ear plate supports. The motor shaft of the rotary gear motor is coaxially connected to the front end of the lead screw. A nut is threaded on the lead screw and is located between the two first rotating wheels. A linkage shaft is fixedly connected between the nut and the central shaft of the two first rotating wheels. An embedded microcontroller is connected to the rotary gear motor for signal transmission.

[0012] The pedestrian recognition module includes two pedestrian recognition cameras, which are infrared cameras, and the two pedestrian recognition cameras are respectively set on the left and right sides of the camera bracket.

[0013] The environmental recognition module includes an environmental recognition camera, which is positioned in the middle of the camera bracket;

[0014] The geolocation identification module includes a GPS locator, which is installed on an embedded microcontroller;

[0015] The emergency avoidance module includes a collision sensor, which is installed at the front end of the telescopic pole and has a foam pad on its exterior.

[0016] The embedded microcontroller is connected to the pedestrian recognition camera, the environmental recognition camera, the GPS locator, and the collision sensor, respectively.

[0017] Using the above technical solution, a method for detecting pedestrians in blind spots on buses with risk avoidance function specifically includes the following steps:

[0018] (i) Install the aforementioned pedestrian detection device with risk avoidance function in the blind spot of the bus above the front of the bus;

[0019] (ii) When the bus is moving normally, the power telescopic frame is in working condition. The embedded microcontroller acquires the blind spot image of the bus collected by the pedestrian recognition module. At the same time, the bus blind spot pedestrian detection device with risk avoidance function automatically avoids collision risk and emergency avoidance.

[0020] (III) The embedded microcontroller detects the blind spot image and obtains the real-time location information of the target pedestrian;

[0021] (iv) When the target pedestrian's current location is determined to be within the blind spot of the bus based on the target pedestrian's current location information, the embedded microcontroller controls the sound and light alarm to output alarm information.

[0022] (v) When the location of the target pedestrian is determined to be outside the blind spot of the bus based on the current location information of the target pedestrian, the embedded microcontroller uses a Kalman filter to predict the trajectory of the target pedestrian.

[0023] (vi) Based on the prediction information, when the trajectory of the target pedestrian enters the blind spot of the bus, the embedded microcontroller controls the sound and light alarm to output alarm information.

[0024] Step (II) is as follows: While the bus is in motion, the power telescopic frame is in working condition: the two first connecting rods and the two second connecting rods lift the top seat, so that the telescopic base is at the normal working lifting angle. At the same time, the mover is connected to the external wire through the wire interface. The embedded microcontroller controls the current direction of the coil inside the mover, so that the magnetic field generated by the mover interacts with the magnetic field of the stator to generate thrust. The mover drives the mover seat to slide forward along the linear guide rail to the front end of the linear guide rail. The mover seat is blocked and limited by the front baffle. The telescopic rod extends forward completely under the action of the mover seat. The two pedestrian recognition cameras extend forward and upward to the front and top of the bus. The two pedestrian recognition cameras capture images of the blind spots on both sides of the front of the bus in real time and transmit the captured blind spot images to the embedded microcontroller. When the driving environment is dark, such as rainy days, nights, tunnels, etc., the pedestrian recognition cameras automatically turn on the infrared mode to ensure that the captured blind spot images are clear enough to be used for target pedestrian detection.

[0025] While the bus is in motion, the pedestrian detection device in the blind spot of the bus with risk avoidance function automatically avoids collision risks in the following two working modes:

[0026] The first working mode utilizes an environmental recognition camera to automatically avoid collision risks;

[0027] The second working mode uses a GPS locator to automatically avoid collision risks;

[0028] The bus blind spot pedestrian detection device with risk avoidance function realizes emergency avoidance by processing collision information collected by collision sensors.

[0029] Step (3) is as follows: The embedded microcontroller uses the YOLOv5s model of the YOLOv5 algorithm to detect the blind spot image of the bus, obtain the current target box corresponding to the target pedestrian, and obtain the current position of the target pedestrian. In reality, there may be multiple pedestrians around the bus. Therefore, when the embedded microcontroller detects the blind spot image, it can obtain multiple current position information corresponding to multiple pedestrians. Among them, the target pedestrian refers to any one of the multiple pedestrians.

[0030] Step (four) specifically involves: Due to the obstruction of the bus's blind spot image by other moving vehicles, pedestrians may suddenly appear within the bus's blind spot. When the target pedestrian's location information, processed by the YOLOv5 algorithm, appears within the pedestrian recognition camera's field of view and is located within the bus's blind spot, the pedestrian recognition camera's field of view completely covers the bus's blind spot. The bus's blind spot is customized based on different vehicle models and operating conditions; that is, users can pre-determine the bus's blind spot range through the images captured by the pedestrian recognition camera. The defined blind spot range is stored in the configuration file of the embedded microcontroller. The blind spot is further divided into high-risk and low-risk areas according to the distance from the bus. When a pedestrian is detected in a low-risk area, the embedded microcontroller controls the audible and visual alarm to operate at a low frequency and issue a warning to remind the driver that a pedestrian has entered the blind spot. When a pedestrian is detected in a high-risk area, the embedded microcontroller controls the audible and visual alarm to operate at a high frequency. The audible and visual alarm increases the flashing frequency of the lights and the decibel of the alarm bell to remind the driver to pay attention to the location of the pedestrian in the blind spot, so as to avoid possible traffic accidents.

[0031] Step (5) specifically involves the following: When the location information of the target pedestrian obtained through the YOLOv5 algorithm appears within the shooting range of the pedestrian recognition camera and is not located in the blind spot of the bus, it indicates that the pedestrian is not in a dangerous situation. At this time, the embedded microcontroller synchronizes the target pedestrian location information and target box information detected by YOLOv5 to the DeepSORT target tracking algorithm. The DeepSORT target tracking algorithm predicts the target pedestrian trajectory through a Kalman filter. The DeepSORT target tracking algorithm predicts the target pedestrian trajectory by including the following steps:

[0032] (1) Motion state prediction: The Kalman filter predicts the position of the target pedestrian in the next frame based on the position of the target pedestrian in the current frame. At this time, the predicted target pedestrian trajectory is in an unconfirmed state. In any video frame, an eight-dimensional state vector X = [u, v, r, h, u', v', r', h'] is used as the model for predicting the target pedestrian trajectory. [u, v] represent the horizontal and vertical positions of the target bounding box in the picture, respectively. [r, h] represent the aspect ratio and height of the target bounding box, respectively. The other four parameters are the corresponding velocity information.

[0033] (2) Motion state update: IOU matching is performed on the unconfirmed target pedestrian trajectory to obtain the optimal result, which has three possible outcomes:

[0034] When the matching result is a trajectory mismatch, that is, the target pedestrian has a trajectory but no corresponding detection box, if its frame number N is less than the maximum number of lost frames 30, it is retained in the tracking chain; otherwise, the target is deleted.

[0035] When the matching result is a detection mismatch, that is, the target pedestrian has a detection box but no corresponding trajectory, it is determined to be a new target pedestrian, and the trajectory of the target pedestrian is created at this time;

[0036] When the matching result is a successful match, the trajectory of the target pedestrian is updated using a Kalman filter. At this time, the predicted trajectory of the target pedestrian is in the confirmed state.

[0037] (3) Cascaded matching: The confirmed target pedestrian trajectory and its corresponding time parameter are cascaded and matched. Each time parameter has its corresponding tracked target pedestrian trajectory. Each target pedestrian trajectory is assigned a priority parameter. If the target pedestrian trajectory fails to match, the time parameter of the target pedestrian trajectory is incremented by 1, otherwise it is set to 0. The smaller the time parameter of the target pedestrian trajectory, the higher its priority. The target pedestrian trajectory that is matched first in the previous frame is given a high priority. Conversely, the target pedestrian trajectory with a larger time parameter means that the target pedestrian trajectory has a lower matching probability. For target pedestrian trajectories that have not been matched for several consecutive frames, their priority will be gradually reduced and eventually the target pedestrian trajectory will be deleted. At this time, the trajectory prediction of the target pedestrian is completed.

[0038] The specific process by which the bus blind spot pedestrian detection device with risk avoidance function uses an environmental recognition camera to automatically avoid collision risks is as follows:

[0039] Step 1: Training Image Acquisition: The embedded microcontroller acquires images of the target area along the fixed route of the bus through an environmental recognition camera, or directly transmits pre-captured target area images to the embedded microcontroller as training images. The target area includes the retractable pole area and the extended pole area. The retractable pole area is the area where the extended pole may pose a collision risk during the bus's movement, and the extended pole area is the area where the bus may pose a collision risk after leaving the extended pole. The training images include images of the retractable pole area and the extended pole area. The training images are taken along the fixed route of the bus, with 20 to 50 images taken at each shooting location.

[0040] Step 2: Extracting SURF Features: The embedded microcontroller uses an improved SURF algorithm to perform feature recognition on the grayscale training images. The recognition process is as follows:

[0041] (A) Generating the Hessian matrix: The Hessian matrix is ​​the core of the improved SURF algorithm. Its function is to generate stable edge points in the image, which is the basis for feature extraction. A Hessian matrix can be calculated for each pixel in the image. In the SURF algorithm, since the feature points need to be scale-independent during the feature extraction process, each pixel in the image needs to be filtered before constructing the Hessian matrix to eliminate the correlation between pixels. The box filter is used for this purpose.

[0042] (B) Constructing scale space: By changing the size of the box filter, the original image is convolved with different sizes of filters in different directions to form a multi-scale space and extract image features around the pixels.

[0043] (C) Locating feature points: Each pixel processed by the Hessian matrix is ​​compared with points in the two-dimensional image space and scale space neighborhood to initially locate key points. Then, key points with weak energy and incorrectly located key points are filtered out to select the final stable key points as feature points.

[0044] (D) Selecting the main direction of the feature point: Taking the feature point as the center, a circular neighborhood of the feature point is obtained with a radius of 6δ. Taking 45° sectors as units, the cumulative values ​​of all pixels in the sector for the Haar wavelet are counted and a feature sub-vector is obtained. The size of the Haar wavelet is 2δ×2δ. Then, the 45° sector is rotated at 45° intervals. After one rotation, eight 45° sector regions are obtained, resulting in eight Haar wavelet cumulative values ​​and eight feature sub-vectors. The eight Haar wavelet cumulative values ​​are compared, and the feature sub-vector direction corresponding to the largest Haar wavelet cumulative value is selected as the main direction.

[0045] (E) Generate feature descriptors: Starting from the feature vector corresponding to the main direction, take the feature vectors of 8 sector regions in order of decreasing Haar response accumulation value to obtain a 4×8=32-dimensional feature descriptor;

[0046] Step 3: Generate an offline visual bag of words:

[0047] The K-means clustering algorithm is used to cluster all training images by features. Then, several cluster centers are used as visual words to generate offline visual bag of words for the target region. The offline visual bag of words histogram is obtained and normalized. To adapt to the matching needs of different urban road environments and the actual application of users, the range of cluster center k is set from 150 to 250. After considering the precision and retrieval efficiency, the cluster center k is set to 200. The cluster center k is the size of the offline visual bag of words.

[0048] Step 4: Input real-time traffic images:

[0049] The environmental recognition camera inputs real-time environmental images of the bus during its journey into the embedded microcontroller. The camera's shooting frequency is 1Hz to 10Hz. The embedded microcontroller uses the same bag-of-words visual algorithm as described in step three above to process the input real-time environmental images, generate a bag-of-words visual model of the real-time environmental images, and obtain a real-time bag-of-words histogram.

[0050] Step 5: Calculate similarity:

[0051] The offline visual bag-of-words histogram is compared with the real-time visual bag-of-words histogram, and the similarity between the offline and real-time visual bag-of-words histograms is calculated. If the calculated similarity is less than a preset threshold, the process returns to the previous step to continue collecting real-time traffic images. If the similarity is greater than the preset threshold, SVM is used to classify the environmental images at this moment and output the image with the highest similarity in the offline image library.

[0052] The formula for calculating similarity is:

[0053]

[0054] Where H1 is the offline visual bag-of-words histogram; H2 is the real-time visual bag-of-words histogram; H1(i) represents the size of the i-th cluster center in the offline visual bag-of-words histogram; H2(i) represents the size of the i-th cluster center in the real-time visual bag-of-words histogram; k is the number of cluster centers, i.e. the size of the bag-of-words. The mean of the offline visual bag-of-words histogram; The mean of the real-time visual bag-of-words histogram;

[0055] The formulas for calculating the mean of the offline visual bag-of-words histogram and the mean of the histogram are as follows:

[0056]

[0057]

[0058] Step 6, SVM Classification:

[0059] The system uses SVM to classify real-time environmental images and outputs the classification results. When the embedded microcontroller identifies that the image obtained through SVM classification belongs to the retractable pole area, it controls the mover to slide the mover seat backward along the linear guide rail to the rear end of the linear guide rail. The mover seat is blocked and limited by the rear baffle. The telescopic pole is fully retracted backward under the action of the mover seat to avoid collision risk. When the telescopic pole has been retracted, the powered telescopic frame maintains its self-protection state and returns to step five above to continue processing real-time road condition image information. When the embedded microcontroller identifies that the image obtained through SVM classification belongs to the extended telescopic pole area, it controls the mover to slide the mover seat forward along the linear guide rail to the front end of the linear guide rail. The mover seat is blocked and limited by the front baffle. The telescopic pole is fully extended forward under the action of the mover seat to continue blind spot pedestrian detection. When the telescopic pole has been extended, the powered telescopic frame maintains its working state and returns to step five above to continue processing real-time road condition image information.

[0060] The specific process by which the bus blind spot pedestrian detection device with risk avoidance function uses a GPS locator to automatically avoid collision risks is as follows:

[0061] Step 1: Collect location information of risk areas along bus routes:

[0062] GPS data information, including longitude and latitude, is collected from locations along the bus route where there is a risk of collision. This GPS data information is stored in an embedded microcontroller. The locations where there is a risk of collision are the locations where the telescopic pole is at risk of collision.

[0063] Step 2: Receive real-time location information from the GPS locator:

[0064] When a bus is traveling on a predetermined route, the GPS locator receives satellite positioning information and transmits the bus's location information to an embedded microcontroller. The embedded microcontroller receives the location information from the GPS locator at a frequency of 2Hz.

[0065] Step 3: Determine the distance between the real-time location and any risk area:

[0066] Each time the embedded microcontroller receives real-time bus location information, it compares the real-time location information with the location information of any risk location stored inside the embedded microcontroller to determine whether the distance between the real-time location of the bus and any risk location is less than the set distance threshold.

[0067] Step 4: Identify the risk area and retract the telescopic pole:

[0068] When the distance between the real-time location of the bus and any risk location calculated by the embedded microcontroller is less than or equal to a preset distance threshold, the embedded microcontroller controls the power telescopic frame to maintain or switch to a self-protection state, retracts the telescopic rod to pause the pedestrian detection task in the blind spot of the bus driver, and returns to the third step above to continue calculating the distance between the real-time location of the bus and any risk location.

[0069] Step 5: Once the bus has left the risk area, the powered telescopic shield will return to working condition.

[0070] When the distance between the real-time location of the bus and any risk location calculated by the embedded microcontroller is greater than the preset distance threshold, the embedded microcontroller controls the power telescopic frame to maintain or switch to working state, extends the telescopic rod forward to continue the pedestrian detection task in the blind spot of the bus driver, and returns to the third step above to continue calculating the distance between the real-time location of the bus and any risk location.

[0071] The distance between the bus's real-time location and any risk location is calculated using the following method: Assuming the Earth is a perfect sphere with radius R, and that east longitude is positive, west longitude is negative, north latitude is positive, and south latitude is negative, then the bus's real-time coordinates A(x, y) are represented as:

[0072]

[0073] The coordinates B(a, b) of any risk location are represented as:

[0074]

[0075] The formula for determining the distance between a bus's real-time location and any risk location is:

[0076]

[0077] The distance threshold S is set by the manufacturer at the factory or by the bus driver based on the specific road conditions. Here, the distance threshold S is set to a range of 15 to 50 meters, taking into account the road conditions and speed information of buses in the city.

[0078] The specific process of the bus blind spot pedestrian detection device with risk avoidance function using collision sensors to achieve emergency avoidance is as follows:

[0079] Step 1: A collision was detected with the telescopic pole.

[0080] Due to urban road construction and tree growth resulting in excessively low tree branches, new unrecorded risk points appeared on the bus route. When the bus passed through these areas, the moving part failed to retract the telescopic rod, causing a collision. When the collision sensor detected the pressure change, it output the collision information and transmitted it to the embedded microcontroller.

[0081] Step 2: Emergency escape using an embedded microcontroller-controlled telescopic frame.

[0082] When the embedded microcontroller receives a collision signal, it controls the front and rear telescopic mechanism to retract the telescopic rod and controls the rotary reduction motor to drive the lead screw to rotate. The lead screw drives the nut to move backward, and the nut drives the rear ends of the two first connecting rods to move backward through the linkage shaft. Then, the two first connecting rods and the two second connecting rods pull the top seat down, causing the top seat to drive the telescopic base to descend to an angle parallel to the roof of the bus. This state is maintained, and the blind spot pedestrian recognition work is stopped.

[0083] This invention has outstanding substantive features and significant progress compared to the prior art. Specifically, the beneficial effects of this invention are as follows:

[0084] (1) The present invention adds a power telescopic frame to the front of the bus and installs a pedestrian recognition camera on the power telescopic frame to collect images of the blind spot of the bus during the bus's movement. This can effectively avoid obstruction from vehicles on the left and right sides. By detecting the blind spot images of the bus, the location of the target pedestrian in the image can be effectively identified. When it is determined that the target pedestrian is located in the blind spot of the bus, the embedded microcontroller controls the sound and light alarm to issue an alarm message to remind the driver to pay attention to driving safety and avoid traffic accidents.

[0085] (2) The bus blind spot pedestrian detection device with risk avoidance function in this invention has a self-protection function. It can automatically switch between working state and protection state of the power telescopic frame through environmental recognition camera and GPS locator. When the bus enters the risk area, the telescopic rod is retracted to avoid collision accident. When the bus leaves the risk area, the telescopic rod is extended to continue the blind spot pedestrian detection work.

[0086] (3) The pedestrian detection device for blind spots in buses with risk avoidance function in this invention also has an emergency avoidance function. A collision sensor is installed at the front end of the telescopic pole. When the collision sensor collides, it transmits the collision information to the embedded microcontroller. The embedded microcontroller controls the action of the power telescopic frame to retract the telescopic pole in an emergency to avoid more serious damage.

[0087] In summary, this invention can effectively identify pedestrians in blind spots on buses, remind drivers to pay attention to driving safety through sound and light alarms to avoid traffic accidents, and also has risk avoidance and emergency avoidance functions. Attached Figure Description

[0088] Figure 1 This is a schematic diagram of the installation of the present invention on a bus. Figure 1 .

[0089] Figure 2 This is a schematic diagram of the installation of the present invention on a bus. Figure 2 .

[0090] Figure 3 This is an isometric view of the present invention, wherein the cover plate is a partially sectional structure.

[0091] Figure 4 This is a top view of the present invention.

[0092] Figure 5 This is a schematic diagram of the lifting mechanism of the present invention.

[0093] Figure 6 This is a block diagram of the control structure of the present invention.

[0094] Figure 7 This is a flowchart illustrating the process of pedestrian detection in blind spots according to the present invention.

[0095] Figure 8 This is a schematic diagram of the shooting range of the pedestrian recognition camera of the present invention.

[0096] Figure 9 This is a flowchart illustrating the workflow of a pedestrian detection device for blind spots on buses with risk avoidance function, which utilizes an environmental recognition camera to automatically avoid collision risks.

[0097] Figure 10 This is a schematic diagram of the feature point neighborhood partitioning in the improved SURF algorithm.

[0098] Figure 11 This is a flowchart illustrating the workflow of a pedestrian detection device for blind spots on buses with risk avoidance function, which utilizes a GPS locator to automatically avoid collision risks.

[0099] Figure 12 This is a flowchart illustrating the emergency avoidance process of a pedestrian blind spot detection device for buses with risk avoidance function in the event of a collision, according to the present invention. Detailed Implementation

[0100] The embodiments of the present invention are further described below with reference to the accompanying drawings.

[0101] like Figure 1-12As shown, a pedestrian detection device for blind spots on a bus with risk avoidance function includes a power telescopic frame, a pedestrian recognition module, an audible and visual alarm 3, an environmental recognition module, a geographic location recognition module, an emergency avoidance module, and an embedded microcontroller 4. The power telescopic frame is installed on the outer roof of the front of the bus 1. The pedestrian recognition module, the environmental recognition module, and the emergency avoidance module are all installed on the power telescopic frame. The audible and visual alarm 3 and the embedded microcontroller 4 are both installed on the inner ceiling of the bus 1 and located directly below the power telescopic frame. The geographic location recognition module is installed on the embedded microcontroller 4. The embedded microcontroller 4 is connected to the power telescopic frame, the pedestrian recognition module, the audible and visual alarm 3, the environmental recognition module, the geographic location recognition module, and the emergency avoidance module via signals.

[0102] The power telescopic frame includes a front and rear telescopic mechanism and a lifting mechanism. The front and rear telescopic mechanism is set above the front roof of the bus 1 in the front-rear direction. The rear end of the front and rear telescopic mechanism is rotatably mounted on the roof of the bus 1. The lifting mechanism is mounted on the front roof of the bus 1 and located below the front and rear telescopic mechanism. The upper end of the lifting mechanism is connected to the lower part of the front and rear telescopic mechanism and drives the front and rear telescopic mechanism to lift.

[0103] The telescopic mechanism includes a telescopic base 5, a mover seat 6, a mover 7, and a telescopic rod 8. The telescopic base 5 is installed above the front roof of the bus 1 along the front-rear direction. Two bearing seats 9 are fixedly installed on the front roof of the bus 1. The two bearing seats 9 are spaced apart on the left and right sides of the rear end of the telescopic base 5. Horizontal rotating shafts are respectively installed on the left and right sides of the rear part of the telescopic base 5, and the two horizontal rotating shafts are respectively installed on the aforementioned two bearing seats 9. A guide groove 10 is opened in the middle of the upper part of the telescopic base 5 along the front-rear direction. Two linear guide rails 11 are also provided on the upper surface of the telescopic base 5 along the front-rear direction. The two linear guide rails 11 are spaced apart and symmetrically arranged on the left and right sides of the guide groove 10. The mover base 6 is slidably mounted on two linear guide rails 11. The mover 7 is fixedly mounted on the lower side of the mover base 6 and slidably embedded in the guide groove 10. Several stators 12, which are rectangular permanent magnets, are fixedly mounted at equal intervals at the bottom of the guide groove 10. The stators 12 are covered with a cover plate 13 (to protect the stators 12). The mover 7 has a built-in coil. The mover base 6 is provided with a wire interface 2 for connecting the wires of the mover 7. The wire interface 2 is connected to an external wire to provide power to the mover 7. A front baffle 14 is fixedly mounted on the upper middle part of the front side of the telescopic base 5, blocking the front end of the guide groove 10. A baffle 14 is fixed on the upper middle part of the rear side of the telescopic base 5, blocking the rear end of the guide groove 10. The upper sides of the rear baffle 15, front baffle 14, and rear baffle 15 are all higher than the upper side of the linear guide rail 11. The front baffle 14 has a guide hole that is open from front to back in the middle. The telescopic rod 8 is installed through the guide hole in the front-back direction. The rear end of the telescopic rod 8 is fixedly installed in the middle of the front side of the moving base 6. The front end of the telescopic rod 8 is fixedly installed with a camera bracket 16 located in front of the front baffle 14. The left and right sides of the rear side of the front baffle 14 and the middle of the front side of the rear baffle 15 are all fixedly installed with anti-collision devices 17 located in the guide groove 10. The anti-collision device 17 is a cylindrical pad made of rubber or sponge. (The anti-collision device 17 absorbs the kinetic energy of the moving base 6 through deformation, so that the moving base 6 decelerates and stops, while preventing...) (When the moving base 6 moves, it will experience a rigid collision). The embedded microcontroller 4 is connected to the moving base 7 via signal. Two limiters 36 with front and rear spacing are also fixedly installed on the upper left edge of the telescopic base 5. The front limiter 36 is located in front of the moving base 6 and close to the front baffle 14. The rear limiter 36 is located in rear of the moving base 6 and close to the rear baffle 15. The height of the limiter 36 is lower than the bottom of the moving base 6. The contact piece of the limiter 36 is set on the top of the limiter 36. When the moving base 6 slides on the linear guide rail 11 and passes above the limiter 36, the bottom of the moving base 6 presses the contact piece of the limiter 36 and closes the contact piece of the limiter 36. The embedded microcontroller 4 is connected to the moving base 7 and the limiter 36 via signal respectively.When the moving base 6 moves forward or backward, and passes above a corresponding limiter 36, the moving base 6 presses down the contact of the limiter 36, closing the contact. The embedded microcontroller 4 then receives the closing signal from the limiter 36 and controls the moving rotor 7 to lose power. The moving rotor 7 then drives the moving base 6 to slide. The remaining travel of the moving base 6 is used to dissipate its remaining kinetic energy. If the moving base 6's speed is too high, it will continue to decelerate due to inertia. If the moving base 6's speed is not zero when it slides forward or backward to its limit position, it will be blocked and limited by the front or rear anti-collision device 17.

[0104] The lifting mechanism includes a base 18, a top seat 19, two first connecting rods 20, two second connecting rods 21, a rotary reduction motor 22, a lead screw 23, and a nut 24. Both the base 18 and the top seat 19 are rectangular frame structures. The top seat 19 and the base 18 are vertically spaced below the telescopic base 5. The top seat 19 is fixedly installed on the lower surface of the telescopic base 5, and the base 18 is fixedly installed on the roof of the bus 1. The two first connecting rods 20 are spaced horizontally and inclined downwards from front to back between the base 18 and the top seat 19. The two second connecting rods 21 are spaced horizontally and inclined downwards from front to back. The first connecting rod 20 and the second connecting rod 21 are offset to each other and are diagonally positioned between the base 18 and the top seat 19. Two bottom guide rails 25, spaced apart laterally and arranged along the front-back direction, are fixedly mounted on the base 18. Each of the two bottom guide rails 25 corresponds vertically to one of the two first connecting rods 20. Two top guide rails 26, spaced apart laterally and arranged along the front-back direction, are fixedly mounted on the top seat 19. Each of the two top guide rails 26 corresponds vertically to one of the two second connecting rods 21. The front ends of the two first connecting rods 20 are hinged to the lower front side of the top seat 19, and the rear ends of both first connecting rods 20 are rotatably connected. The base 18 is equipped with two first rotating wheels 27, which are respectively rolled on two bottom guide rails 25. The front ends of two second connecting rods 21 are hinged to the upper front side of the base 18, and the rear ends of the two second connecting rods 21 are rotatably connected to second rotating wheels 28, which are respectively rolled on two top guide rails 26. A horizontal support plate 29 is fixedly connected to the front side of the base 18. Ear plate supports 30 located between the two bottom guide rails 25 are fixedly installed on the rear side of the horizontal support plate 29 and the rear side frame of the base 18. The rotary geared motor 22 is fixedly mounted on the horizontal support plate 29 and located in front of the ear plate support 30 on the front side. The lead screw 23 is set in the front-back direction and its two ends are respectively rotatably mounted on the two ear plate supports 30. The motor shaft of the rotary geared motor 22 is coaxially connected to the front end of the lead screw 23. The nut 24 is threaded on the lead screw 23 and is located in the middle of the two first rotating wheels 27. A linkage shaft 31 is fixedly connected between the nut 24 and the central shaft of the two first rotating wheels 27. The embedded microcontroller 4 is connected to the rotary geared motor 22 via signal.

[0105] The pedestrian recognition module includes two pedestrian recognition cameras 32, which are infrared cameras. The two pedestrian recognition cameras 32 are respectively set on the left and right sides of the camera bracket 16.

[0106] The environmental recognition module includes an environmental recognition camera 33, which is located in the middle of the camera bracket 16;

[0107] The geographic location identification module includes a GPS locator 34, which is mounted on an embedded microcontroller 4.

[0108] The emergency avoidance module includes a collision sensor 35, which is installed at the front end of the telescopic pole 8, and a sponge pad is installed on the outside of the collision sensor 35.

[0109] The embedded microcontroller 4 is connected to the pedestrian recognition camera 32, the environmental recognition camera 33, the GPS locator 34, and the collision sensor 35 respectively.

[0110] Using the above technical solution, a method for detecting pedestrians in blind spots on buses with risk avoidance function specifically includes the following steps:

[0111] (i) Install the aforementioned pedestrian detection device with risk avoidance function in the blind spot of the bus above the front roof of the bus 1;

[0112] (ii) When the bus 1 is moving normally, the power telescopic frame is in working condition. The embedded microcontroller 4 acquires the blind spot image of the bus 1 collected by the pedestrian recognition module. At the same time, the bus blind spot pedestrian detection device with risk avoidance function automatically avoids collision risk and emergency avoidance.

[0113] (III) The embedded microcontroller detects the blind spot image and obtains the real-time location information of the target pedestrian.

[0114] (iv) When the target pedestrian's current location is determined to be within the blind spot of bus 1 based on the target pedestrian's current location information, the embedded microcontroller 4 controls the sound and light alarm 3 to output alarm information.

[0115] (v) When it is determined that the target pedestrian's location is not within the blind spot of bus 1 based on the target pedestrian's current location information, the embedded microcontroller 4 uses a Kalman filter to predict the target pedestrian's trajectory;

[0116] (vi) According to the prediction information, when the trajectory of the target pedestrian enters the blind spot of the bus 1, the embedded microcontroller 4 controls the sound and light alarm 3 to output alarm information.

[0117] Step (II) is as follows: When bus 1 is in motion, the power telescopic frame is in working condition: the two first connecting rods 20 and the two second connecting rods 21 lift the top seat 19, so that the telescopic base 5 is at the normal working lifting angle. At the same time, the mover 7 is connected to the external wire through the wire interface 2. The embedded microcontroller 4 controls the current direction of the coil inside the mover 7, so that the magnetic field generated by the mover 7 interacts with the magnetic field of the stator 12 to generate thrust. The mover 7 drives the mover seat 6 to slide forward along the linear guide rail 11 to the front end of the linear guide rail 11. The mover seat 6 is blocked by the front baffle. 14 blocks the limit switch, and the telescopic rod 8 extends forward completely under the action of the moving base 6. The two pedestrian recognition cameras 32 extend forward and upward to the front roof of the bus 1. The two pedestrian recognition cameras 32 capture images of the blind spots on both sides of the front of the bus 1 in real time and transmit the captured blind spot images to the embedded microcontroller 4. When the driving environment is dark, such as in rainy days, at night, or in tunnels, the pedestrian recognition cameras 32 automatically turn on the infrared mode to ensure that the captured blind spot images are clear enough for target pedestrian detection.

[0118] During the operation of bus 1, the blind spot pedestrian detection device with risk avoidance function automatically avoids collision risks in the following two working modes:

[0119] The first working mode utilizes the environmental recognition camera 33 to automatically avoid collision risks;

[0120] The second working mode utilizes the GPS locator 34 to automatically avoid collision risks;

[0121] The bus blind spot pedestrian detection device with risk avoidance function uses collision sensor 35 to achieve emergency avoidance.

[0122] Step (3) is as follows: The embedded microcontroller 4 uses the YOLOv5s model of the YOLOv5 algorithm to detect the blind spot image of the bus 1, obtain the current target box corresponding to the target pedestrian, and obtain the current position of the target pedestrian. In reality, there may be multiple pedestrians around the bus 1. Therefore, when the embedded microcontroller 4 detects the blind spot image, it can obtain multiple current position information corresponding to multiple pedestrians. Among them, the target pedestrian refers to any one of the multiple pedestrians.

[0123] Step (four) specifically involves the following: Due to the obstruction of the blind spot image of bus 1 by other moving vehicles, pedestrians may suddenly appear within the blind spot of bus 1. When the location information of the target pedestrian obtained through the YOLOv5 algorithm appears within the shooting range of pedestrian recognition camera 32 and is located within the blind spot of bus 1, where the shooting range of pedestrian recognition camera 32 completely covers the blind spot of bus 1, the blind spot of bus 1 is customized according to different vehicle models and different operating conditions. That is, the user can determine the blind spot of bus 1 in advance through the image captured by pedestrian recognition camera 32. The determined blind spot range is then stored in the configuration file of the embedded microcontroller 4. The blind spot is further divided into high-risk and low-risk areas according to its distance from the bus 1. When a pedestrian is detected in the low-risk area, the embedded microcontroller 4 controls the audible and visual alarm 3 to work at a low frequency and issue a warning to remind the driver that a pedestrian has entered the blind spot range. When a pedestrian is detected in the high-risk area, the embedded microcontroller 4 controls the audible and visual alarm 3 to work at a high frequency. The audible and visual alarm 3 increases the flashing frequency of the lights and increases the decibel of the alarm bell to remind the driver to pay attention to the location of the pedestrian in the blind spot in order to avoid possible traffic accidents.

[0124] Step (5) specifically involves the following: When the location information of the target pedestrian obtained through the YOLOv5 algorithm appears within the shooting range of the pedestrian recognition camera 32 and is not located in the blind spot of the bus 1, it indicates that the pedestrian is not in a dangerous state. At this time, the embedded microcontroller 4 synchronizes the target pedestrian location information and target box information detected by YOLOv5 to the DeepSORT target tracking algorithm. The DeepSORT target tracking algorithm predicts the target pedestrian trajectory through a Kalman filter. The DeepSORT target tracking algorithm predicts the target pedestrian trajectory by including the following steps:

[0125] (1) Motion state prediction: The Kalman filter predicts the position of the target pedestrian in the next frame based on the position of the target pedestrian in the current frame. At this time, the predicted target pedestrian trajectory is in an unconfirmed state. In any video frame, an eight-dimensional state vector X = [u, v, r, h, u', v', r', h'] is used as the model for predicting the target pedestrian trajectory. [u, v] represent the horizontal and vertical positions of the target bounding box in the picture, respectively. [r, h] represent the aspect ratio and height of the target bounding box, respectively. The other four parameters are the corresponding velocity information.

[0126] (2) Motion state update: IOU matching is performed on the unconfirmed target pedestrian trajectory to obtain the optimal result, which has three possible outcomes:

[0127] When the matching result is a trajectory mismatch, that is, the target pedestrian has a trajectory but no corresponding detection box, if its frame number N is less than the maximum number of lost frames 30, it is retained in the tracking chain; otherwise, the target is deleted.

[0128] When the matching result is a detection mismatch, that is, the target pedestrian has a detection box but no corresponding trajectory, it is determined to be a new target pedestrian, and the trajectory of the target pedestrian is created at this time;

[0129] When the matching result is a successful match, the trajectory of the target pedestrian is updated using a Kalman filter. At this time, the predicted trajectory of the target pedestrian is in the confirmed state.

[0130] (3) Cascaded matching: The confirmed target pedestrian trajectory and its corresponding time parameter are cascaded and matched. Each time parameter has its corresponding tracked target pedestrian trajectory. Each target pedestrian trajectory is assigned a priority parameter. If the target pedestrian trajectory fails to match, the time parameter of the target pedestrian trajectory is incremented by 1, otherwise it is set to 0. The smaller the time parameter of the target pedestrian trajectory, the higher its priority. The target pedestrian trajectory that is matched first in the previous frame is given a high priority. Conversely, the target pedestrian trajectory with a larger time parameter means that the target pedestrian trajectory has a lower matching probability. For target pedestrian trajectories that have not been matched for several consecutive frames, their priority will be gradually reduced and eventually the target pedestrian trajectory will be deleted. At this time, the trajectory prediction of the target pedestrian is completed.

[0131] Another mode of blind spot pedestrian recognition in this invention is as follows: When the pedestrian recognition camera 32 is working, the embedded microcontroller 4 first reads the blind spot of the bus 1. When a pedestrian appears in the image captured by the pedestrian recognition camera 32, the embedded microcontroller 4 assigns a unique tracking frame to each person and continuously predicts the pedestrian's movement trajectory. Based on the prediction result, it determines whether the target pedestrian is in the blind spot of the bus 1. If so, the embedded microcontroller 4 controls the audible and visual alarm 3 to output alarm information. If not, it continues to track the target pedestrian until the target pedestrian enters the preset blind spot, at which point the embedded microcontroller 4 controls the audible and visual alarm 3 to work, or the target pedestrian leaves the image captured by the pedestrian recognition camera 32, at which point the tracking frame cancels tracking.

[0132] The specific process by which the bus blind spot pedestrian detection device with risk avoidance function uses the environmental recognition camera 33 to automatically avoid collision risks is as follows:

[0133] Step 1: Training Image Acquisition: The embedded microcontroller 4 acquires images of the target area along the fixed route of the bus 1 through the environment recognition camera 33, or directly transmits pre-captured target area images to the embedded microcontroller 4 as training images. The target area includes the retracted area and the extended area of ​​the telescopic pole 8. The retracted area of ​​the telescopic pole 8 is the area where the telescopic pole 8 may be at risk of collision during the movement of the bus 1, and the extended area of ​​the telescopic pole 8 is the area where the bus 1 may be at risk of collision after leaving the telescopic pole 8. The training images include images of the retracted area and the extended area of ​​the telescopic pole 8. The training images are taken on the fixed route of the bus 1, and 20 to 50 images are taken at each shooting location.

[0134] Step 2: Extracting SURF features: The embedded microcontroller 4 uses an improved SURF algorithm to perform feature recognition on the grayscale training images. The recognition process is as follows:

[0135] (A) Generating the Hessian matrix: The Hessian matrix is ​​the core of the improved SURF algorithm. Its function is to generate stable edge points in the image, which is the basis for feature extraction. A Hessian matrix can be calculated for each pixel in the image. In the SURF algorithm, since the feature points need to be scale-independent during the feature extraction process, each pixel in the image needs to be filtered before constructing the Hessian matrix to eliminate the correlation between pixels. The box filter is used for this purpose.

[0136] (B) Constructing scale space: By changing the size of the box filter, the original image is convolved with different sizes of filters in different directions to form a multi-scale space and extract image features around the pixels.

[0137] (C) Locating feature points: Each pixel processed by the Hessian matrix is ​​compared with points in the two-dimensional image space and scale space neighborhood to initially locate key points. Then, key points with weak energy and incorrectly located key points are filtered out to select the final stable key points as feature points.

[0138] (D) Selecting the main direction of the feature point: Taking the feature point as the center, a circular neighborhood of the feature point is obtained with a radius of 6δ. Taking 45° sectors as units, the cumulative values ​​of all pixels in the sector for the Haar wavelet are counted and a feature sub-vector is obtained. The size of the Haar wavelet is 2δ×2δ. Then, the 45° sector is rotated at 45° intervals. After one rotation, eight 45° sector regions are obtained, resulting in eight Haar wavelet cumulative values ​​and eight feature sub-vectors. The eight Haar wavelet cumulative values ​​are compared, and the feature sub-vector direction corresponding to the largest Haar wavelet cumulative value is selected as the main direction.

[0139] (E) Generate feature descriptors: Starting from the feature vector corresponding to the main direction, take the feature vectors of 8 sector regions in order of decreasing Haar response accumulation value to obtain a 4×8=32-dimensional feature descriptor;

[0140] Step 3: Generate an offline visual bag of words:

[0141] The K-means clustering algorithm is used to cluster all training images by features. Then, several cluster centers are used as visual words to generate offline visual bag of words for the target region. The offline visual bag of words histogram is obtained and normalized. To adapt to the matching needs of different urban road environments and the actual application of users, the range of cluster center k is set from 150 to 250. After considering the precision and retrieval efficiency, the cluster center k is set to 200. The cluster center k is the size of the offline visual bag of words.

[0142] Step 4: Input real-time traffic images:

[0143] The environmental recognition camera 33 inputs real-time environmental images of the bus 1 during its journey into the embedded microcontroller 4. The shooting frequency of the environmental recognition camera 33 is 1Hz to 10Hz. The embedded microcontroller 4 uses the same bag-of-words visual construction algorithm as in the third step above to process the input real-time environmental images, generate a bag-of-words visual model of the real-time environmental images, and obtain a real-time bag-of-words histogram.

[0144] Step 5: Calculate similarity:

[0145] The offline visual bag-of-words histogram is compared with the real-time visual bag-of-words histogram, and the similarity between the offline and real-time visual bag-of-words histograms is calculated. If the calculated similarity is less than a preset threshold, the process returns to the previous step to continue collecting real-time traffic images. If the similarity is greater than the preset threshold, SVM is used to classify the environmental images at this moment and output the image with the highest similarity in the offline image library.

[0146] The formula for calculating similarity is:

[0147]

[0148] Where H1 is the offline visual bag-of-words histogram; H2 is the real-time visual bag-of-words histogram; H1(i) represents the size of the i-th cluster center in the offline visual bag-of-words histogram; H2(i) represents the size of the i-th cluster center in the real-time visual bag-of-words histogram; k is the number of cluster centers, i.e. the size of the bag-of-words. The mean of the offline visual bag-of-words histogram; The mean of the real-time visual bag-of-words histogram;

[0149] The formulas for calculating the mean of the offline visual bag-of-words histogram and the mean of the histogram are as follows:

[0150]

[0151]

[0152] Step 6, SVM Classification:

[0153] SVM is used to classify real-time environmental images and output the classification results. When the embedded microcontroller 4 recognizes that the image obtained by SVM classification belongs to the retractable area of ​​the telescopic rod 8, the embedded microcontroller 4 controls the mover 7 to drive the mover seat 6 to slide backward along the linear guide rail 11 to the rear end of the linear guide rail 11. The mover seat 6 is blocked and limited by the rear baffle 15. The telescopic rod 8 is completely retracted backward under the action of the mover seat 6 to avoid the risk of collision. When the telescopic rod 8 has been retracted, the power telescopic frame maintains a self-protection state and returns to the fifth step above to continue processing. Real-time road condition image information; when the embedded microcontroller 4 recognizes that the image obtained by SVM classification belongs to the extended area of ​​the telescopic pole 8, the embedded microcontroller 4 controls the mover 7 to drive the mover seat 6 to slide forward along the linear guide rail 11 to the front end of the linear guide rail 11. The mover seat 6 is blocked and limited by the front baffle 14. The telescopic pole 8 extends forward completely under the action of the mover seat 6, and continues to perform blind spot pedestrian detection. When the telescopic pole 8 has been extended, the power telescopic frame remains in working state and returns to the fifth step above to continue processing the real-time road condition image information.

[0154] The specific process by which the bus blind spot pedestrian detection device with risk avoidance function uses GPS locator 34 to automatically avoid collision risks is as follows:

[0155] Step 1: Collect location information of risk areas along the route of bus 1:

[0156] GPS data information of locations with collision risk along the route of bus 1 is collected. The GPS data information includes longitude and latitude. This GPS data information is stored in the embedded microcontroller 4. The locations with collision risk are the locations where the telescopic pole 8 has a collision risk.

[0157] Step 2: Receive real-time location information from the GPS locator 34:

[0158] When bus 1 is traveling on a predetermined route, after receiving satellite positioning information, GPS locator 34 transmits the location information of bus 1 to embedded microcontroller 4. Embedded microcontroller 4 receives the location information from GPS locator 34 at a frequency of 2Hz.

[0159] Step 3: Determine the distance between the real-time location and any risk area:

[0160] Each time the embedded microcontroller 4 receives real-time location information of bus 1, it compares the real-time location information with the location information of any risk location stored inside the embedded microcontroller 4 to determine whether the distance between the real-time location of bus 1 and any risk location is less than the set distance threshold.

[0161] Step 4: Identify the risk area and retract the telescopic pole 8.

[0162] When the distance between the real-time position of bus 1 calculated by the embedded microcontroller 4 and any risk location is less than or equal to the preset distance threshold, the embedded microcontroller 4 controls the power telescopic frame to maintain or switch to self-protection mode, retracts the telescopic rod 8 to pause the pedestrian detection task in the driver's blind spot of bus 1, and returns to the third step above to continue calculating the distance between the real-time position of bus 1 and any risk location.

[0163] Step 5: Bus 1 leaves the risk area, and the powered telescopic frame resumes operation.

[0164] When the distance between the real-time position of bus 1 calculated by the embedded microcontroller 4 and any risk location is greater than the preset distance threshold, the embedded microcontroller 4 controls the power telescopic frame to maintain or switch to working state, extends the telescopic rod 8 forward to continue the pedestrian detection task in the blind spot of the bus driver's field of vision, and returns to the third step above to continue calculating the distance between the real-time position of bus 1 and any risk location.

[0165] The distance between the real-time location of bus 1 and any risk location is calculated using the following method: Assuming the Earth is a standard sphere with radius R, and assuming east longitude is positive, west longitude is negative, north latitude is positive, and south latitude is negative, then the real-time coordinates A(x, y) of bus 1 are represented as:

[0166]

[0167] The coordinates B(a, b) of any risk location are represented as:

[0168]

[0169] The formula for determining the distance between bus 1's real-time location and any risk location is:

[0170]

[0171] The distance threshold S is set by the manufacturer at the factory or by the bus driver based on the specific road conditions. Here, the distance threshold S is set to a range of 15 to 50 meters, taking into account the road conditions and speed information of bus 1 in the city.

[0172] The specific process of the bus blind spot pedestrian detection device with risk avoidance function using collision sensor 35 to achieve emergency avoidance is as follows:

[0173] Step 1: A collision was detected with telescopic pole 8.

[0174] Due to urban road construction and tree growth resulting in low tree branch height, new risk points that were not stored appeared on the route of bus 1. When bus 1 passed through these places, the moving seat 6 failed to retract the telescopic rod 8, resulting in a collision accident. When the collision sensor 35 detected the pressure change, it output the collision information and transmitted it to the embedded microcontroller 4.

[0175] Step 2: Embedded microcontroller 4-bit control of the power telescopic frame for emergency escape:

[0176] When the embedded microcontroller 4 receives the collision information, it controls the front and rear telescopic mechanism to retract the telescopic rod 8 and controls the rotary reduction motor 22 to drive the lead screw 23 to rotate. The lead screw 23 drives the nut 24 to move backward. The nut 24 drives the rear ends of the two first connecting rods 20 to move backward through the linkage shaft 31. Then the two first connecting rods 20 and the two second connecting rods 21 pull the top seat 19 downward, so that the top seat 19 drives the telescopic base 5 to descend to an angle parallel to the roof of the bus 1. This state is maintained and the blind spot pedestrian recognition work is stopped.

[0177] The sound and light alarm 3, embedded microcontroller 4, mover 7, stator 12, wire interface 2, rotary gear motor 22, lead screw 23, nut 24, pedestrian recognition camera 32, environmental recognition camera 33, GPS locator 34 and collision sensor 35 are all conventional components. Their specific structures and working principles will not be described in detail. The control part of this invention uses conventional control technology and does not involve new computer programs.

[0178] The above embodiments are only used to illustrate and not limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention without departing from the spirit and scope of the present invention. Any modifications or partial substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A pedestrian detection device for blind spots on buses with risk avoidance function, characterized in that: The system includes a power-operated telescopic frame, a pedestrian recognition module, an audible and visual alarm, an environmental recognition module, a geolocation recognition module, an emergency avoidance module, and an embedded microcontroller. The power-operated telescopic frame is installed on the outer roof of the front of the bus. The pedestrian recognition module, environmental recognition module, and emergency avoidance module are all located on the power-operated telescopic frame. The audible and visual alarm and the embedded microcontroller are both installed on the ceiling inside the bus and located directly below the power-operated telescopic frame. The geolocation recognition module is installed on the embedded microcontroller. The embedded microcontroller is connected to the power-operated telescopic frame, pedestrian recognition module, audible and visual alarm, environmental recognition module, geolocation recognition module, and emergency avoidance module via signals. The power telescopic frame includes a front and rear telescopic mechanism and a lifting mechanism. The front and rear telescopic mechanism is set above the front roof of the bus in the front-rear direction. The rear end of the front and rear telescopic mechanism is rotatably mounted on the roof of the bus. The lifting mechanism is mounted on the front roof of the bus and located below the front and rear telescopic mechanism. The upper end of the lifting mechanism is connected to the lower part of the front and rear telescopic mechanism and drives the front and rear telescopic mechanism to lift. The telescopic mechanism includes a telescopic base, a mover seat, a mover, and a telescopic rod. The telescopic base is positioned above the front roof of the bus along the front direction. Two bearing seats are fixedly installed on the front roof of the bus, spaced apart on the left and right sides of the rear end of the telescopic base. Horizontal rotating shafts are respectively installed on the left and right sides of the rear end of the telescopic base, and the two horizontal rotating shafts are respectively installed on the aforementioned two bearing seats. A guide groove is formed in the middle of the upper part of the telescopic base along the front direction, and two linear guide rails are also provided on the upper surface of the telescopic base along the front direction. Two linear guide rails are symmetrically arranged on the left and right sides of the guide groove, spaced apart. The mover seat is slidably mounted on the two linear guide rails. The mover is fixedly mounted on the lower side of the mover seat and slidably embedded in the guide groove. Several stators, which are rectangular permanent magnets, are fixedly mounted at equal intervals at the bottom of the guide groove. The stators are covered with a cover plate. The mover has a coil inside. The mover seat is provided with a wire interface for connecting to the mover wire. The wire interface is connected to an external wire to provide power to the mover. A telescopic base is fixedly mounted on the upper middle part of the front side. A front baffle is positioned at the front end of the guide chute, and a rear baffle is positioned at the upper middle part of the rear side of the telescopic base, positioned at the rear end of the guide chute. The upper sides of both the front and rear baffles are higher than the upper side of the linear guide rail. A guide hole with front and rear penetration is provided in the middle of the front baffle, and the telescopic rod is installed through the guide hole in the front-rear direction. The rear end of the telescopic rod is fixedly installed in the middle of the front side of the moving base, and a camera bracket located in front of the front baffle is fixedly installed at the front end of the telescopic rod. Anti-collision devices located within the guide chute are fixedly installed on the left and right sides of the rear side of the front baffle and the middle of the front side of the rear baffle. The device, the anti-collision device is a cylindrical pad made of rubber or sponge. Two limiters with front and rear spacing are fixedly installed on the upper left edge of the telescopic base. The front limiter is located in front of the moving base and close to the front baffle, and the rear limiter is located in rear of the moving base and close to the rear baffle. The height of the limiters is lower than the bottom of the moving base. The contact plate of the limiter is located on the top of the limiter. When the moving base slides on the linear guide rail and passes over the limiter, the bottom of the moving base presses the contact plate of the limiter and closes the contact plate of the limiter. The embedded microcontroller is connected to the moving base and the limiter signals respectively.

2. The pedestrian detection device for blind spots on buses with risk avoidance function according to claim 1, characterized in that: The lifting mechanism includes a base, a top seat, two first connecting rods, two second connecting rods, a rotary reduction motor, a lead screw, and a nut. Both the base and top seat are rectangular frame structures, positioned vertically below the telescopic base. The top seat is fixedly installed on the lower surface of the telescopic base, and the base is fixedly installed on the roof of the bus's front end. The two first connecting rods are spaced horizontally and inclined downwards from the front to the rear between the base and the top seat. Similarly, the two second connecting rods are also spaced horizontally and inclined downwards from the front to the rear between the base and the top seat. The first and second connecting rods are staggered horizontally. Two bottom guide rails, spaced horizontally and arranged along the front-rear direction, are fixedly installed on the base, each corresponding vertically to one of the two first connecting rods. Two top guide rails, spaced horizontally and arranged along the front-rear direction, are fixedly installed on the top seat, each corresponding vertically to one of the two second connecting rods. The front ends of the two first connecting rods are hinged to the lower front side of the top seat, and the rear ends of the two first connecting rods are rotatably connected. There are two first rotating wheels, which are respectively rolled and connected to two bottom guide rails. The front ends of two second connecting rods are hinged to the upper front part of the base, and the rear ends of the two second connecting rods are rotatably connected to second rotating wheels. The two second rotating wheels are respectively rolled and connected to two top guide rails. A horizontal support plate is fixedly connected to the front part of the base. The rear end of the horizontal support plate and the rear side frame of the base are both fixedly provided with ear plate supports located in the middle of the two bottom guide rails. The two ear plate supports are corresponding to each other. A rotary geared motor is fixedly installed on the horizontal support plate and located in front of the front ear plate support. A lead screw is set in the front-back direction and its two ends are respectively rotatably installed on the two ear plate supports. The motor shaft of the rotary geared motor is coaxially connected to the front end of the lead screw. A nut is threaded on the lead screw and is located in the middle of the two first rotating wheels. A linkage shaft is fixedly connected between the nut and the central shaft of the two first rotating wheels. An embedded microcontroller is connected to the rotary geared motor for signal transmission.

3. The pedestrian detection device for blind spots on buses with risk avoidance function according to claim 1, characterized in that: The pedestrian recognition module includes two pedestrian recognition cameras, which are infrared cameras, and the two pedestrian recognition cameras are respectively set on the left and right sides of the camera bracket. The environmental recognition module includes an environmental recognition camera, which is positioned in the middle of the camera bracket; The geolocation identification module includes a GPS locator, which is installed on an embedded microcontroller; The emergency avoidance module includes a collision sensor, which is installed at the front end of the telescopic pole and has a foam pad on its exterior. The embedded microcontroller is connected to the pedestrian recognition camera, the environmental recognition camera, the GPS locator, and the collision sensor signals, respectively.

Images