Vehicle and pedestrian anti-collision method and system, computer equipment and storage medium
By combining multi-view cameras and deep learning algorithms with vehicle communication bus data, the intelligent problem of pedestrian collision avoidance for engineering vehicles in complex environments has been solved, achieving high-precision collision risk assessment and active protection, and significantly reducing the probability of accidents.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-16
- Publication Date
- 2026-04-14
AI Technical Summary
Existing technologies are unable to provide all-weather, highly reliable, and proactive intelligent protection for pedestrians in complex and harsh working environments, leading to frequent collisions between engineering vehicles and pedestrians.
It uses a multi-view camera group to collect environmental images in real time, identifies and tracks pedestrians through deep learning algorithms, combines vehicle communication bus data to conduct collision risk assessment, and triggers warnings or vehicle control commands to generate a panoramic top-down view and achieve intelligent protection.
It achieves high-precision intelligent perception and dynamic risk assessment of pedestrians, reducing the probability of collision accidents and improving the safety of vehicle operations.
Smart Images
Figure CN121861925A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of engineering machinery safety technology, and more specifically, to a method, system, computer equipment, and storage medium for avoiding collisions between vehicles and pedestrians. Background Technology
[0002] Large engineering vehicles, such as excavators, loaders, dump trucks, road rollers, and cranes, are key equipment in infrastructure construction, mining, and logistics transportation. However, due to their large size, high cab position, and complex and variable operating environments (such as construction sites, mines, and ports), drivers face significant blind spots. Furthermore, these environments often involve close interaction between pedestrians and vehicles, leading to an extremely high risk of collisions. Once an accident occurs, it frequently results in severe personal injury and property damage.
[0003] Currently, the industry mainly offers the following technical solutions for pedestrian collision protection of engineering vehicles, but all of them have significant limitations: 1) Passive physical warning devices: Current technological status: These mainly include reversing buzzers, rotating warning lights, wide-angle rearview mirrors, and blind spot convex mirrors. These devices are inexpensive and widely used.
[0004] Existing drawbacks: Its effectiveness relies entirely on the subjective attention and judgment of pedestrians and drivers. In noisy construction sites, the warning sound can easily be drowned out; rearview mirrors and convex mirrors still have blind spots and are greatly affected by weather and lighting, failing to fundamentally eliminate blind spot risks. This is a reactive warning rather than a preventative measure.
[0005] 2) Detection technology based on a single sensor: Current technological status: Primarily includes ultrasonic radar, millimeter-wave radar, or lidar. Ultrasonic radar is commonly used for short-range reversing radar; millimeter-wave radar and lidar are used for obstacle detection in high-level autonomous driving.
[0006] Existing defects: Ultrasonic radar has a short detection range, low accuracy, and cannot effectively identify target types (it cannot distinguish between pedestrians and piles, stones, etc.), resulting in a high false alarm rate.
[0007] Millimeter-wave radar is sensitive to metallic objects, but it has a weak ability to detect and identify "soft targets" such as pedestrians, making it difficult to accurately distinguish their shapes, and it also has difficulties in identifying stationary targets.
[0008] LiDAR: Although it offers high point cloud accuracy, its performance drops sharply in adverse weather conditions such as rain, snow, fog, and dust, which are typical operating environments for construction vehicles. Furthermore, its high cost makes it difficult to widely adopt in the construction machinery sector.
[0009] Common problem: Most of the sensors mentioned above can only provide information such as "the presence of obstacles" and "approximate distance", and cannot obtain rich semantic information such as the target's category, posture, and orientation like vision, thus making it impossible to predict the pedestrian's intentions.
[0010] 3) Simple visual monitoring system: Current technological status: Some high-end engineering vehicles are equipped with surround-view camera systems, which stitch together surrounding images and display them on a screen in the driver's cab.
[0011] Existing shortcomings: This system is essentially just a "video display," leaving all judgment responsibility to the driver. During long hours of highly stressful work, drivers are easily fatigued or distracted and fail to detect dangers on the screen in time, lacking proactive intelligent analysis and early warning functions.
[0012] 4) Basic visual recognition early warning system: Current state of technology: A few solutions attempt to use simple image processing algorithms for moving object detection (such as frame difference method).
[0013] Existing drawbacks: These algorithms have extremely poor anti-interference capabilities and are easily affected by environmental factors such as changes in lighting, shadows, and swaying trees, resulting in a very high false alarm rate. Frequent false alarms can cause driver annoyance and psychological dependence, ultimately leading them to ignore system prompts, rendering the system ineffective. Furthermore, these systems typically lack deep integration with vehicle control systems (such as the CAN bus), failing to obtain the vehicle's own operating status (such as speed and steering angle), thus hindering accurate collision risk modeling, let alone proactive vehicle control intervention.
[0014] In summary, existing technologies either provide passive warnings, have insufficient perception capabilities and poor reliability, or lack intelligent decision-making and proactive control capabilities, failing to meet the urgent need for engineering vehicles to provide all-weather, highly reliable, and proactive intelligent protection for pedestrian safety in complex and harsh operating environments. Summary of the Invention
[0015] In view of the above-mentioned defects in the existing technology, the purpose of this invention is to provide a vehicle-pedestrian collision avoidance method, system, computer equipment and storage medium, which aims to solve the problems of inaccurate pedestrian detection, untimely warning and lack of active intervention capability in complex operating environments. It realizes an integrated system that can perceive risks, assess decisions and actively implement protective measures in all weather, with high precision and intelligence, thereby greatly reducing the probability of vehicle-pedestrian collision accidents.
[0016] To achieve the above objectives, the present invention provides the following technical solution: On the one hand, a method for avoiding collisions between vehicles and pedestrians is provided, which includes: Real-time images of the vehicle's surrounding environment are captured using a multi-view camera system. The collected images of the vehicle's surrounding environment are fused to generate a panoramic top-down view. Deep learning algorithms are then used to identify and track pedestrians, determining their pose, real-time position, speed, and trajectory relative to the vehicle. Monitor the vehicle's communication bus data and parse the communication bus data to obtain the vehicle's operating status data; The collision risk level is determined based on the pedestrian's pose, real-time location, speed and trajectory relative to the vehicle, as well as the vehicle's operating status data. Based on the collision risk level, a corresponding warning command or vehicle control command is triggered, and visual and / or auditory warning information is issued.
[0017] Furthermore, determining the collision risk level based on the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle, as well as the vehicle's operating status data, includes: Identify the vehicle speed, gear, steering angle, throttle and brake opening in the vehicle's operating status data; Predict the vehicle's trajectory based on the vehicle's operating status data; The collision time (TTC) and shortest distance per second (DCPA) are calculated based on the pedestrian's pose, real-time position, speed and trajectory relative to the vehicle and the vehicle's trajectory, and the collision risk level is determined accordingly.
[0018] Furthermore, the calculation of the time to collision (TTC) and shortest distance per second (DCPA) based on the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle, as well as the vehicle's trajectory, and the determination of the collision risk level accordingly, includes: Based on the pedestrian's real-time position, speed, and trajectory relative to the vehicle, predict the pedestrian's movement path over a future period of time; The vehicle's direction of travel is determined based on the gear and steering angle in the vehicle's operating status data, and the risk weight of pedestrians in the vehicle's direction of travel is increased. The shortest distance between the vehicle and the pedestrian's movement path and the time required to reach the shortest distance are calculated by combining the pedestrian's risk weight, and the collision time is determined. Based on the shortest distance and the collision time, combined with a preset safety threshold, the collision risk level is determined.
[0019] Furthermore, the step of triggering corresponding warning commands or vehicle control commands based on the collision risk level, and issuing visual and / or auditory warning information, includes: Obtain the collision risk level, which includes low, medium, and high levels; When the collision risk level is low, an audible and visual warning will be issued. When the collision risk level is medium, an audible and visual warning is issued, and at the same time, a speed limit or slow-down command is sent to the vehicle control system via the vehicle's communication bus. When the collision risk level is high and a collision is imminent, an emergency braking or stop operation command is sent to the vehicle control system via the vehicle's communication bus.
[0020] On the other hand, a vehicle-pedestrian collision avoidance system is provided, comprising: The image acquisition module is deployed at multiple preset locations on the vehicle to collect multi-view image data of the vehicle's surrounding environment in real time. The data processing and AI computing module is communicatively connected to the image acquisition module. It is used to fuse the acquired images of the vehicle's surrounding environment to generate a panoramic top-down view, and to use deep learning algorithms to identify and track pedestrians, determine the pedestrians' pose, real-time position, speed and trajectory relative to the vehicle, monitor the vehicle's communication bus data, and parse the communication bus data to obtain the vehicle's operating status data. The risk assessment and decision-making module is communicatively connected to the data processing and AI computing module, and is used to determine the collision risk level based on the pedestrian's pose, real-time position, speed and trajectory relative to the vehicle and the vehicle's operating status data. The graded early warning and execution module is communicatively connected to the risk assessment and decision-making module, and is used to trigger corresponding early warning commands or vehicle control commands according to the collision risk level. The human-machine interaction module is used to issue visual and / or auditory warning information to the vehicle operator according to the warning command.
[0021] Furthermore, the data processing and AI computing module includes: An image fusion unit is used to stitch and fuse the multi-view image data to generate a panoramic top view of the vehicle perimeter. The pedestrian recognition and tracking unit has a built-in trained deep learning neural network model for recognizing pedestrians in the panoramic top view and / or the multi-view image data, and calculating the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle.
[0022] Furthermore, the image acquisition module includes multiple wide-angle cameras and / or infrared night vision cameras, and the multiple preset positions include the front of the vehicle, the rear of the vehicle, the left and right rearview mirrors, and both sides of the vehicle body.
[0023] Furthermore, the system also includes: The data storage and remote communication module is used to record data throughout the system's early warning, decision-making, and execution processes, and can remotely transmit the recorded data to the cloud management platform.
[0024] In another aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.
[0025] In another aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the preceding methods.
[0026] Compared with the prior art, the present invention has the following beneficial effects: This application acquires real-time images of the vehicle's surrounding environment using multi-view cameras, and accurately identifies and tracks pedestrians in the fused panoramic top-down view using deep learning algorithms, obtaining their pose, position, speed, and trajectory. Simultaneously, it combines real-time operational status data obtained from the vehicle's communication bus to achieve joint perception of the dynamic information of both pedestrians and the vehicle. Based on this multi-dimensional information, a collision risk level is determined, accurately predicting potential collision scenarios and triggering corresponding warning commands or vehicle control commands at different risk levels, issuing audible and visual warnings to the driver or actively intervening in vehicle operation. This achieves high-precision intelligent perception of pedestrians around the vehicle, dynamic and accurate assessment of collision risks, and timely response to dangerous situations, significantly improving the vehicle's pedestrian protection capabilities in complex environments, effectively reducing the probability of collision accidents, and greatly enhancing the safety of vehicle operations. Attached Figure Description
[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the specific implementation or prior art description will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a layout diagram of the application environment for the vehicle and pedestrian collision avoidance method of the present invention; Figure 2 This is a hardware wiring diagram of the data processing and AI computing module of the present invention; Figure 3 This is a schematic diagram of the vehicle and pedestrian collision avoidance method of the present invention; Figure 4 This is a structural block diagram of the vehicle and pedestrian collision avoidance system of the present invention; Figure 5 This is an internal structural diagram of the computer device of the present invention. Detailed Implementation
[0029] The present invention will now be described in further detail with reference to a specific embodiment of a forklift and the accompanying drawings, but the implementation of the present invention is not limited thereto.
[0030] Example 1 This embodiment uses an internal combustion counterbalance forklift as the application object (vehicle) to describe in detail the implementation of this system. The operating characteristics of forklifts include frequent forward and backward movement, right-angle turns, mixed pedestrian and machine movement in rack areas, and dynamic blind spots caused by lifting masts, which place specific requirements on pedestrian collision avoidance systems.
[0031] I. Hardware Deployment and Connection: 1. Installation of the image acquisition module: like Figure 1 As shown, five ultra-wide-angle (190°) high-definition cameras were selected. Their specific installation positions were optimized for the forklift structure. Front-facing camera 201a: Installed in the middle of the front crossbeam of the forklift overhead guard, tilted forward and downward, to cover the area directly in front of the forklift, the fork teeth and the ground, which is the forklift's main blind spot for forward operation.
[0032] Left-side camera 201b: Mounted on the left-side upright of the overhead support, tilted downwards to the left, to cover the left-side aisle and shelving area, which is especially important when making right-angle turns.
[0033] Right-side camera 201c: Installed on the right-side upright of the top support frame, tilted downwards to the right, to cover the right-side aisle and shelving area.
[0034] Rear camera 201d: Mounted above the rear counterweight, tilted downwards and backwards, to cover blind spots directly behind and to the sides, ensuring safety when reversing.
[0035] 201e mast camera: Installed in the middle of the mast or on the lifting cylinder, it rises and falls with the mast to dynamically cover the constantly changing blind spots on the ground caused by the forks being raised, which is a critical blind spot unique to forklifts.
[0036] All cameras are shockproof, oil-resistant, and IP67 protected, and are connected to the central processing unit via abrasion-resistant shielded cables.
[0037] 2. Deployment of data processing and AI computing modules: The core of this module is an embedded AI computing box, preferably using a module with sufficient computing power and low power consumption as the hardware carrier. The AI computing box, also known as the AI host, is simply referred to as the computing box. This computing box is fixedly installed in a waterproof and shockproof enclosure within the forklift overhead guard. AI is short for Artificial Intelligence.
[0038] like Figure 2 As shown, the AI computing box connects to all cameras via a switch. Simultaneously, the AI computing box connects to the forklift's CAN bus network via a CAN transceiver to acquire key status information such as vehicle speed, gear (forward / reverse), steering angle, and mast lifting height.
[0039] 3. Installation of early warning and execution terminals: An anti-glare LCD display is integrated on the instrument panel in front of the steering wheel in the driver's cab to display a 360° panoramic view, pedestrian position tracking frame, and alarm information.
[0040] A high-brightness LED warning light (blue or yellow) is installed in each of the four directions (front, back, left, and right) of the roof guard to provide visual warning to pedestrians outside the vehicle.
[0041] A buzzer is installed in the driver's cab to provide an auditory warning to the driver.
[0042] The system's execution commands are sent directly to the forklift's vehicle controller and hydraulic controller via the CAN bus, thereby limiting the electronic speed governor and triggering slow braking in extreme situations.
[0043] II. Software Implementation and Workflow: like Figure 3 As shown, the software algorithm of this system runs on an embedded AI computing box, and its workflow is optimized for forklift operation modes: 1. Image acquisition and preprocessing: After the system starts up, each camera synchronously acquires video streams. For the gantry camera 201e, the system dynamically adjusts its image distortion correction parameters based on the real-time gantry height read from the CAN bus to ensure an accurate viewing angle at any height.
[0044] 2. Image fusion and target recognition: The image fusion unit combines five video streams into a seamless surround view panorama and provides multiple display modes (viewing all or zooming in on one video stream individually) for the driver.
[0045] The pedestrian recognition and tracking unit loads a lightweight, high-speed deep learning model, which is specifically trained for situations where pedestrians may be partially obscured by shelves or pallets in forklift scenarios. The model performs real-time inference on a panoramic overhead view to identify and track pedestrians.
[0046] 3. Risk Assessment and Decision-Making (Specialized for Forklift Scenarios): The risk assessment and decision-making module reads key status data of the forklift from the CAN bus in real time, including: vehicle speed, gear (forward / reverse signal), steering angle, mast lifting height, and fork tilt angle.
[0047] The risk assessment algorithm fully considers forklift operating conditions: When the system detects that the gear is in "reverse" mode, it automatically increases the risk weight of pedestrians within the rear camera's field of view.
[0048] When the system detects that the turning angle is greater than a certain threshold (when turning), it automatically increases the risk weight of pedestrians in the field of view of the camera on the inside of the turn (such as the left side when turning left), because the blind spot is the largest there.
[0049] When the system detects that the gantry is lifting at a high height, it automatically expands the ground monitoring range and safety threshold within the field of view of the gantry camera, because the driver's ground visibility is extremely poor at this time.
[0050] Calculate the TTC and DCPA between the dynamic profiles of pedestrians and forklifts.
[0051] 4. Tiered response and execution: Level 1 Response (Low Risk): Pedestrians are marked with a green box on the display screen. There is no audible alarm; only the exterior LED lights flash slowly to alert nearby pedestrians.
[0052] Level 2 Response (Medium Risk): Pedestrian markers turn yellow and flash. The in-cab buzzer emits intermittent "beep" sounds, and the exterior LED lights begin flashing rapidly. The system limits the forklift's speed via the CAN bus (e.g., reducing it to 2 km / h).
[0053] Level 3 Response (High Risk): The pedestrian marker turns into a bright red, flashing box. The buzzer sounds a continuous, long blast. The system immediately applies deceleration braking (not a sudden stop, to prevent cargo tipping) and restricts the hydraulic system's operation (e.g., preventing the gantry from continuing to rise), while the exterior LED lights remain on to warn pedestrians.
[0054] 5. Data recording and uploading: The system records all event data. Normal operational data without alarms is sampled and saved at a certain ratio; for all alarm events, complete data (video + vehicle data) for 15 seconds before and after the alarm is saved and uploaded to the fleet management system via Wi-Fi (in a designated area of the warehouse) or 4G module.
[0055] In summary, this embodiment successfully achieves effective proactive protection for pedestrians in complex human-machine mixed environments such as warehouses and logistics centers by optimizing hardware layout and software strategies specifically tailored to the structural characteristics and operating modes of forklifts. Such adaptive adjustments for specific forklift models are all within the scope of protection of this invention.
[0056] Compared with the prior art, this embodiment has the following beneficial effects: 1) A qualitative leap in perception capabilities: Through multi-view visual fusion and deep learning AI recognition, the system can not only perceive "obstacles" but also accurately identify "pedestrians" and track their movement trajectories. This provides far richer environmental semantic information than radar and ultrasound, greatly reducing false alarm and false negative rates.
[0057] 2) More accurate and intelligent risk assessment: It innovatively integrates the dynamic information of pedestrians perceived by vision with the vehicle's own operating status information, so that the risk assessment model is no longer an isolated distance measurement, but a dynamic prediction of the motion status of both parties. The calculated indicators such as TTC are more in line with the actual risks, and the decision-making is more scientific.
[0058] 3) It has achieved a leap from passive alarm to active protection: The original three-level response mechanism, especially the active speed limiting and braking function that is deeply integrated with the vehicle control system, has changed the shortcomings of the traditional system that only "warns" but does not "act". It can take over the vehicle at critical moments, form an effective intervention, and truly build an active safety defense line.
[0059] 4) Strong environmental adaptability: By adopting wide-angle and infrared camera configurations and AI models trained with a large amount of real-world scene data, the system can effectively cope with common harsh environments for vehicles, such as dim lighting, dusty conditions, rain and snow, ensuring the system's reliability in all weather conditions.
[0060] 5) Evolutionary capability: Through the data black box and cloud connection, the system can continuously collect data (edge case) for iterative optimization of the algorithm, making the system "smarter" the more it is used and continuously improving its protection capabilities.
[0061] In summary, this application provides a complete, efficient, and intelligent vehicle-pedestrian collision avoidance solution that can effectively address long-standing pain points in the industry and significantly improve operational safety.
[0062] Example 2 Please see Figure 3 , Figure 4 This embodiment provides a method for avoiding collisions between vehicles and pedestrians, which includes: Real-time images of the vehicle's surrounding environment are captured using a multi-view camera system. The collected images of the vehicle's surrounding environment are fused to generate a panoramic top-down view. Deep learning algorithms are then used to identify and track pedestrians, determining their pose, real-time position, speed, and trajectory relative to the vehicle. Monitor the vehicle's communication bus data and parse the communication bus data to obtain the vehicle's operating status data; The collision risk level is determined based on the pedestrian's pose, real-time location, speed and trajectory relative to the vehicle, as well as the vehicle's operating status data. Based on the collision risk level, a corresponding warning command or vehicle control command is triggered, and visual and / or auditory warning information is issued.
[0063] This application acquires real-time images of the vehicle's surrounding environment using multi-view cameras, and accurately identifies and tracks pedestrians in the fused panoramic top-down view using deep learning algorithms, obtaining their pose, position, speed, and trajectory. Simultaneously, it combines real-time operational status data obtained from the vehicle's communication bus to achieve joint perception of the dynamic information of both pedestrians and the vehicle. Based on this multi-dimensional information, a collision risk level is determined, accurately predicting potential collision scenarios and triggering corresponding warning commands or vehicle control commands at different risk levels, issuing audible and visual warnings to the driver or actively intervening in vehicle operation. This significantly improves the vehicle's pedestrian protection capabilities in complex environments, effectively reduces the probability of collision accidents, and greatly enhances the safety of vehicle operations.
[0064] Furthermore, determining the collision risk level based on the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle, as well as the vehicle's operating status data, includes: Identify the vehicle speed, gear, steering angle, throttle and brake opening in the vehicle's operating status data; Predict the vehicle's trajectory based on the vehicle's operating status data; The collision time (TTC) and shortest distance per second (DCPA) are calculated based on the pedestrian's pose, real-time position, speed and trajectory relative to the vehicle and the vehicle's trajectory, and the collision risk level is determined accordingly.
[0065] Furthermore, the calculation of the time to collision (TTC) and shortest distance per second (DCPA) based on the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle, as well as the vehicle's trajectory, and the determination of the collision risk level accordingly, includes: Based on the pedestrian's real-time position, speed, and trajectory relative to the vehicle, predict the pedestrian's movement path over a future period of time; The vehicle's direction of travel is determined based on the gear and steering angle in the vehicle's operating status data, and the risk weight of pedestrians in the vehicle's direction of travel is increased. The shortest distance between the vehicle and the pedestrian's movement path and the time required to reach the shortest distance are calculated by combining the pedestrian's risk weight, and the collision time is determined. Based on the shortest distance and the collision time, combined with a preset safety threshold, the collision risk level is determined.
[0066] Furthermore, the step of triggering corresponding warning commands or vehicle control commands based on the collision risk level, and issuing visual and / or auditory warning information, includes: Obtain the collision risk level, which includes low, medium, and high levels; When the collision risk level is low, an audible and visual warning will be issued. When the collision risk level is medium, an audible and visual warning is issued, and at the same time, a speed limit or slow-down command is sent to the vehicle control system via the vehicle's communication bus. When the collision risk level is high and a collision is imminent, an emergency braking or stop operation command is sent to the vehicle control system via the vehicle's communication bus.
[0067] like Figure 4 As shown, a vehicle-pedestrian collision avoidance system is provided, comprising: The image acquisition module is deployed at multiple preset locations on the vehicle to collect multi-view image data of the vehicle's surrounding environment in real time. The data processing and AI computing module is communicatively connected to the image acquisition module. It is used to fuse the acquired images of the vehicle's surrounding environment to generate a panoramic top-down view, and to use deep learning algorithms to identify and track pedestrians, determine the pedestrians' pose, real-time position, speed and trajectory relative to the vehicle, monitor the vehicle's communication bus data, and parse the communication bus data to obtain the vehicle's operating status data. The risk assessment and decision-making module is communicatively connected to the data processing and AI computing module, and is used to determine the collision risk level based on the pedestrian's pose, real-time position, speed and trajectory relative to the vehicle and the vehicle's operating status data. The graded early warning and execution module is communicatively connected to the risk assessment and decision-making module, and is used to trigger corresponding early warning commands or vehicle control commands according to the collision risk level. The human-machine interaction module is used to issue visual and / or auditory warning information to the vehicle operator according to the warning command.
[0068] The advantage of this embodiment is that it enables all-weather, high-precision detection of pedestrians in blind spots, and through deep integration with the vehicle control system, it achieves a leap from passive early warning to active intervention, greatly improving the safety of vehicle operations.
[0069] Furthermore, the data processing and AI computing module includes: An image fusion unit is used to stitch and fuse the multi-view image data to generate a panoramic top view of the vehicle perimeter. The pedestrian recognition and tracking unit has a built-in trained deep learning neural network model for recognizing pedestrians in the panoramic top view and / or the multi-view image data, and calculating the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle.
[0070] Among them, the deep learning neural network model is a fusion model of object detection algorithm and multi-object tracking algorithm.
[0071] Furthermore, the image acquisition module includes multiple wide-angle cameras and / or infrared night vision cameras, and the multiple preset positions include the front of the vehicle, the rear of the vehicle, the left and right rearview mirrors, and both sides of the vehicle body.
[0072] Furthermore, the system also includes: The data storage and remote communication module is used to record data throughout the system's early warning, decision-making, and execution processes, and can remotely transmit the recorded data to the cloud management platform.
[0073] In another aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.
[0074] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores vehicle and pedestrian collision avoidance data. The network interface communicates with external terminals via a network connection. When the processor executes the computer program, it implements a vehicle and pedestrian collision avoidance method.
[0075] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Real-time images of the vehicle's surrounding environment are captured using a multi-view camera system. The collected images of the vehicle's surrounding environment are fused to generate a panoramic top-down view. Deep learning algorithms are then used to identify and track pedestrians, determining their pose, real-time position, speed, and trajectory relative to the vehicle. Monitor the vehicle's communication bus data and parse the communication bus data to obtain the vehicle's operating status data; The collision risk level is determined based on the pedestrian's pose, real-time location, speed and trajectory relative to the vehicle, as well as the vehicle's operating status data. Based on the collision risk level, a corresponding warning command or vehicle control command is triggered, and visual and / or auditory warning information is issued.
[0076] For specific limitations on the steps a processor takes when executing a computer program, please refer to the limitations on methods for avoiding collisions between vehicles and pedestrians mentioned above, which will not be repeated here.
[0077] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Real-time images of the vehicle's surrounding environment are captured using a multi-view camera system. The collected images of the vehicle's surrounding environment are fused to generate a panoramic top-down view. Deep learning algorithms are then used to identify and track pedestrians, determining their pose, real-time position, speed, and trajectory relative to the vehicle. Monitor the vehicle's communication bus data and parse the communication bus data to obtain the vehicle's operating status data; The collision risk level is determined based on the pedestrian's pose, real-time location, speed and trajectory relative to the vehicle, as well as the vehicle's operating status data. Based on the collision risk level, a corresponding warning command or vehicle control command is triggered, and visual and / or auditory warning information is issued.
[0078] For specific limitations on the steps implemented when a computer program is executed by a processor, please refer to the limitations on methods for avoiding collisions between vehicles and pedestrians mentioned above, which will not be repeated here.
[0079] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. A method for avoiding collisions between vehicles and pedestrians, characterized in that, include: Real-time images of the vehicle's surrounding environment are captured using a multi-view camera system. The collected images of the vehicle's surrounding environment are fused to generate a panoramic top-down view. Deep learning algorithms are then used to identify and track pedestrians, determining their pose, real-time position, speed, and trajectory relative to the vehicle. Monitor the vehicle's communication bus data and parse the communication bus data to obtain the vehicle's operating status data; The collision risk level is determined based on the pedestrian's pose, real-time location, speed and trajectory relative to the vehicle, as well as the vehicle's operating status data. Based on the collision risk level, a corresponding warning command or vehicle control command is triggered, and visual and / or auditory warning information is issued.
2. The vehicle-pedestrian collision avoidance method according to claim 1, characterized in that, The step of determining the collision risk level based on the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle, as well as the vehicle's operating status data, includes: Identify the vehicle speed, gear, steering angle, throttle and brake opening in the vehicle's operating status data; Predict the vehicle's trajectory based on the vehicle's operating status data; The collision time and shortest distance are calculated based on the pedestrian's pose, real-time location, speed and trajectory relative to the vehicle and the vehicle's trajectory, and the collision risk level is determined accordingly.
3. The vehicle-pedestrian collision avoidance method according to claim 2, characterized in that, The calculation of collision time and shortest distance based on the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle, as well as the vehicle's trajectory, and the determination of collision risk level accordingly, includes: Based on the pedestrian's real-time position, speed, and trajectory relative to the vehicle, predict the pedestrian's movement path over a future period of time; The vehicle's direction of travel is determined based on the gear and steering angle in the vehicle's operating status data, and the risk weight of pedestrians in the vehicle's direction of travel is increased. The shortest distance between the vehicle and the pedestrian's movement path and the time required to reach the shortest distance are calculated by combining the pedestrian's risk weight, and the collision time is determined. Based on the shortest distance and the collision time, combined with a preset safety threshold, the collision risk level is determined.
4. The vehicle-pedestrian collision avoidance method according to claim 1, characterized in that, The step of triggering corresponding warning commands or vehicle control commands based on the collision risk level and issuing visual and / or auditory warning information includes: Obtain the collision risk level, which includes low, medium, and high levels; When the collision risk level is low, an audible and visual warning will be issued. When the collision risk level is medium, an audible and visual warning is issued, and at the same time, a speed limit or slow-down command is sent to the vehicle control system via the vehicle's communication bus. When the collision risk level is high and a collision is imminent, an emergency braking or stop operation command is sent to the vehicle control system via the vehicle's communication bus.
5. A vehicle-pedestrian collision avoidance system, characterized in that, include: The image acquisition module is deployed at multiple preset locations on the vehicle to collect multi-view image data of the vehicle's surrounding environment in real time. The data processing and AI computing module is communicatively connected to the image acquisition module. It is used to fuse the acquired images of the vehicle's surrounding environment to generate a panoramic top-down view, and to use deep learning algorithms to identify and track pedestrians, determine the pedestrians' pose, real-time position, speed and trajectory relative to the vehicle, monitor the vehicle's communication bus data, and parse the communication bus data to obtain the vehicle's operating status data. The risk assessment and decision-making module is communicatively connected to the data processing and AI computing module, and is used to determine the collision risk level based on the pedestrian's pose, real-time position, speed and trajectory relative to the vehicle and the vehicle's operating status data. The graded early warning and execution module is communicatively connected to the risk assessment and decision-making module, and is used to trigger corresponding early warning commands or vehicle control commands according to the collision risk level. The human-machine interaction module is used to issue visual and / or auditory warning information to the vehicle operator according to the warning command.
6. The vehicle and pedestrian collision avoidance system according to claim 5, characterized in that, The data processing and AI computing module includes: An image fusion unit is used to stitch and fuse the multi-view image data to generate a panoramic top view of the vehicle perimeter. The pedestrian recognition and tracking unit has a built-in trained deep learning neural network model for recognizing pedestrians in the panoramic top view and / or the multi-view image data, and calculating the pedestrian's pose, real-time position, speed, and trajectory relative to the vehicle.
7. The vehicle and pedestrian collision avoidance system according to claim 5, characterized in that, The image acquisition module includes multiple wide-angle cameras and / or infrared night vision cameras, and the multiple preset positions include the front of the vehicle, the rear of the vehicle, the left and right rearview mirrors, and both sides of the vehicle body.
8. The vehicle and pedestrian collision avoidance system according to claim 5, characterized in that, The system also includes: The data storage and remote communication module is used to record data throughout the system's early warning, decision-making, and execution processes, and can remotely transmit the recorded data to the cloud management platform.
9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.