Unmanned vehicle obstacle avoidance method and system based on visual detection

Abstract

The invention discloses an unmanned vehicle obstacle avoidance method and system based on visual inspection, and relates to the technical field of visual inspection, and the method comprises the following steps: extracting historical image information of an unmanned vehicle in a historical driving process, carrying out the obstacle detection of the historical image information, and obtaining a historical obstacle detection result set; determining a historical target obstacle according to a first detection result of the historical obstacle detection result set; marking obstacles which are adjacent to the historical target obstacle or generate linkage influence with the historical target obstacle in the historical obstacle detection result set as a historical associated obstacle set; performing distance and speed processing analysis on the first detection result and the second detection result of the historical obstacle detection result set to obtain first historical associated motion data, and extracting a first target historical associated obstacle from the historical associated obstacle set according to the first historical associated motion data; the method has the effects of effectively avoiding the collision risk and remarkably improving the driving safety of the unmanned vehicle.

Description

Technical Field

[0001] The present invention relates to the field of visual detection technology, and more specifically, to an unmanned vehicle obstacle avoidance method and system based on visual detection. Background Art

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, the safe and efficient operation of autonomous vehicles has become a core requirement, and obstacle avoidance is a key component in ensuring their reliable operation. Traditional obstacle avoidance methods for autonomous vehicles focus solely on obstacle information in the current frame, lacking in-depth utilization of historical driving data and unable to draw on experience gained from similar scenarios. Faced with the interactive movement of obstacles in dynamic traffic flows (such as the chain reaction caused by lane changes at intersections), it is difficult to accurately predict their motion trajectories, resulting in delayed obstacle avoidance strategies and irrational path planning. This can lead to excessive braking that affects traffic efficiency or safety risks caused by untimely obstacle avoidance. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide an unmanned vehicle obstacle avoidance method and system based on visual detection.

[0004] To achieve the above object, the present invention provides the following technical solutions: A method for avoiding obstacles in an unmanned vehicle based on visual detection, the method comprising the following steps: Extracting historical image information of the unmanned vehicle during its historical driving process, performing obstacle detection on the historical image information to obtain a historical obstacle detection result set, and determining a historical target obstacle based on a first detection result of the historical obstacle detection result set; The obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle are marked as the historical associated obstacle set; Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set based on distance and speed to obtain first historical associated motion data, and extracting a first target historical associated obstacle from the historical associated obstacle set based on the first historical associated motion data; Filtering the second historical target-related obstacles in a preset historical period from the historical driving data; Processing the motion conditions of the historical target obstacle, the first target historically associated obstacle, and the second target historically associated obstacle to obtain a historical motion feature set; The current target obstacles of the unmanned vehicle during its current driving process are collected, obstacles associated with the current target obstacles are marked as currently associated obstacles, motion feature information of the current target obstacles and the currently associated obstacles are obtained to obtain a current motion feature information set, processing motion features are matched from a historical motion feature set based on the current motion feature information set, and the driving path of the unmanned vehicle is planned and obstacle avoidance control is performed based on the processed motion features.

[0005] Preferably, the historical motion feature set is obtained by processing the motion conditions of the historical target obstacle, the first target historically associated obstacle, and the second historical target associated obstacle, specifically including: Extracting motion feature information of the first target's historically associated obstacle and the historical target obstacle to obtain a first historical motion feature, and extracting motion feature information of the second historical target's associated obstacle to obtain a second historical motion feature; The first historical movement feature and the second historical movement feature are combined into a historical movement feature set.

[0006] Preferably, performing obstacle detection on historical image information to obtain a historical obstacle detection result set, and determining a historical target obstacle according to a first detection result of the historical obstacle detection result set specifically includes the following steps: Performing target detection on historical image information to obtain the location, shape and type information of obstacles to form a historical obstacle detection result set; The obstacle detection standard closest to the unmanned vehicle or posing the greatest threat to the driving safety of the unmanned vehicle is set as the detection standard, the first detection result that meets the detection standard is screened out from the historical obstacle detection result set, and the target obstacle is determined based on the first detection result.

[0007] Preferably, marking obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle as a historical associated obstacle set specifically includes the following steps: Determine the scope of the adjacent area based on the location and size of historical target obstacles; Obstacles located in adjacent areas are screened out from the historical obstacle detection result set to obtain a first obstacle association set; Filtering a second detection result from a set of historical obstacle detection results, where the second detection result refers to an obstacle detection result that poses the greatest threat to driving safety; Marking obstacles in the historical associated obstacle set that have a linkage effect with the second detection result as a second obstacle association set; The first obstacle association set and the second obstacle association set are combined into a historical association obstacle set.

[0008] Preferably, the first detection result and the second detection result of the historical obstacle detection result set are processed and analyzed by distance and speed to obtain the first historical associated motion data, which specifically includes the following steps: Calculating the distance and speed of the obstacle in the first detection result to obtain a first motion parameter set; Calculating the distance and speed of the obstacle in the second detection result to obtain a second motion parameter set; Obtaining a historical correlation motion data set of relative distance change and relative speed change according to the first motion parameter set and the second motion parameter set; A motion correlation threshold range is preset, and first historical correlation motion data within the preset motion correlation threshold range is screened out from the historical correlation motion data set.

[0009] Preferably, the first target historically associated obstacle is extracted from the historically associated obstacle set according to the first historically associated motion data, specifically: A first target-associated obstacle having a high degree of motion correlation with the historical target obstacle is extracted from the historically associated obstacle set according to the first historically associated motion data.

[0010] Preferably, extracting motion feature information of the first target historically associated obstacle and the historical target obstacle to obtain the first historical motion feature, and extracting motion feature information of the second historical target associated obstacle to obtain the second historical motion feature, specifically includes the following steps: The historical motion state feature set is obtained by extracting the motion direction, speed change and acceleration of the historical target obstacle and the first target historical associated obstacle respectively; The historical motion state feature set and the first historical associated motion data are combined into a first historical motion feature; Extracting obstacle motion data within a preset historical period from the historical driving data to obtain a second historical associated motion data set; extracting a second target historically associated obstacle from the historically associated obstacle set according to the second historically associated motion data; The second historical motion feature is obtained by respectively collecting motion feature information of the historical target obstacle and the second target historical associated obstacle within the historical period.

[0011] Preferably, the motion feature information of the current target obstacle and the current associated obstacles is obtained to obtain a current motion feature information set, a processing motion feature is matched from a historical motion feature set based on the current motion feature information set, and a driving path of the unmanned vehicle is planned and obstacle avoidance control is performed based on the processed motion feature, specifically including the following steps: The current motion feature information set includes the position, speed and motion direction of the current target obstacle and the current associated obstacles; Matching and processing motion features from the historical motion feature set according to the current motion feature information set; Predict the motion trajectory dataset of the current target obstacle and the current associated obstacles based on the current motion feature information set; Determine the driving area of ​​the unmanned vehicle based on its current location and driving status; The motion trajectory dataset, driving area and processed motion features are input into the obstacle avoidance decision model to plan the driving path and perform obstacle avoidance control for the unmanned vehicle.

[0012] An unmanned vehicle obstacle avoidance system based on visual detection, comprising: Extraction module: extracts historical image information of the unmanned vehicle during its historical driving process, performs obstacle detection on the historical image information to obtain a historical obstacle detection result set, and determines the historical target obstacle based on the first detection result of the historical obstacle detection result set; Marking module: Marks obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle as a historical associated obstacle set; Processing module: Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set based on distance and speed to obtain first historical associated motion data, and extracting the first target historical associated obstacle from the historical associated obstacle set based on the first historical associated motion data; Screening module: Screening out the second historical target-related obstacles in a preset historical period from historical driving data; Processing module: Processing the movement of the historical target obstacle, the first target historically associated obstacle, and the second target historically associated obstacle to obtain a historical movement feature set; Obstacle avoidance module: collects the current target obstacles of the unmanned vehicle during its current driving process, marks the obstacles associated with the current target obstacles as currently associated obstacles, obtains the motion feature information of the current target obstacles and the currently associated obstacles to obtain the current motion feature information set, matches the processed motion features from the historical motion feature set based on the current motion feature information set, and plans the driving path of the unmanned vehicle and performs obstacle avoidance control based on the processed motion features.

[0013] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, an obstacle avoidance method for an unmanned vehicle based on visual detection is implemented.

[0014] Compared with the prior art, the present invention has the following beneficial effects: From a safety perspective, the present invention uses multi-source perception fusion to accurately capture the position, speed, and direction of movement of the current target obstacle and related obstacles. Combined with historical motion feature matching and trajectory prediction, the unmanned vehicle can gain early insight into the movement trends of obstacles. Predicting danger can significantly shorten obstacle avoidance reaction time. Whether it is a sudden static obstacle, a complex interaction of dynamic obstacles, or a trajectory fuzzy scene in severe weather, the historical adaptation strategy and real-time path planning can effectively avoid collision risks, significantly improve the driving safety of unmanned vehicles, and reduce the accident rate. In terms of efficiency, the unmanned vehicle no longer blindly brakes or changes lanes drastically when avoiding obstacles. Instead, it rationally utilizes road space based on its current position and driving status, matching historical effective strategies, ensuring safety while maintaining smooth traffic flow, reducing road congestion caused by frequent sudden braking and unreasonable detours, improving the overall traffic efficiency of urban roads, and enabling unmanned vehicles to operate efficiently even in complex road conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the steps of an unmanned vehicle obstacle avoidance method based on visual detection proposed by the present invention; Figure 2 The present invention proposes a module schematic diagram of an unmanned vehicle obstacle avoidance system based on visual detection; Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention.

[0016] 610 , processor; 620 , communication interface; 630 , memory; 640 , communication bus. DETAILED DESCRIPTION

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0020] Reference Figure 1-Figure 3 shown.

[0021] Example 1 further illustrates an obstacle avoidance method for an unmanned vehicle based on visual detection proposed by the present invention.

[0022] A method for avoiding obstacles in an unmanned vehicle based on visual detection, the method comprising the following steps: Extracting historical image information of the unmanned vehicle during its historical driving process, performing obstacle detection on the historical image information to obtain a historical obstacle detection result set, and determining a historical target obstacle based on a first detection result of the historical obstacle detection result set; The obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle are marked as the historical associated obstacle set; Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set based on distance and speed to obtain first historical associated motion data, and extracting a first target historical associated obstacle from the historical associated obstacle set based on the first historical associated motion data; Filtering the second historical target-related obstacles in a preset historical period from the historical driving data; Processing the motion conditions of the historical target obstacle, the first target historically associated obstacle, and the second target historically associated obstacle to obtain a historical motion feature set; The current target obstacles of the unmanned vehicle during its current driving process are collected, obstacles associated with the current target obstacles are marked as currently associated obstacles, motion feature information of the current target obstacles and the currently associated obstacles are obtained to obtain a current motion feature information set, processing motion features are matched from a historical motion feature set based on the current motion feature information set, and the driving path of the unmanned vehicle is planned and obstacle avoidance control is performed based on the processed motion features.

[0023] This application first extracts image information from the driverless vehicle's historical driving records and uses deep learning object detection models (such as YOLO and FasterR-CNN) to identify the location, shape, and type of obstacles within the image to generate a historical obstacle detection result set. Historical target obstacles are then screened based on the closest distance or highest safety threat criteria to identify the core obstacle avoidance targets. Next, historically associated obstacle marking is performed. On the one hand, adjacent areas are delineated based on the location and size of historical target obstacles to filter out a first set of physically adjacent obstacles. On the other hand, the second detection result markings posing the greatest driving safety threat are identified from the result set, along with obstacles with motion linkage to form a second set of obstacles. These two are combined to form a historically associated obstacle set, comprehensively covering potential threats surrounding the target obstacle and its chain effects.

[0024] The system calculates motion parameters such as distance and speed for obstacles from the first and second detection results. After analyzing changes in relative distance and speed, it constructs a historically correlated motion dataset. Highly correlated first historically correlated motion data is then filtered using a preset threshold. Based on this data, first historically correlated obstacles with a high degree of motion correlation with the historical target obstacle are extracted from the historically correlated obstacle dataset, focusing on key factors such as depth that influence the target obstacle's motion. Simultaneously, second historically correlated obstacles with motion characteristics associated with the historical target obstacle are filtered from the historical driving database based on preset time periods.

[0025] On this basis, a historical motion feature set is constructed. The motion state characteristics of the historical target obstacle and the first target's historically associated obstacles, such as movement direction, speed change, and acceleration, are extracted and fused with the first historically associated motion data to form the first historical motion feature that reflects short-term interaction patterns. The motion characteristics of the second historically associated obstacle and the historical target obstacle within a preset time period are extracted to form the second historical motion feature that reflects long-term patterns. The two are combined to form a historical motion feature set that covers multiple time scales and interaction patterns.

[0026] Finally, let's focus on real-time obstacle avoidance decision-making. While driving, the autonomous vehicle uses visual sensors to detect the current target obstacle and related obstacles, collecting information such as position, speed, and direction of movement. This generates a set of current motion feature information. This information is then compared with historical motion feature sets for similarity (e.g., using Euclidean distance or cosine similarity algorithms) to select suitable motion features for processing. Based on this feature, the current obstacle trajectory is predicted. A safe driving area is defined based on the autonomous vehicle's current position and driving status. This is then input into an obstacle avoidance decision-making model (e.g., reinforcement learning, rule-based reasoning), which outputs path planning instructions to achieve obstacle avoidance control.

[0027] The historical motion feature set is obtained by processing the motion conditions of the historical target obstacle, the first target historically associated obstacle, and the second historical target associated obstacle, specifically including: Extracting motion feature information of the first target's historically associated obstacle and the historical target obstacle to obtain a first historical motion feature, and extracting motion feature information of the second historical target's associated obstacle to obtain a second historical motion feature; The first historical movement feature and the second historical movement feature are combined into a historical movement feature set.

[0028] This application focuses on constructing a historical motion feature set to support obstacle avoidance decisions for autonomous vehicles. Three key obstacle categories are first identified: historical target obstacles, first-target historically associated obstacles, and second-target historically associated obstacles. For first-target historically associated obstacles and historical target obstacles, their motion characteristics, such as direction of motion, speed changes, acceleration, and dynamic changes in relative position to the autonomous vehicle, are extracted and integrated to form a first historical motion feature set. This step explores the short-term motion patterns of obstacles that are directly associated with and closely interact with the historical target obstacles. Furthermore, for second-target historically associated obstacles, their motion characteristics, such as trajectory patterns and speed fluctuations within a specific historical period, are extracted separately to form a second historical motion feature set. This feature set primarily captures common long-term, potentially related obstacle motion characteristics. Finally, the first and second historical motion features are combined to form a historical motion feature set that covers the motion patterns of obstacles at varying degrees of correlation and over varying time spans. Subsequent autonomous vehicle driving can then match these historical features to the current scenario to aid in precise obstacle avoidance decisions.

[0029] Obstacle detection is performed on historical image information to obtain a historical obstacle detection result set, and a historical target obstacle is determined according to a first detection result of the historical obstacle detection result set, specifically comprising the following steps: Performing target detection on historical image information to obtain the location, shape and type information of obstacles to form a historical obstacle detection result set; The obstacle detection standard closest to the unmanned vehicle or posing the greatest threat to the driving safety of the unmanned vehicle is set as the detection standard, the first detection result that meets the detection standard is screened out from the historical obstacle detection result set, and the target obstacle is determined based on the first detection result.

[0030] This application first constructs a historical obstacle detection result set. During the driverless vehicle's historical driving phase, onboard cameras and other visual sensors continuously collect road scene images. Object detection algorithms (such as the YOLO series and Faster R-CNN deep learning models) process historical image information frame by frame to identify the location of obstacles in the image. Obstacles are located using pixel coordinates or actual road coordinates to determine their spatial distribution within the scene. The shape is determined, distinguishing between regular vehicle outlines and irregular pedestrian or debris shapes. The type is identified, classifying them as pedestrians, motor vehicles, non-motor vehicles, or static obstacles (such as roadblocks and piles). The position, shape, and type information of all obstacles in each frame are compiled and integrated to form a historical obstacle detection result set covering multiple scenarios and multiple obstacle data.

[0031] The selection criteria are based on the closest obstacle to the autonomous vehicle or the one that poses the greatest safety threat. The closest distance is quantified by calculating the distance between the obstacle and the autonomous vehicle in the image or in real space. The greatest safety threat is determined by combining obstacle type (e.g., high-speed motor vehicles pose a greater threat than pedestrians), motion state (e.g., vehicles that suddenly change lanes pose a greater threat), and location (e.g., obstacles in the center of the lane pose a greater threat than obstacles on the side of the road). Based on this criterion, the historical obstacle detection results set is traversed to select the obstacle detection data that meets the criteria, which is defined as the first detection result. This result focuses on the obstacle that poses the greatest immediate safety threat to the autonomous vehicle in the current historical scenario, and this is used as the basis for determining the historical target obstacle.

[0032] Obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle are marked as a historical associated obstacle set. The specific steps include: Determine the scope of the adjacent area based on the location and size of historical target obstacles; Obstacles located in adjacent areas are screened out from the historical obstacle detection result set to obtain a first obstacle association set; Filtering out a second detection result from a set of historical obstacle detection results, where the second detection result refers to the obstacle detection result that poses the greatest threat to driving safety; Marking obstacles in the historical associated obstacle set that have a linkage effect with the second detection result as a second obstacle association set; The first obstacle association set and the second obstacle association set are combined into a historical association obstacle set.

[0033] This application first constructs a first obstacle association set based on spatial proximity. The position (such as coordinates in the road scene) and size (length, width, height or image pixel size) of the historical target obstacle are key bases for dynamically determining the range of the adjacent area. If the obstacle is large (such as a large truck), the adjacent area is expanded to ensure that the space around it that may interact is covered; if the size is small (such as pedestrians), the range is reduced. Subsequently, the historical obstacle detection result set is traversed to filter out obstacles whose spatial positions are within the adjacent area. These obstacles have direct spatial interactions with the historical target obstacles due to their close physical distance (such as parallel vehicles, close pedestrians), and are formed into the first obstacle association set to capture associated obstacles in the spatial dimension.

[0034] The second obstacle association set is constructed based on threat linkage. The second obstacle detection result is selected from the historical obstacle detection results based on the criteria that pose the greatest threat to driving safety (combining factors such as obstacle type, motion state, and lane location, e.g., vehicles changing lanes at high speeds pose a higher threat than stationary roadblocks). This result represents the most dangerous obstacle source in the historical scenario. The historical associated obstacle set (preliminarily selected associated obstacles) is then analyzed for obstacles that have a "linked impact" with the second detection result. For example, if a vehicle changing lanes in the second detection result causes the vehicle behind it to brake or vehicles in adjacent lanes to evade, these obstacles affected by the chain reaction are marked as the second obstacle association set.

[0035] The first set of spatially adjacent obstacles in the focus area is combined with the second set of obstacles associated with the captured threat linkage to form a historical set of associated obstacles. This set includes obstacles directly adjacent to the historical target obstacle as well as indirectly associated obstacles caused by linkage effects from other high-threat obstacles, comprehensively covering the multiple associated factors in historical scenarios that may have affected the autonomous vehicle's obstacle avoidance decisions.

[0036] Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set by distance and speed to obtain first historical associated motion data specifically includes the following steps: Calculating the distance and speed of the obstacle in the first detection result to obtain a first motion parameter set; Calculating the distance and speed of the obstacle in the second detection result to obtain a second motion parameter set; Obtaining a historical correlation motion data set of relative distance change and relative speed change according to the first motion parameter set and the second motion parameter set; A motion correlation threshold range is preset, and first historical correlation motion data within the preset motion correlation threshold range is screened out from the historical correlation motion data set.

[0037] This application mines historical motion correlation data between obstacles to provide a dynamic interactive basis for obstacle avoidance decisions by autonomous vehicles. First, the distance and velocity parameters of the first detection result (key obstacles directly related to the historical target obstacle) and the second detection result (obstacles posing the greatest threat to driving safety) in the historical obstacle detection result set are calculated. For each obstacle in the first detection result, its real-time distance from the autonomous vehicle and its own velocity are calculated, combining its historical scene location information (e.g., relative coordinates relative to the autonomous vehicle) and time series data to form a first motion parameter set. Similarly, the second motion parameter set is obtained by performing the same process for the obstacles in the second detection result. Next, based on these two parameter sets, the relative distance changes (e.g., whether they are approaching or receding over time) and relative velocity changes (dynamic changes in the speed difference) between the two types of obstacles are determined to construct a historical correlation motion dataset reflecting the interaction between the two types of obstacles. Finally, a motion correlation threshold range is preset (defined based on historical experience or road safety standards to define the degree of motion correlation that is practical for obstacle avoidance). Data within this range is filtered from the historical correlation motion dataset to form the first historical correlation motion data. These data accurately capture the dynamic motion correlations between key obstacles in historical scenes, and can subsequently be used to analyze the movement patterns of obstacles, assist unmanned vehicles in predicting obstacle interaction risks during current driving, and optimize obstacle avoidance strategies.

[0038] Extracting a first target historically associated obstacle from the historically associated obstacle set according to the first historically associated motion data is specifically as follows: A first target-associated obstacle having a high degree of motion correlation with the historical target obstacle is extracted from the historically associated obstacle set according to the first historically associated motion data.

[0039] This application accurately screens key obstacles from a set of historically correlated obstacles. Using pre-calculated primary historically correlated motion data (information reflecting the dynamic motion correlation between key obstacles), this process is performed within a previously constructed set of historically correlated obstacles (including multiple, spatially adjacent, and threat-linked obstacles). By evaluating motion correlation data such as changes in relative distance and relative speed between obstacles, the application quantitatively assesses the degree of motion correlation between each obstacle in the set and the historical target obstacle. The obstacles with the highest correlation are then defined as the primary target historically correlated obstacles.

[0040] Extracting motion feature information of the first target historically associated obstacle and the historical target obstacle to obtain a first historical motion feature, and extracting motion feature information of the second historical target associated obstacle to obtain a second historical motion feature, specifically includes the following steps: The historical motion state feature set is obtained by extracting the motion direction, speed change and acceleration of the historical target obstacle and the first target historical associated obstacle respectively; The historical motion state feature set and the first historical associated motion data are combined into a first historical motion feature; Extracting obstacle motion data within a preset historical period from the historical driving data to obtain a second historical associated motion data set; extracting a second target historically associated obstacle from the historically associated obstacle set according to the second historically associated motion data; The second historical motion feature is obtained by respectively collecting motion feature information of the historical target obstacle and the second target historical associated obstacle within the historical period.

[0041] This application first extracts core motion state information. For the historical target obstacle (the obstacle that poses the most immediate threat to the autonomous vehicle's safety in the historical scenario) and the first target historically associated obstacle (an obstacle with a high degree of motion correlation with the historical target obstacle), the application focuses on three core motion state parameters: motion direction (such as the directional trends of going straight, turning left, and changing lanes), speed change (the magnitude and rate of acceleration and deceleration), and acceleration (dynamic changes in motion trends, such as sudden acceleration and constant deceleration). These parameters are accurately extracted by analyzing historical image sequences and on-board sensor data (such as millimeter-wave radar speed information) and integrated into a historical motion state feature set.

[0042] Next, the dynamic correlation data is integrated, introducing the previously calculated first historical correlation motion data (reflecting the dynamic interaction between the relative distance and relative speed changes between the historical target obstacle and the first target historical correlation obstacle). This data is then fused with the historical motion state feature set. For example, the historical motion state feature set might show "the historical target obstacle accelerates and moves straight, while the first target historical correlation obstacle decelerates and avoids it." The first historical correlation motion data then adds "the relative distance between the two decreases from 5 meters to 2 meters, and the relative speed difference decreases from 10 m / s to 3 m / s." The combination of these two forms the first historical motion feature, which captures both the obstacle's own motion state and the dynamic interaction between them.

[0043] The motion data (changes in position, speed, and direction over time) of all obstacles within a preset historical period (customizable, such as typical scenarios such as the morning rush hour and continuous rainy days in the past three months) are extracted from the unmanned vehicle historical driving database to form a second historical associated motion dataset.

[0044] Next, we extract the second target's historically associated obstacles. Based on the second historically associated motion dataset, we determine the motion correlation (such as long-term speed coordination and directional synchronization) between each obstacle in the historically associated obstacle set and the historical target obstacle. Obstacles with high correlation are selected and defined as the second target historically associated obstacles. For example, if a historical target obstacle has been traveling on a certain route for a long time and the second target historically associated obstacle frequently experiences alternating speed changes with it on the same road section, we select these obstacles. These obstacles reflect potential linkage relationships in long-term scenarios.

[0045] Finally, long-term motion features are collected and integrated. Within a preset historical period, motion feature information (such as the periodicity of movement direction and the time period regularity of speed changes) is collected from the historical target obstacle and the secondary target historical associated obstacles. This data, which reflects the long-term motion pattern, is integrated to construct the second historical motion feature.

[0046] Obtain the motion feature information of the current target obstacle and the current associated obstacles to obtain the current motion feature information set. Match the processed motion features from the historical motion feature set based on the current motion feature information set. Plan the driving path of the unmanned vehicle and perform obstacle avoidance control based on the processed motion features. Specifically, the following steps are included: The current motion feature information set includes the position, speed, and motion direction of the current target obstacle and the current associated obstacles; Matching and processing motion features from the historical motion feature set according to the current motion feature information set; Predict the motion trajectory dataset of the current target obstacle and the current associated obstacles based on the current motion feature information set; Determine the driving area of ​​the unmanned vehicle based on its current location and driving status; The motion trajectory dataset, driving area and processed motion features are input into the obstacle avoidance decision model to plan the driving path and perform obstacle avoidance control for the unmanned vehicle.

[0047] During the driving process of the unmanned vehicle in this application, the visual camera continuously captures the road ahead and combines the multi-source perception data of millimeter-wave radar and lidar to accurately identify the current target obstacle (such as a food delivery rider who suddenly enters the lane) and the current associated obstacles (such as private vehicles that suddenly brake around the rider, and adjacent motor vehicles whose trajectories change due to avoidance). The image frames are analyzed by computer vision algorithms, and the distance and speed measurements of the sensors are combined to extract the position of these obstacles (based on the high-precision map coordinate system, accurate to the centimeter level, to clarify their coordinates in the road grid), speed (distinguishing instantaneous speed from movement trend, such as whether the vehicle decelerates uniformly to 5m / s or brakes suddenly to a standstill), and direction of movement (identifying complex trajectories such as diagonal lane changes, right-angle turns, or serpentine movements). These multi-dimensional data are then integrated into the current motion feature information set.

[0048] The historical motion feature set constructed earlier stores the motion patterns of "target obstacles + associated obstacles" from a vast number of historical scenarios (e.g., the interaction patterns of vehicle chain braking on slippery roads during rainy weather, and the motion coordination characteristics of rush hour traffic). After obtaining the current motion feature information set, the most suitable motion features are selected by calculating the similarity between the current obstacle's motion parameters (position change rate, velocity vector difference) and historical features. For example, if the current scenario is "a target obstacle (a disabled vehicle) stationary in the lane, and an associated obstacle (an oncoming vehicle) suddenly changes lanes," the motion features corresponding to the effective obstacle avoidance strategy at that time are retrieved from the historical database for similar scenarios such as "a static roadblock causing the following vehicle to avoid danger" (e.g., the trajectory parameters of the self-driving car's historical "early deceleration and small lane change").

[0049] Based on the current motion feature information set, for obstacles moving in a uniform straight line (such as a truck traveling at a uniform speed), the classic model of "position = initial position + speed × time" is used to predict the trajectory; for complex interactive scenarios (such as multi-vehicle games at intersections), time series prediction models such as LSTM (long short-term memory network) are introduced and combined with historical trajectory data to learn the obstacle motion pattern to predict the motion trajectory of the current target obstacle and related obstacles in the next few seconds (such as 3-5 seconds, covering the decision cycle of braking and lane changing of the unmanned vehicle) to generate a motion trajectory dataset.

[0050] The driving area is dynamically defined based on the vehicle's current position (using GPS + inertial navigation fusion positioning to determine the precise lane on the road and distance to intersections / exits) and driving status (real-time speed, steering angle, throttle / brake opening, and whether it is cruising at a constant speed, preparing to accelerate for overtaking, or about to slow down and stop). If driving at a constant speed (60 km / h) on a straight road, the driving area is a rectangular space 100 meters in front and behind the current lane and 1.5 meters to the left and right (lane width) to ensure safety when driving straight. If you turn on the turn signal to change lanes, the driving area will be expanded to the "travelable area of ​​the target lane" and the space behind the original lane will be reduced to avoid conflict with the vehicle behind. If an emergency braking need is detected, the driving area will focus on the "braking buffer zone within a safe distance ahead", giving priority to ensuring parking without rear-end collisions.

[0051] The obstacle avoidance decision model is fed with a trajectory dataset (where obstacles will appear in the future), a driving area (where the vehicle can go), and processed motion features (historically verified obstacle avoidance patterns). The model architecture is flexibly adaptable.

[0052] Rule-driven: Based on traffic regulations and safe driving experience, hard rules such as "prioritize lane changes when encountering static obstacles, and slow down to less than 5m / s if lane changes are not feasible" are set to directly output path instructions.

[0053] Reinforcement Learning: Through "reward-and-penalty" training based on historical scenarios (+1 point for successful obstacle avoidance, -10 points for collision), the model autonomously learns optimal strategies. In complex scenarios like "multi-vehicle collisions at intersections," it outputs flexible solutions such as "slightly slowing down to observe, then looking for an opportunity to maneuver diagonally."

[0054] After calculation, the model outputs a specific driving path (such as "change lanes left to lane 3 and maintain driving at 40km / h") and converts it into control signals for the accelerator, brake, and steering, adjusting the movement state of the unmanned vehicle in real time to achieve a seamless connection of "sensing danger → planning risk avoidance → executing action".

[0055] Example 2 further explains the unmanned vehicle obstacle avoidance system based on visual detection proposed by the present invention. An unmanned vehicle obstacle avoidance system based on visual detection, comprising: Extraction module: extracts historical image information of the unmanned vehicle during its historical driving process, performs obstacle detection on the historical image information to obtain a historical obstacle detection result set, and determines the historical target obstacle based on the first detection result of the historical obstacle detection result set; Marking module: Marks obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle as a historical associated obstacle set; Processing module: Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set based on distance and speed to obtain first historical associated motion data, and extracting the first target historical associated obstacle from the historical associated obstacle set based on the first historical associated motion data; Screening module: Screening out the second historical target-related obstacles in a preset historical period from historical driving data; Processing module: Processing the movement of the historical target obstacle, the first target historically associated obstacle, and the second target historically associated obstacle to obtain a historical movement feature set; Obstacle avoidance module: collects the current target obstacles of the unmanned vehicle during its current driving process, marks the obstacles associated with the current target obstacles as currently associated obstacles, obtains the motion feature information of the current target obstacles and the currently associated obstacles to obtain the current motion feature information set, matches the processed motion features from the historical motion feature set based on the current motion feature information set, and plans the driving path of the unmanned vehicle and performs obstacle avoidance control based on the processed motion features.

[0056] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, an obstacle avoidance method for an unmanned vehicle based on visual detection is implemented.

[0057] like Figure 3As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute a method for avoiding obstacles for an unmanned vehicle based on visual detection.

[0058] Furthermore, the logic instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0059] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute an unmanned vehicle obstacle avoidance method based on visual detection.

[0060] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform an unmanned vehicle obstacle avoidance method based on visual detection.

[0061] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0062] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for avoiding obstacles in an unmanned vehicle based on visual detection, characterized in that: The method comprises the following steps: Extracting historical image information of the unmanned vehicle during its historical driving process, performing obstacle detection on the historical image information to obtain a historical obstacle detection result set, and determining a historical target obstacle based on a first detection result of the historical obstacle detection result set; The obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle are marked as the historical associated obstacle set; Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set based on distance and speed to obtain first historical associated motion data, and extracting a first target historical associated obstacle from the historical associated obstacle set based on the first historical associated motion data; Filtering the second historical target-related obstacles in a preset historical period from the historical driving data; Processing the motion conditions of the historical target obstacle, the first target historically associated obstacle, and the second target historically associated obstacle to obtain a historical motion feature set; The current target obstacles of the unmanned vehicle during its current driving process are collected, obstacles associated with the current target obstacles are marked as currently associated obstacles, motion feature information of the current target obstacles and the currently associated obstacles are obtained to obtain a current motion feature information set, processing motion features are matched from a historical motion feature set based on the current motion feature information set, and the driving path of the unmanned vehicle is planned and obstacle avoidance control is performed based on the processed motion features.

2. The unmanned vehicle obstacle avoidance method based on visual detection according to claim 1, characterized in that: The historical motion feature set is obtained by processing the motion conditions of the historical target obstacle, the first target historically associated obstacle, and the second historical target associated obstacle, specifically including: Extracting motion feature information of the first target's historically associated obstacle and the historical target obstacle to obtain a first historical motion feature, and extracting motion feature information of the second historical target's associated obstacle to obtain a second historical motion feature; The first historical movement feature and the second historical movement feature are combined into a historical movement feature set.

3. The unmanned vehicle obstacle avoidance method based on visual detection according to claim 2, characterized in that: Obstacle detection is performed on historical image information to obtain a historical obstacle detection result set, and a historical target obstacle is determined according to a first detection result of the historical obstacle detection result set, specifically comprising the following steps: Performing target detection on historical image information to obtain the location, shape and type information of obstacles to form a historical obstacle detection result set; The obstacle detection standard closest to the unmanned vehicle or posing the greatest threat to the driving safety of the unmanned vehicle is set as the detection standard, the first detection result that meets the detection standard is screened out from the historical obstacle detection result set, and the target obstacle is determined based on the first detection result.

4. The unmanned vehicle obstacle avoidance method based on visual detection according to claim 3 is characterized in that: Obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle are marked as a historical associated obstacle set. The specific steps include: Determine the scope of the adjacent area based on the location and size of historical target obstacles; Obstacles located in adjacent areas are screened out from the historical obstacle detection result set to obtain a first obstacle association set; Filtering a second detection result from a set of historical obstacle detection results, where the second detection result refers to an obstacle detection result that poses the greatest threat to driving safety; Marking obstacles in the historical associated obstacle set that have a linkage effect with the second detection result as a second obstacle association set; The first obstacle association set and the second obstacle association set are combined into a historical association obstacle set.

5. The unmanned vehicle obstacle avoidance method based on visual detection according to claim 4, characterized in that: Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set by distance and speed to obtain first historical associated motion data specifically includes the following steps: Calculating the distance and speed of the obstacle in the first detection result to obtain a first motion parameter set; Calculating the distance and speed of the obstacle in the second detection result to obtain a second motion parameter set; Obtaining a historical correlation motion data set of relative distance change and relative speed change according to the first motion parameter set and the second motion parameter set; A motion correlation threshold range is preset, and first historical correlation motion data within the preset motion correlation threshold range is screened out from the historical correlation motion data set.

6. The unmanned vehicle obstacle avoidance method based on visual detection according to claim 5, characterized in that: Extracting a first target historically associated obstacle from the historically associated obstacle set according to the first historically associated motion data is specifically as follows: A first target-associated obstacle having a high degree of motion correlation with the historical target obstacle is extracted from the historically associated obstacle set according to the first historically associated motion data.

7. The unmanned vehicle obstacle avoidance method based on visual detection according to claim 6, characterized in that: Extracting motion feature information of the first target historically associated obstacle and the historical target obstacle to obtain a first historical motion feature, and extracting motion feature information of the second historical target associated obstacle to obtain a second historical motion feature, specifically includes the following steps: The historical motion state feature set is obtained by extracting the motion direction, speed change and acceleration of the historical target obstacle and the first target historical associated obstacle respectively; The historical motion state feature set and the first historical associated motion data are combined into a first historical motion feature; Extracting obstacle motion data within a preset historical period from the historical driving data to obtain a second historical associated motion data set; extracting a second target historically associated obstacle from the historically associated obstacle set according to the second historically associated motion data; The second historical motion feature is obtained by respectively collecting motion feature information of the historical target obstacle and the second target historical associated obstacle within the historical period.

8. The unmanned vehicle obstacle avoidance method based on visual detection according to claim 7, characterized in that: Obtain the motion feature information of the current target obstacle and the current associated obstacles to obtain the current motion feature information set. Match the processed motion features from the historical motion feature set based on the current motion feature information set. Plan the driving path of the unmanned vehicle and perform obstacle avoidance control based on the processed motion features. Specifically, the following steps are included: The current motion feature information set includes the position, speed and motion direction of the current target obstacle and the current associated obstacles; Matching and processing motion features from the historical motion feature set according to the current motion feature information set; Predict the motion trajectory dataset of the current target obstacle and the current associated obstacles based on the current motion feature information set; Determine the driving area of ​​the unmanned vehicle based on its current location and driving status; The motion trajectory dataset, driving area and processed motion features are input into the obstacle avoidance decision model to plan the driving path and perform obstacle avoidance control for the unmanned vehicle.

9. An unmanned vehicle obstacle avoidance system based on visual detection, applied to an unmanned vehicle obstacle avoidance method based on visual detection according to any one of claims 1 to 8, characterized in that: include: Extraction module: extracts historical image information of the unmanned vehicle during its historical driving process, performs obstacle detection on the historical image information to obtain a historical obstacle detection result set, and determines the historical target obstacle based on the first detection result of the historical obstacle detection result set; Marking module: Marks obstacles in the historical obstacle detection result set that are adjacent to or have a linkage effect on the historical target obstacle as a historical associated obstacle set; Processing module: Processing and analyzing the first detection result and the second detection result of the historical obstacle detection result set based on distance and speed to obtain first historical associated motion data, and extracting the first target historical associated obstacle from the historical associated obstacle set based on the first historical associated motion data; Screening module: Screening out the second historical target-related obstacles in a preset historical period from historical driving data; Processing module: Processing the movement of the historical target obstacle, the first target historically associated obstacle, and the second target historically associated obstacle to obtain a historical movement feature set; Obstacle avoidance module: collects the current target obstacles of the unmanned vehicle during its current driving process, marks the obstacles associated with the current target obstacles as currently associated obstacles, obtains the motion feature information of the current target obstacles and the currently associated obstacles to obtain the current motion feature information set, matches the processed motion features from the historical motion feature set based on the current motion feature information set, and plans the driving path of the unmanned vehicle and performs obstacle avoidance control based on the processed motion features.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, an unmanned vehicle obstacle avoidance method based on visual detection as described in any one of claims 1 to 8 is implemented.

Images