Dynamic and static obstacle recognition method based on YOLO model

Abstract

The present invention relates to the field of target detection technology, and in particular to a method for identifying dynamic and static obstacles based on the YOLO model. The method comprises: in an existing YOLOv8 network, replacing the C2f module of the backbone network with a SACM module to form a lightweight C2f_SACM module; replacing the C2f module in the native PANet in the neck network with a HAT module for multi-scale feature fusion; using experimental data obtained in the experiment as a dataset to train the YOLO-SH network obtained in step S1 to obtain a YOLO-SH model, and outputting a detection frame for each obstacle; selecting the geometric center point of the detection frame as the feature point of the obstacle, analyzing the movement of the feature point in the interval frame image, and setting feature point movement thresholds for static and dynamic obstacles respectively to achieve differentiation between dynamic and static obstacles. The method has the advantages of simplifying the calculation process and improving processing efficiency, thereby achieving accurate obstacle identification and classification.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a dynamic and static obstacle recognition method based on a YOLO model. Background Art

[0002] With the continuous advancement of modern automation and intelligent robotics, especially in specific environments such as smart factories and warehouses, obstacle detection and classification must be real-time, efficient, accurate, and cost-effective. To achieve efficient and safe operation, automated guided vehicles (AGVs) and other types of mobile robots require strong environmental perception capabilities, particularly the ability to accurately identify and distinguish dynamic and static obstacles. Currently, there are two main approaches for obstacle detection and dynamic and static obstacle differentiation: dynamic obstacle detection based on multi-sensor fusion and dynamic obstacle detection based on optical flow. The first approach first uses information collected by multiple sensors to obtain obstacle information within a known map and then determines whether the obstacle is moving or not based on its motion relative to the map. A typical example is the dynamic and static obstacle classification method based on 3D point cloud registration. This approach is costly, especially when large coverage is required. Multiple sensors not only increase the overall system cost but also make data processing and maintenance more difficult and complex. Another approach uses optical flow, which first uses machine learning to detect obstacles and then analyzes pixel displacement between consecutive frames to estimate the object's direction and speed, thereby distinguishing between dynamic and static obstacles. This method has good flexibility in theory, but in reality the optical flow method is computationally intensive and extremely complex, making it difficult for mobile robots with limited resources to meet the requirements of real-time processing. It is also easily affected by changes in lighting, and its accuracy is limited in dynamic backgrounds.

[0003] In summary, the existing technology has problems such as high cost, poor adaptability, high computational complexity and insufficient real-time performance. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a dynamic and static obstacle recognition method based on the YOLO model.

[0005] The present invention aims to provide a method for identifying dynamic and static obstacles based on the YOLO model, which specifically includes the following steps:

[0006] S1. Build a YOLO-SH network: In the existing YOLOv8 network, replace the C2f module in the backbone network with the SACM module to form a lightweight C2f_SACM module; replace the C2f module in the native PANet in the neck network with the HAT module for multi-scale feature fusion;

[0007] S2. Use the experimental data obtained in the experiment as a dataset to train the YOLO-SH network obtained in step S1 to obtain a YOLO-SH model;

[0008] S3. Based on the detection results of the YOLO-SH model, output the detection frame of each obstacle; select the geometric center point of the detection frame as the feature point of the obstacle, and analyze the movement of the feature point in the interval frame image; according to the movement of the feature point, set the feature point movement threshold of static obstacles and dynamic obstacles respectively to distinguish between dynamic and static obstacles.

[0009] Preferably, the C2f_SACM module first performs preliminary processing on the input feature map through a basic convolution layer, adjusts the number of channels of the feature map and extracts basic features; utilizes the dynamic convolution kernel adjustment mechanism to dynamically adjust the convolution kernel parameters based on the content of the input feature map to generate a set of optimized convolution kernels; applies the optimized convolution kernel to perform a convolution operation on the feature map to extract an enhanced feature map; then applies the spatial attention mechanism to enhance the focus on key areas and generate a weighted feature map; merges the weighted feature map with the original input feature map to obtain the final output feature map.

[0010] Preferably, dynamically adjusting the convolution kernel parameters includes: first calculating the importance weight W of each position, using the spatial attention mechanism to evaluate the importance weight W of each position; and adjusting the predefined basic convolution kernel K according to the obtained weight W to generate a new convolution kernel K'.

[0011] Preferably, the HAT module resizes each input feature map so that the feature maps have the same resolution; convolution is performed point by point on each size-aligned feature map, global information of each channel is extracted through global average pooling, and then weights are generated through a fully connected layer or a convolutional layer, and all feature maps are weighted summed according to the weights to form the final fused feature map.

[0012] Preferably, the experiment in step S2 has the following specific process: the mobile car carrying the camera moves continuously, static obstacles and dynamic obstacles appear in front continuously, and the camera collects images and performs annotation processing.

[0013] Preferably, the camera is installed 40 to 50 cm above the mobile cart to ensure an unobstructed field of view; the mobile cart moves forward in a uniform linear motion at a speed of 1 to 2 m / s.

[0014] Preferably, the method for setting the feature point movement threshold is: according to the movement of the feature point, the static obstacle feature point movement threshold is set in the X and Y directions respectively ( X α , Y α ) and the dynamic obstacle feature point movement threshold (X β , Y β );

[0015] If the movement of the feature point in both directions does not exceed the static obstacle feature point movement threshold, the obstacle is determined to be a static obstacle;

[0016] If the movement of the feature point in any direction exceeds both the static obstacle feature point movement threshold and the dynamic obstacle feature point movement threshold, the obstacle is determined to be a dynamic obstacle;

[0017] In other cases, the obstacle is considered a static obstacle.

[0018] Preferably, the method for selecting the static obstacle feature point movement threshold is: based on the maximum value of the static obstacle feature point movement between interval frames ( X αmax , Y αmax ) Determine the static obstacle feature point movement threshold; if the square difference between the maximum value and the second maximum value is greater than 1.2 times the population standard deviation of the total sample data, then select the second maximum value as the new static obstacle feature point movement threshold;

[0019] The method for selecting the dynamic obstacle feature point movement threshold is: based on the minimum value of the dynamic obstacle feature point movement between interval frames ( X βmin , Y βmin ) Determine the moving threshold of the dynamic obstacle feature point; if the square difference between the minimum value and the second minimum value is greater than 1.2 times the population standard deviation of the total sample data, then select the second minimum value as the new moving threshold of the static obstacle feature point.

[0020] Preferably, the movement of the feature point in the interval frame image includes the distance and direction of movement.

[0021] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0022] This paper provides a method for detecting dynamic and static obstacles that combines the YOLO model with a threshold judgment method based on feature point movement. This method optimizes applications under conditions of constant vehicle speed, reduces reliance on expensive hardware, and reduces overall system costs. By simplifying the computational process and improving processing efficiency, it achieves real-time and accurate obstacle identification and classification while ensuring operational safety, significantly enhancing the system's adaptability and robustness in complex and changing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1This is an overall flow chart of a dynamic and static obstacle recognition method based on a YOLO model provided according to an embodiment of the present invention.

[0024] Figure 2 2 is a structural diagram of an improved YOLO-SH detection model provided according to an embodiment of the present invention.

[0025] Figure 3 2 is a structural diagram of a SACM module according to an embodiment of the present invention.

[0026] Figure 4 2 is a structural diagram of a C2f_SACM module provided according to an embodiment of the present invention.

[0027] Figure 5 2 is a structural diagram of a HAT module provided according to an embodiment of the present invention.

[0028] Figure 6 This is an experimental scenario and data annotation diagram provided according to an embodiment of the present invention.

[0029] Figure 7 This is a diagram of the post-processing results of the YOLO-SH model training provided by an embodiment of the present invention.

[0030] Figure 8 It is an experimental case diagram provided according to an embodiment of the present invention.

[0031] Figure 9 is a feature point movement processing diagram provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, identical modules are denoted by identical reference numerals. In the case of identical reference numerals, their names and functions are also identical. Therefore, their detailed description will not be repeated.

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0034] The present invention provides a method for identifying dynamic and static obstacles based on the YOLO model. Figure 1 , specifically including the following steps:

[0035] S1. Build a YOLO-SH network (an improved YOLOv8 network). Improve its feature extraction and fusion networks to reduce model size while maintaining good accuracy, making it easier to deploy on a development board. In the existing YOLOv8 network, replace the C2f module in the backbone network with the Bottleneck module in the original C2f module and replace it with the SACM module, forming a lightweight C2f_SACM module. In the neck network, replace the C2f module in the original PANet (Path Aggregation Network) with the HAT module, which is responsible for multi-scale feature fusion.

[0036] The improved YOLO-SH network structure is as follows Figure 2 shown.

[0037] The Backbone network is lightweight: Combining a dynamic convolution kernel adjustment mechanism with a spatial attention mechanism, a minimalist yet efficient lightweight Smart Adaptive Convolution Module (SACM) is designed. The Bottleneck module in the original network's C2f module is replaced with a SACM module, forming the innovative C2f_SACM lightweight module. This module then replaces the original C2f module. This not only simplifies the network structure, reduces the amount of computation and parameters, and improves the model's runtime and computational efficiency, but also enhances the model's spatial feature extraction capabilities, ensuring high-precision detection performance on detection tasks. The C2f_SACM module works as follows:

[0038] Basic convolution layer processing: First, the feature map is input and processed through a basic convolution layer (BaseConv) to adjust the number of channels of the feature map and extract basic features;

[0039] Dynamic convolution kernel adjustment: Using the dynamic kernel adjustment mechanism, the convolution kernel parameters are dynamically adjusted based on the content of the input feature map to generate a set of optimized convolution kernels. In this step, the importance weight W of each position is first calculated and the spatial attention mechanism is used to evaluate the importance weight W of each position. Then, the predefined base convolution kernel K is adjusted according to these weights W to generate a new convolution kernel K'.

[0040] Optimized convolution operation: Apply the optimized convolution kernel K' to perform convolution operation on the feature map to extract the enhanced feature map;

[0041] Weighted feature map: Spatial attention is applied again to enhance the focus on key areas and generate a weighted feature map;

[0042] Feature merging: Merge the weighted feature map with the original input feature map to obtain the final output feature map.

[0043] The whole process is shown in the following formula:

[0044] X1=BaseConv(X);

[0045] W=Spatial Attention(X1);

[0046] K'=Adjust Kernel(K,W);

[0047] X2=Conv(X1,K');

[0048] X3=Spatial Attention(X2);

[0049] Y=Concat(X3,X);

[0050] Among them, X represents the input feature, Y represents the output feature, X1, X2, X3 represent the intermediate features, BaseConv represents the basic convolution layer, W is the spatial attention weight, and K' is the adjusted convolution kernel. The SACM module structure is as follows Figure 3 , C2f_SACM module structure is as follows Figure 4 .

[0051] The Neck network is lightweight: the C2f module in the native PANet (Path Aggregation Network) is replaced with the HAT module, responsible for multi-scale feature fusion. The HAT module integrates feature maps from different Backbone layers and introduces lightweight transformations and attention mechanisms to reduce computation and parameter requirements while maintaining model accuracy. The main goal is to enhance the model's ability to utilize feature information at different levels through efficient fusion and lightweight processing of multi-scale features. The HAT module works as follows:

[0052] Resizing: resize (upsample or downsample) each input feature map to make it have the same resolution; this step ensures that feature maps at different levels can be effectively fused in the spatial dimension;

[0053] Lightweight Transform: Apply a Lightweight Transform to each size-aligned feature map to enhance its expressiveness and reduce redundant information. LightTransform uses a simple pointwise convolution to reduce computational complexity.

[0054] Channel attention mechanism: Global average pooling (GAP) is used to extract global information from each channel. Then, weights are generated through a fully connected layer or convolutional layer. All feature maps are weighted and summed according to the weights to form the final fused feature map. The specific implementation is shown in the following formula:

[0055] Xi'=Resize(Xi)

[0056] Xi”=Light Transform(Xi')

[0057] W=Channel Attention({Xi”})

[0058] Yhat=∑(Wi×Xi”)

[0059] Among them, Xi is the input multi-scale feature, Yhat is the output feature, Xi' and Xi" are intermediate features, LightTransform is point-by-point convolution, and Channel Attention is a channel attention mechanism that introduces Global Average Pooling (GAP). The HAT module structure is as follows Figure 5 .

[0060] S2. Use the experimental data obtained from a large number of experiments as a dataset to train the YOLO-SH network obtained in step S1 to obtain the YOLO-SH model. The YOLO-SH model can detect obstacles in the captured images and roughly distinguish the specific types of obstacles (static or dynamic). Finally, the model can separately select two different types of obstacles.

[0061] Specifically, the experimental process is as follows: a mobile car equipped with a camera keeps moving, and static obstacles (boxes) and dynamic obstacles (people) keep appearing in front of it. The images collected by the camera are annotated and processed and then used as a data set for model training.

[0062] The experimental scene is as follows Figure 6 As shown in the attached figure, the image data processed after YOLO-SH model training is shown in the attached figure. Figure 7 shown.

[0063] S3. Based on the detection results of the YOLO-SH model, output the detection frame of each obstacle; for each detected obstacle, select the geometric center point of the detection frame as the feature point of the obstacle; collect image data through experiments; use these collected images as a data set to analyze the movement of the obstacle feature points in the interval frame images, including distance and direction, such as (X: -9, Y: +21); according to the movement of the feature points, set the feature point movement thresholds for static obstacles and dynamic obstacles respectively, so as to distinguish between dynamic and static obstacles.

[0064] The specific experimental scenario is as follows: a car equipped with a camera travels in a straight line at a constant speed, static obstacles are placed in the center and edge of the camera's field of view, and dynamic obstacles enter the camera's field of view in different ways (crossing, diagonally, moving vertically or away from the camera) to collect image data.

[0065] Setting of feature point movement threshold: According to the movement of feature points, set the static obstacle feature point movement threshold in X and Y directions respectively ( X α , Y α ) and the dynamic obstacle feature point movement threshold ( X β , Y β If the movement of the feature point in both directions does not exceed the static obstacle feature point movement threshold, the obstacle is determined to be a static obstacle; if the movement of the feature point in any direction exceeds both the static obstacle feature point movement threshold and the dynamic obstacle feature point movement threshold, the obstacle is determined to be a dynamic obstacle; in other cases, the obstacle is also determined to be a static obstacle.

[0066] The threshold selection method is as follows: static obstacle feature point moving threshold ( X α , Y α ) is usually based on the maximum value of the feature point movement of the static obstacle between interval frames ( X αmax , Y αmax ) is determined. However, to account for potential errors in the information collection process, if the squared difference between the maximum and the next maximum is greater than 1.2 times the population standard deviation of the total sample data, the next maximum is selected as the new threshold. This method has been proven to not only account for the variability of the data itself but also take into account the uncertainties in practical applications, improving the adaptability and reliability of the system.

[0067] The selection method of the dynamic obstacle feature point movement threshold is similar, but the reference is the minimum value of the dynamic obstacle feature point movement between interval frames ( Xβmin , Y βmin ). It should be noted that these thresholds are closely related to factors such as the speed and field of view of the camera mounted on the car. Figure 8 As shown in the attached figure, the feature point movement process is as follows: Figure 9 shown.

[0068] The specific implementation formula is as follows:

[0069] like

[0070]

[0071] but = , = ;otherwise = , = .

[0072] like

[0073]

[0074] but = , = otherwise = , = .

[0075] like and It is judged as a static obstacle.

[0076] like and ,or and It is determined to be a dynamic obstacle.

[0077] in: , is the static obstacle feature point moving threshold, ( , ) is the maximum value of the movement of static obstacle feature points between interval frames in the dataset, ( , ) is the second maximum value of the movement of static obstacle feature points between interval frames, ( , ) is the movement of the static obstacle feature point at time t, where , ,T is the number of frames in the acquisition interval. ( , ) is the moving threshold of dynamic obstacle feature points, ( , ) is the minimum value of the dynamic obstacle feature point movement between interval frames in the dataset, ( , ) is the sub-minimum value of the movement of static obstacle feature points between interval frames, ( , ) is the movement of the dynamic obstacle feature point at time t, where , ,T is the number of frames in the acquisition interval.

[0078] In this specific embodiment, the camera was mounted 45 cm above the mobile cart, with an unobstructed field of view and ample lighting in the test area. During the experiment in step S3, the mobile cart moved forward in a uniform linear motion at a speed of 1.5 m / s. If the camera model, test area, agent speed, or obstacle type differ, re-data collection will be required to train the model and re-select the threshold according to the method in step S3.

[0079] The key technical points of this invention include: (1) A new technical solution that combines an improved YOLOv8 model with feature point motion analysis to detect and distinguish dynamic and static obstacles, which not only improves detection accuracy, but also enhances adaptability and feasibility at a low cost. (2) A threshold judgment algorithm based on the displacement of obstacle feature points in interval frame images under the condition of a vehicle traveling at a constant speed is proposed. Users can select the appropriate threshold based on the obstacle data collected in the actual scene. (3) A set of easy-to-implement integrated solutions from image acquisition to obstacle detection and classification is provided, which simplifies system design and shortens the development cycle.

[0080] Other alternatives of the present invention include:

[0081] (1) In addition to using the YOLOv8 model, other advanced deep learning models can also be used for target detection, such as Faster R-CNN, SSD (Single Shot Multibox Detector) or Efficient Det.

[0082] (2) The present invention uses a threshold judgment method based on the displacement of the geometric center point to distinguish between dynamic and static obstacles. In addition to this method, optical flow method or Kanade-Lucas-Tomasi (KLT) feature tracking can also be used to distinguish between dynamic and static obstacles.

[0083] (3) The present invention makes lightweight improvements to the YOLOv8 model. In addition to making lightweight improvements to the YOLOv8 model, the present invention can also use technologies such as knowledge distillation, pruning, or quantization to reduce the model size and computational complexity.

[0084] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0085] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for identifying dynamic and static obstacles based on the YOLO model, characterized by: The specific steps include: S1. Build a YOLO-SH network: In the existing YOLOv8 network, replace the C2f module in the backbone network with the SACM module to form a lightweight C2f_SACM module; replace the C2f module in the native PANet in the neck network with the HAT module for multi-scale feature fusion; The C2f_SACM module performs preliminary processing on the input feature map through a basic convolution layer, adjusts the number of channels of the feature map and extracts basic features; uses a dynamic convolution kernel adjustment mechanism to dynamically adjust the convolution kernel parameters based on the content of the input feature map to generate a set of optimized convolution kernels; applies the optimized convolution kernels to perform a convolution operation on the feature map to extract an enhanced feature map; Then apply the spatial attention mechanism to enhance the focus on key areas and generate weighted feature maps; Merge the weighted feature map with the original input feature map to obtain the final output feature map; The HAT module resizes each input feature map so that the feature maps have the same resolution; Convolution is performed point by point on each size-aligned feature map, and global information of each channel is extracted through global average pooling. Then, weights are generated through a fully connected layer or a convolutional layer, and all feature maps are weighted summed according to the weights to form the final fused feature map. S2. Use the experimental data obtained in the experiment as a dataset to train the YOLO-SH network obtained in step S1 to obtain a YOLO-SH model; S3. Based on the detection results of the YOLO-SH model, output the detection frame of each obstacle; select the geometric center point of the detection frame as the feature point of the obstacle, and analyze the movement of the feature point in the interval frame image; according to the movement of the feature point, set the feature point movement threshold of static obstacles and dynamic obstacles respectively to distinguish between dynamic and static obstacles.

2. The method for identifying dynamic and static obstacles based on the YOLO model according to claim 1, wherein: The dynamic adjustment of convolution kernel parameters includes: first calculating the importance weight W of each position, using the spatial attention mechanism to evaluate the importance weight W of each position; according to the obtained weight W, adjusting the predefined basic convolution kernel K to generate a new convolution kernel K'.

3. The method for identifying dynamic and static obstacles based on the YOLO model according to claim 1, wherein: The specific process of the experiment in step S2 is as follows: the mobile car carrying the camera moves continuously, static obstacles and dynamic obstacles appear in front of it, and the camera collects images and performs annotation processing.

4. The method for identifying dynamic and static obstacles based on the YOLO model according to claim 3, wherein: The camera is installed 40-50 cm above the mobile cart to ensure an unobstructed field of view; the mobile cart moves forward in a uniform linear motion at a speed of 1-2 m / s.

5. The method for identifying dynamic and static obstacles based on the YOLO model according to claim 1, wherein: The method for setting the feature point movement threshold is as follows: according to the movement of the feature point, the static obstacle feature point movement threshold is set in the X and Y directions respectively ( X α , Y α ) and the dynamic obstacle feature point movement threshold ( X β , Y β ); If the movement of the feature point in both directions does not exceed the static obstacle feature point movement threshold, the obstacle is determined to be a static obstacle; If the movement of the feature point in any direction exceeds both the static obstacle feature point movement threshold and the dynamic obstacle feature point movement threshold, the obstacle is determined to be a dynamic obstacle; In other cases, the obstacle is considered a static obstacle.

6. The method for identifying dynamic and static obstacles based on the YOLO model according to claim 5, characterized in that: The method for selecting the static obstacle feature point movement threshold is: based on the maximum value of the static obstacle feature point movement between interval frames ( X αmax , Y αmax ) Determine the static obstacle feature point movement threshold; if the square difference between the maximum value and the second maximum value is greater than 1.2 times the population standard deviation of the total sample data, then select the second maximum value as the new static obstacle feature point movement threshold; The method for selecting the dynamic obstacle feature point movement threshold is: based on the minimum value of the dynamic obstacle feature point movement between interval frames ( X βmin , Y βmin ) Determine the moving threshold of the dynamic obstacle feature point; if the square difference between the minimum value and the second minimum value is greater than 1.2 times the population standard deviation of the total sample data, then select the second minimum value as the new moving threshold of the static obstacle feature point.

7. The method for identifying dynamic and static obstacles based on the YOLO model according to claim 1, wherein: The movement of the feature point in the interval frame image, including the distance and direction of movement.

Images