Blind guiding and obstacle avoiding system based on improved YOLOv5s

Abstract

The invention discloses an improved YOLOv5s-based blind guiding and obstacle avoiding system, which comprises an image acquisition module, and is characterized in that the image acquisition module is responsible for acquiring road condition information in front of a visually impaired person during walking, automatically processing a video stream into a format accepted by a subsequent module, and transmitting the video stream to a target detection and tracking module; the target detection and tracking module is responsible for processing an incoming video stream frame by frame, and separating obstacles and blind guiding objects from a complex sidewalk environment through an improved YOLOv5s target detection algorithm; the method has the advantages that the blind guiding and obstacle avoiding method based on the target detection technology is adopted, the visually impaired person can be accurately informed of the category of a blind guiding object or an obstacle in front, and the visually impaired person can further know road condition information in front of walking; target detection and target tracking technologies are combined and applied to the field of blind guiding and obstacle avoidance, the blind guiding and obstacle avoidance effect can be improved, and the accuracy of blind guiding and obstacle avoidance service is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical fields of deep learning, computer vision and blind-guiding obstacle avoidance, and in particular relates to a blind-guiding obstacle avoidance system based on improved YOLOv5s. Background Art

[0002] According to data released by the National Bureau of Statistics, by the end of 2023, the number of visually impaired people in my country will exceed 28.56 million, and about one in every 80 Chinese people's lives will be affected by visual impairment; relevant surveys show that among the visually impaired, about 9% will go out independently four to six times a week, about 15% will go out independently one to three times a week, and another 30% of the visually impaired people basically do not go out; although my country ranks first in the world in the mileage of blind paths, the visually impaired rarely or almost never go out. The fundamental reason for this situation is that the planning and construction of early urban roads could not keep pace with the rapid development of cities in a timely manner, and the construction workers did not pay enough attention to the construction of blind paths during road construction. In addition, with the rise of shared bicycle travel in my country in recent years, it is not uncommon for the most common and basic barrier-free facilities, such as blind paths, to be occupied by immovable infrastructure such as telephone poles and bus signs and movable obstacles such as shared bicycles and trash cans; this makes the visually impaired extremely insecure when traveling alone, and they are easily injured due to bumps and tripping over obstacles, infringing on their right to travel freely and enjoy life.

[0003] In today's society, the most commonly used tools for visually impaired people to guide and avoid obstacles are canes and guide dogs. The cane is the most common tool to assist the visually impaired in walking. It is an extension of the visually impaired's arm's sense of touch. It has the advantages of low cost and portability. It has a long history of use, but its shortcomings are also very obvious. The cane can only detect a cone-shaped area 1-2 meters in front of the user. Not only is the detection range small, but it also limits the walking speed of the visually impaired to a certain extent. In addition, the cane can only detect the presence or absence of obstacles, and cannot specifically identify what the obstacles are, their size, and their material. The information such as the quality of the guide dogs is of limited help to the visually impaired. Guiding by guide dogs is an ideal means of assisting the visually impaired in traveling. However, according to statistics, there are only about 200 legal guide dogs with relevant certificates in my country. It is a drop in the bucket to meet the travel needs of the huge visually impaired group. In addition, the average cost of training a guide dog is about 200,000 yuan, and the service life of a guide dog is 6-8 years. The necessary expenses of the guide dog owner on the guide dog are more than 1,000 yuan per month. Such a high cost is not affordable for the general group.

[0004] Blind guide obstacle avoidance systems based on ultrasonic ranging and SLAM (Simultaneous Localization and Mapping) algorithms have been developed and applied in the daily lives of visually impaired people; however, these two methods each have certain defects, which makes it impossible for them to provide satisfactory blind guide services for visually impaired people; Blind guide obstacle avoidance systems based on ultrasonic obstacle avoidance technology, such as the "Blind Guide Travel Intelligent Terminal (for Visually Impaired People)" with patent application number CN2023308030236 and "An Ultrasonic Blind Guide System" with patent number CN2021104305193, although simple to implement and low cost, their detection range radius is only a few meters more than that of traditional blind sticks, and the farther the distance, the lower the detection accuracy; the SLAM algorithm cannot perceive the specific type of obstacles in front of the visually impaired person, which makes the information that the visually impaired person can obtain from the blind guide obstacle avoidance system very limited. Summary of the invention

[0005] The purpose of the present invention is to provide a blind guide obstacle avoidance system based on improved YOLOv5s, which takes target detection algorithm as the core, and is supplemented by target tracking algorithm and independently designed danger intelligent analysis algorithm. It is carried on edge computing device, has simple operation, comprehensive functions, low cost and good portability, and can provide accurate and efficient blind guide obstacle avoidance services for visually impaired people.

[0006] To achieve the above object, the present invention provides the following technical solution: a blind guide obstacle avoidance system based on improved YOLOv5s, comprising:

[0007] An image acquisition module, which is responsible for acquiring the road condition information ahead of the visually impaired person, and automatically processes the video stream into a format accepted by subsequent modules, and transmits it to the target detection and tracking module;

[0008] The target detection and tracking module is responsible for processing the incoming video stream frame by frame, separating obstacles and guides from the complex sidewalk environment through the improved YOLOv5s target detection algorithm, and then using the Deep-Sort target tracking algorithm to track the detected targets, obtain the detection frame coordinates, category, and unique ID information of each target, and pass this information to the dangerous intelligent analysis module;

[0009] A hazard intelligent analysis module, which has a built-in hazard intelligent analysis algorithm, uses the inferred target information to perform hazard analysis and judge the conditions of the road ahead and obstacles; when the road conditions ahead of the visually impaired person change or the person is likely to bump into an obstacle, the hazard intelligent analysis module will promptly generate a voice broadcast sentence containing detailed information about the road ahead or obstacles and avoidance prompts, and transmit it to the voice broadcast module;

[0010] The voice broadcast module is responsible for sending the voice broadcast sentences generated by the hazard intelligent analysis module at an appropriate volume and speed to inform the visually impaired, enhance the visually impaired's perception of the surrounding road conditions, and assist the visually impaired to avoid obstacles in time and travel safely.

[0011] As a preferred technical solution of the present invention, it also includes using the TensorRT deep learning inference engine to perform half-precision compression on the improved YOLOv5s algorithm model.

[0012] As a preferred technical solution of the present invention, in the image acquisition module, a Sony IMX219 camera with a field of view of 77° is selected as the image acquisition module to obtain image input of the road conditions ahead when the visually impaired person is walking; in the tentative blind guide obstacle avoidance system, the camera is installed at a height of 1.1 meters.

[0013] As a preferred technical solution of the present invention, it also includes detection target classification and data set construction; the detection targets are classified into two major categories: obstacles and guide objects.

[0014] As a preferred technical solution of the present invention, the obstacle category is subdivided into three subcategories: dynamic obstacles, static obstacles and unstable obstacles.

[0015] As a preferred technical solution of the present invention, during the dataset construction process, the dataset has three sources: screening from the WOTR dataset; Internet collection; offline street scene real shooting.

[0016] As a preferred technical solution of the present invention, the target detection algorithm improvement includes introducing a Ghost convolution module, adding a CBAM attention mechanism, and replacing the PANet network with a weighted bidirectional feature pyramid network BiFPN.

[0017] As a preferred technical solution of the present invention, the improved YOLOv5s algorithm runs as follows:

[0018] The image frame to be processed is passed from the input end to the backbone network of the model. The backbone network consists of a CBS module, four GConv modules, four GC3 modules, a CBAM module and a spatial pyramid pooling layer, where

[0019] The CBS module is at the front end of the backbone network and is responsible for preheating the network;

[0020] The alternating GConv modules and GC3 modules are responsible for gradually extracting the feature maps of the input image, including shallow position feature maps and deep semantic feature maps;

[0021] The CBAM module is responsible for enhancing the extraction of spatial and semantic feature information of small objects;

[0022] The final spatial pyramid pooling layer of the backbone network is responsible for fusing feature maps of different scales and passing them into the neck of the model;

[0023] The neck of the model consists of four GConv modules, four GC3 modules, four BiFPN networks and two upsampling operations. The overall neck has a "bottom-up and then top-down" structure. Among them, the GConv module and the GC3 module are responsible for further feature extraction of the incoming input; the BiFPN network is responsible for feature fusion of the incoming features at each level, and the fused feature maps of different levels will be passed to the head of the model at different stages; the head of the model uses the incoming features to generate multiple target detection candidate boxes and confidence scores of the predicted categories in the original input image, and outputs this information.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] The blind guide obstacle avoidance method based on target detection technology can accurately inform the visually impaired of the type of blind guide or obstacle ahead, allowing the visually impaired to further understand the road conditions ahead;

[0026] Combining target detection and target tracking technology and applying it to the field of blind guide obstacle avoidance can help improve the effect of blind guide obstacle avoidance and ensure the accuracy of blind guide obstacle avoidance services;

[0027] Combining the software-level blind guide obstacle avoidance algorithm with edge computing devices has realized a truly practical blind guide obstacle avoidance system that is easy to operate, portable, and has high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is the overall structure diagram of the system of the present invention;

[0029] Figure 2 This is a diagram for selecting and classifying detection targets of the present invention;

[0030] Figure 3 A diagram is provided for the processing logic of the present invention for different types of targets. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1-Figure 3The present invention provides a blind guide obstacle avoidance system based on improved YOLOv5s, which includes an image acquisition module, a target detection and tracking module, a dangerous intelligent analysis module, and a voice broadcast module;

[0033] The image acquisition module is responsible for obtaining the road condition information in front of the visually impaired person, and automatically processes the video stream into a format accepted by the subsequent modules and transmits it to the target detection and tracking module;

[0034] The target detection and tracking module is responsible for processing the incoming video stream frame by frame, separating obstacles and guides from the complex sidewalk environment through the improved YOLOv5s target detection algorithm, and then using the Deep-Sort target tracking algorithm to track the detected targets, obtain the detection frame coordinates, category, unique ID and other information of each target, and pass this information to the hazard intelligent analysis module;

[0035] The intelligent hazard analysis module has our self-designed intelligent hazard analysis algorithm built in. It uses the target information inferred by the previous module to perform hazard analysis, mainly to judge the conditions of the road ahead and obstacles. When the road conditions ahead of the visually impaired person change or there is a possibility of hitting an obstacle next, the intelligent hazard analysis module will promptly generate a voice broadcast sentence containing detailed information about the road ahead or obstacles and avoidance prompts, and transmit it to the voice broadcast module.

[0036] The voice broadcast module is responsible for sending the voice broadcast sentences generated by the hazard intelligent analysis module at an appropriate volume and speed to inform the visually impaired, thereby enhancing the visually impaired's ability to perceive the surrounding road conditions and assisting the visually impaired to avoid obstacles in time and travel safely.

[0037] Classification of detection targets and construction of data sets:

[0038] Objects that are common on sidewalks and may affect the walking of visually impaired people, such as stray cats, warning posts, and bicycles, are included in the detection target sequence; in order to establish a systematic analysis and processing of these targets, the detection targets are classified into two major categories: obstacles and guide objects, among which the obstacle category is subdivided into three subcategories: dynamic obstacles, static obstacles, and unstable obstacles;

[0039] The definitions of each class are as follows:

[0040] Dynamic obstacles: Intelligent biological individuals will actively avoid visually impaired people;

[0041] Static obstacles: Due to road planning defects, various infrastructures are set up on and near the blind path, which the visually impaired need to actively avoid;

[0042] Unstable obstacles: mainly various motor vehicles, which are sometimes parked on the road and sometimes in motion under the control of the driver. Visually impaired people need to judge their status and make different actions;

[0043] Guides for the blind: various infrastructures that can assist the visually impaired in walking.

[0044] In the process of constructing the dataset, our dataset mainly comes from three sources: (a) screened from the WOTR dataset, (b) collected from the Internet, and (c) offline street scene shots; a total of 42,000 images were collected, and 115,000 detection frame samples of obstacles or guides including blind paths, zebra crossings, pedestrians, bicycles, tree trunks, trash cans, warning columns, etc. were manually annotated for the target detection algorithm to perform model training and evaluation.

[0045] Improvements to the object detection algorithm:

[0046] In order to take into account the real-time and accuracy of target detection, the YOLOv5s algorithm is selected as the target detection algorithm of this system; in order to balance the detection accuracy and detection speed of the model, YOLOv5s is selected as the target detection algorithm of this system, and the model is improved with the goal of "improving the detection speed without losing the detection accuracy as much as possible". The specific improvements are as follows;

[0047] Introducing the Ghost convolution module: In order to reduce the amount of model inference operations as much as possible without losing detection accuracy, all standard convolution modules except the first one in the model backbone network are replaced with the Ghost convolution module; the basic idea of ​​the Ghost convolution is to split the general convolution into three steps according to the connection between feature maps. First, a small amount of conventional convolution is performed to generate a feature map, and then the generated feature map is subjected to cheap group convolution to obtain the Ghost feature map. Finally, the feature maps obtained in the first and second steps are spliced ​​to generate the final feature map.

[0048] Add CBAM attention mechanism: To address the problem that the native YOLOv5s algorithm cannot identify small targets (such as manhole covers and fire hydrants far away from the camera) well, we added a convolutional block attention module (CBAM) at the end of the backbone network responsible for feature extraction in YOLOv5s. CBAM is a hybrid attention mechanism that combines the channel attention mechanism and the spatial attention mechanism. It first extracts global features channel by channel to generate a channel attention feature map, which is then used as the input of the spatial attention module to obtain a mixed domain feature map. CBAM can simultaneously consider the feature information of the spatial and channel dimensions, enhance the model's ability to focus on the target area, and enable the model to more accurately locate and identify small targets.

[0049] Use the weighted bidirectional feature pyramid network BiFPN to replace the PANet network: In order to improve the feature fusion capability of the model, the BiFPN network is used to replace the original PANet path aggregation network in the YOLOv5s model; compared with PANet, BiFPN deletes the nodes with only one input end and adds jump links in the middle layer. These improvements make BiFPN have more efficient feature fusion capabilities and lower computational costs compared to PANet;

[0050] The improved YOLOv5s algorithm runs as follows: the image frame to be processed is passed from the input end to the backbone network of the model. The backbone network consists of a CBS module, four GConv modules, four GC3 modules, a CBAM module and a spatial pyramid pooling layer. The CBS module is at the front end of the backbone network and is responsible for preheating the network. The alternating GConv modules and GC3 modules are responsible for gradually extracting the feature maps of the input image. The main network consists of four GConv modules, four GC3 modules, four BiFPN networks and two upsampling operations. The neck of the model is composed of four GConv modules, four GC3 modules, four BiFPN networks and two upsampling operations. The overall structure of the neck is "bottom-up and then top-down". The GConv module and the GC3 module are responsible for further feature extraction of the incoming input. The upsampling operation is to enlarge the size of the input feature map. The BiFPN network is responsible for feature fusion of the incoming features at all levels. The fused feature maps of different levels will be passed to the head of the model at different stages. Here, the head of the model uses the incoming features to generate multiple target detection candidate boxes and confidence scores of the predicted categories in the original input image, and outputs this information.

[0051] Design of intelligent analysis algorithm for obstacle hazard:

[0052] In order to further reflect the different times that different types of obstacles require advance warning, the images obtained by the camera are manually divided into red and yellow warning areas according to the distance from the visually impaired person; for obstacles, the areas where the system triggers the alarm are different according to the different danger levels of static obstacles, dynamic obstacles, and unstable obstacles;

[0053] Different processing logics are set for different types of targets. For each target passed into the intelligent risk analysis module, the system will first determine which category it belongs to based on the target category, and then determine whether the target will cause harm to the visually impaired person and whether the harm is serious based on the target's direction, distance from the visually impaired person, and speed, and generate corresponding voice broadcast sentences. Guide objects for the blind are broadcast once at fixed intervals, while obstacles are broadcast only once for each ID. The specific processing methods of this system for obstacles and guide objects for the blind are as follows:

[0054] Handling of dynamic obstacles: If the target is a type of dynamic obstacle, considering that the target categories in this category are all intelligent creatures, they will actively avoid the visually impaired and will not cause too much impact on the normal walking of the visually impaired. Therefore, for dynamic obstacles, we choose to let the system not handle them;

[0055] Handling of static obstacles: For static obstacles, since they do not move, the possible harm to the visually impaired is relatively small. When a static obstacle enters the red alert zone, the system is set to generate a voice broadcast sentence to remind the visually impaired to detour to the left or right according to the position of the static obstacle relative to the visually impaired. At this time, the obstacle is close to the visually impaired, but the visually impaired still has enough time to detour to avoid the obstacle. If the system prompts the visually impaired to detour when the obstacle is far away from the visually impaired, it is not conducive to the visually impaired to control the detour distance, and the visually impaired may still not detour the obstacle after detouring and returning to the original walking route;

[0056] Processing of unsteady obstacles: For unsteady obstacles, the system first uses the coordinate displacement of the obstacle detection frame in multiple consecutive frames to determine whether the motion state of the unsteady obstacle is steady or dynamic; if the target is static, it is treated as a static obstacle; if the obstacle is dynamic, the obstacle at this time (such as an oncoming electric bicycle on the sidewalk or a private car parked on the sidewalk) is usually highly dangerous. Once the visually impaired collide with these obstacles, they will suffer extremely serious injuries; therefore, when the system detects that a moving unsteady obstacle enters the yellow zone, it generates a voice broadcast sentence to prompt the visually impaired to stop and give way immediately;

[0057] Handling of guide objects for the blind: Unlike obstacles, guide objects for the blind will appear in the camera for a long time. If the voice broadcast frequency of the guide objects is set to once for each ID, the system will broadcast the voice too frequently, causing the visually impaired to continue to be distracted and focus on the voice broadcast content, reducing the visually impaired's attention to other information, which may easily cause accidents; and if the voice broadcast thread is occupied for a long time by the sentences broadcasting the information related to the guide objects for the blind, and there is no timely voice broadcast for the suddenly appearing obstacles, the system will not be able to alarm the visually impaired before the visually impaired collides with the obstacles; therefore, it is chosen to generate corresponding voice broadcast sentences for the guide objects that appear in the camera according to their positions at fixed time intervals.

[0058] In addition, since different guides have different effects on the visually impaired, the system needs to be set up to handle different guides differently and generate different voice broadcast sentences. Normally, railings are used to separate roads and sidewalks. If the system detects railings on both sides, it means that the visually impaired may be too close to the roadside. If the system detects railings in front, there may be a turn in the road ahead. When the system detects a blind path, in order to ensure that the walking direction of the visually impaired is consistent with the direction of the blind path, it is necessary to always detect the direction of the blind path in front relative to the visually impaired, but only generate corresponding voice broadcast sentences when the visually impaired deviates or the blind path ends. The processing of zebra crossings is relatively simple. When the zebra crossing appears in the red warning zone, the intelligent hazard analysis module generates a voice broadcast sentence to remind the visually impaired that there is an intersection ahead. When stairs are detected to appear directly in front of the camera, the system will generate a voice broadcast sentence to remind the visually impaired "There are stairs directly ahead, please walk carefully and avoid stepping on empty air."

[0059] Physical construction of the blind guide obstacle avoidance system:

[0060] The image acquisition module is responsible for recording the road conditions ahead when the visually impaired person is walking, and transmitting the video stream to the system for subsequent processing. After several rounds of performance and price comparison and verification, this system uses the Sony IMX219 camera with a field of view of 77° as the image acquisition module to obtain image input of the road conditions ahead when the visually impaired person is walking; in the tentative blind guide obstacle avoidance system, the camera is set up at a position of 1.1 meters. According to the sine theorem, the detection blind area of ​​the camera is about 1.3 meters in front of the visually impaired person, which is close to the feeling of normal people walking.

[0061] The task of the target detection and tracking module is to receive the video stream collected by the image acquisition module of the road conditions ahead when the visually impaired person is walking, run the YOLOv5s target detection algorithm to perform target detection on the incoming video stream frame by frame, and pass the detection structure into the Deep-Sort target tracking algorithm, calibrate the detection frame for all detected targets, and obtain the unique ID identification, category, detection frame coordinates and other data of each target detection frame.

[0062] In the intelligent risk analysis module, the system uses the categories, unique IDs, detection frame coordinates and other data of all detected targets obtained by the target detection and tracking module as input, performs a risk analysis on the targets, and generates corresponding voice broadcast sentences.

[0063] The voice broadcast module is the part where the system interacts directly with the visually impaired. It is responsible for broadcasting the voice broadcast sentences generated by the hazard intelligent analysis module to inform the visually impaired of the road conditions ahead. A relatively cheap and mainstream speaker on the market is selected as the voice broadcast module of this system, which is connected to the Jetson Nano motherboard through a DuPont cable. After field testing, when the system is deployed at the waist of the visually impaired, the maximum volume of the YB-MVR01 voice broadcast module is sufficient for the visually impaired to hear the voice content of the system in a slightly noisy street environment.

[0064] In addition, this system uses the TensorRT deep learning inference engine to perform half-precision compression on the improved YOLOv5s algorithm model, thereby improving the inference speed of the algorithm model and reducing energy consumption with almost no loss of accuracy. Finally, all the software and hardware parts of the system are integrated into the Jetson Nano B01 edge computing device as the system physical terminal.

[0065] Although the embodiments of the present invention have been shown and described, as detailed above, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A blind guide obstacle avoidance system based on improved YOLOv5s, characterized by: include An image acquisition module, which is responsible for acquiring the road condition information ahead of the visually impaired person, and automatically processes the video stream into a format accepted by subsequent modules and transmits it to the target detection and tracking module; The target detection and tracking module is responsible for processing the incoming video stream frame by frame, separating obstacles and guides from the complex sidewalk environment through the improved YOLOv5s target detection algorithm, and then using the Deep-Sort target tracking algorithm to track the detected targets, obtain the detection frame coordinates, category, and unique ID information of each target, and pass this information to the dangerous intelligent analysis module; A hazard intelligent analysis module, which has a built-in hazard intelligent analysis algorithm, uses the inferred target information to perform hazard analysis and judge the conditions of the road ahead and obstacles; when the road conditions ahead of the visually impaired person change or the person is likely to bump into an obstacle, the hazard intelligent analysis module will promptly generate a voice broadcast sentence containing detailed information about the road ahead or obstacles and avoidance prompts, and transmit it to the voice broadcast module; The voice broadcast module is responsible for sending the voice broadcast sentences generated by the hazard intelligent analysis module at an appropriate volume and speed to inform the visually impaired, enhance the visually impaired's perception of the surrounding road conditions, and assist the visually impaired to avoid obstacles in time and travel safely.

2. A blind-guiding obstacle avoidance system based on improved YOLOv5s according to claim 1, characterized in that: It also includes half-precision compression of the improved YOLOv5s algorithm model using the TensorRT deep learning inference engine.

3. A blind-guiding obstacle avoidance system based on improved YOLOv5s according to claim 1, characterized in that: In the image acquisition module, a Sony IMX219 camera with a field of view of 77° is selected as the image acquisition module to obtain image input of the road conditions ahead when the visually impaired person is walking; in the tentative blind guide obstacle avoidance system, the camera is installed at a height of 1.1 meters.

4. A blind-guiding obstacle avoidance system based on improved YOLOv5s according to claim 1, characterized in that: It also includes detection target classification and data set construction; detection targets are classified into two major categories: obstacles and guide objects.

5. A blind-guiding obstacle avoidance system based on improved YOLOv5s according to claim 4, characterized in that: The major category of obstacles is subdivided into three subcategories: dynamic obstacles, static obstacles and unsteady-state obstacles.

6. A blind-guiding obstacle avoidance system based on improved YOLOv5s according to claim 4, characterized in that: During the dataset construction process, the dataset has three sources: screening from the WOTR dataset; Internet collection; offline street scene shooting.

7. The blind-guiding obstacle avoidance system based on improved YOLOv5s according to claim 1, characterized in that: Improvements to the target detection algorithm include the introduction of the Ghost convolution module, the addition of the CBAM attention mechanism, and the use of the weighted bidirectional feature pyramid network BiFPN to replace the PANet network.

8. The blind-guiding obstacle avoidance system based on improved YOLOv5s according to claim 1, characterized in that: The improved YOLOv5s algorithm runs as follows: The image frame to be processed is passed from the input end to the backbone network of the model. The backbone network consists of a CBS module, four GConv modules, four GC3 modules, a CBAM module and a spatial pyramid pooling layer, where The CBS module is at the front end of the backbone network and is responsible for preheating the network; The alternating GConv modules and GC3 modules are responsible for gradually extracting the feature maps of the input image, including shallow position feature maps and deep semantic feature maps; The CBAM module is responsible for enhancing the extraction of spatial and semantic feature information of small objects; The final spatial pyramid pooling layer of the backbone network is responsible for fusing feature maps of different scales and passing them into the neck of the model; The neck of the model consists of four GConv modules, four GC3 modules, four BiFPN networks and two upsampling operations. The overall neck has a "bottom-up and then top-down" structure. Among them, the GConv module and the GC3 module are responsible for further feature extraction of the incoming input; the BiFPN network is responsible for feature fusion of the incoming features at each level, and the fused feature maps of different levels will be passed to the head of the model at different stages; the head of the model uses the incoming features to generate multiple target detection candidate boxes and confidence scores of the predicted categories in the original input image, and outputs this information.

Images