Model training method and multi-scale vehicle and pedestrian detection method
By replacing the Bottleneck structure as a partial convolution module in the YOLOv8 model, and introducing a lightweight cross-scale feature fusion module and heavy parameter detection head, the problem of large parameters and calculations in the deployment of the target detection model in the embedded platform is solved, and the problem of large amount of parameters and calculations is improved. The accuracy and efficiency of vehicle and pedestrian detection.
Patent Information
- Application Number
- CN202411150660.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-13
AI Technical Summary
The existing target detection model has a huge amount of parameters and calculations, which is not easy to deploy in embedded, resulting in insufficient target detection accuracy in traffic scenarios, limiting the detection effect of the model after deployment on the embedded platform.
By replacing the Bottleneck structure in the backbone network of the YOLOv8 model as a partial convolution module, a lightweight cross-scale feature fusion module CCFM and a re-parameter detection head are introduced to reduce the parameter amount and calculation amount of the model, while enhancing the adaptability of the model to scale changes. and detection capabilities of small-scale objects.
On the basis of ensuring detection accuracy, the number of model parameters and calculations is significantly reduced, the detection effect of the model after deployment of the embedded platform is improved, and the performance of vehicle and pedestrian detection is improved.
Smart Images
Figure CN119131747A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of target detection technology, and in particular to a model training method and a multi-scale vehicle and pedestrian detection method. Background Art
[0002] As a new high-tech complex in recent years, smart cars have made great improvements in vehicle safety. The car's automatic driving system can reduce traffic accidents caused by fatigue driving. Relying on the vehicle's environmental perception system, the automatic driving system can quickly make safer and more accurate decisions in emergency situations. Object detection in the automatic driving system detects vehicles and pedestrians to accurately perform tasks such as automatic obstacle avoidance and vehicle cruising.
[0003] The current target detection method mainly uses radar sensors in combination with cameras, uses lidar to measure distance, and uses target detection models to identify target categories in real time in the picture, and presents each category of target on the screen in the form of a prediction box. The real-time image provides a reference for the driver and also provides a basis for the decision-making of the autonomous driving system.
[0004] However, the above-mentioned target detection model has a huge number of parameters and a large amount of calculation, which is not easy to be deployed in an embedded system. As the number of parameters and the amount of calculation of the model decreases, the model accuracy will decrease, resulting in insufficient target detection accuracy in traffic scenarios, which limits the detection effect of the model after deployment on the embedded platform. Summary of the invention
[0005] In view of this, the present application provides a model training method and a multi-scale vehicle and pedestrian detection method, which can reduce the number of model parameters and calculation amount on the basis of ensuring the detection accuracy of vehicles and pedestrians, and improve the detection effect of the model after being deployed on an embedded platform.
[0006] Specifically, the following technical solutions are included:
[0007] In a first aspect, an embodiment of the present application provides a model training method, the method comprising:
[0008] A sample vehicle and pedestrian training set, a sample vehicle and pedestrian annotated training set, and a YOLOv8 improved model are obtained. The improved method of the YOLOv8 improved model includes replacing a traditional convolution module of a Bottleneck structure in a C2f module of a backbone network of the YOLOv8 model with a partial convolution module, introducing a lightweight cross-scale feature fusion module CCFM, and introducing a heavy parameter detection head;
[0009] Based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian labeled training set, the YOLOv8 improved model is trained to obtain a target vehicle and pedestrian detection model.
[0010] In some embodiments, the training of the YOLOv8 improved model based on the sample vehicle pedestrian training set and the sample vehicle pedestrian annotated training set to obtain the target vehicle pedestrian detection model includes:
[0011] Based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian labeled training set, training a YOLOv8 improved model to obtain an intermediate YOLOv8 improved model;
[0012] In response to the performance of the intermediate YOLOv8 improved model satisfying the training completion condition, the intermediate YOLOv8 improved model satisfying the training completion condition is used as the target vehicle pedestrian detection model.
[0013] In some embodiments, the method for improving the YOLOv8 improved model specifically includes:
[0014] The first convolution module of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model is replaced with a partial convolution module with a partial ratio of one quarter, and the second convolution module of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model is replaced with a partial convolution module with a partial ratio of one third, to obtain a first improved model;
[0015] In the neck network of the first improved model, a lightweight cross-scale feature fusion module CCFM is introduced to obtain the second improved model;
[0016] Based on the second improved model, the detection head is improved by the re-parameter module to obtain the YOLOv8 improved model.
[0017] In some embodiments, before obtaining the second improved model, the method further includes changing the number of channels of the input feature map of feature fusion, and modifying the number of channels of the three extracted feature maps to 128.
[0018] In some embodiments, based on the second improved model, the detection head is improved by a re-parameter module to obtain a YOLOv8 improved model, which specifically includes:
[0019] The ConvModule modules of the first two detection branches in the second improved model are replaced by the reparameterized module RepConv.
[0020] In some embodiments, the training completion condition is that the mAP of the intermediate YOLOv8 improved model does not increase for 50 consecutive rounds.
[0021] In some embodiments, obtaining a sample vehicle and pedestrian training set includes:
[0022] A KITTI dataset is obtained, and the KITTI dataset is divided into a sample vehicle and pedestrian training set, a sample vehicle and pedestrian verification set, and a sample vehicle and pedestrian test set in a ratio of 7:2:1.
[0023] In some embodiments, obtaining a sample vehicle and pedestrian labeled training set includes:
[0024] The sample vehicle-pedestrian training set, the sample vehicle-pedestrian verification set and the sample vehicle-pedestrian test set are labeled to obtain the sample vehicle-pedestrian labeled training set, the sample vehicle-pedestrian labeled verification set and the sample vehicle-pedestrian labeled test set, and the format of the label file is converted into a txt label file that meets the requirements of the YOLOv8 improved model. The labeled label categories include cars, trucks, people and cyclists.
[0025] In some embodiments, the method further comprises:
[0026] In response to the performance of the intermediate YOLOv8 improved model not meeting the training completion condition, adjusting the training weights and training parameters of the intermediate YOLOv8 improved model until the performance of the intermediate YOLOv8 improved model meets the training completion condition, the training parameters including an initial learning rate, a number of training iterations, and a number of samples in each training batch.
[0027] In a second aspect, the present application provides a multi-scale vehicle and pedestrian detection method, the method comprising:
[0028] Obtaining a target vehicle and pedestrian detection model in the model training method as described in the first aspect;
[0029] The vehicle and pedestrian training set to be tested is input into the target vehicle and pedestrian detection model to obtain a vehicle and pedestrian detection result, wherein the vehicle and pedestrian detection result includes a target category, a target bounding box position and a confidence level, and the target categories include cars, trucks, people and cyclists.
[0030] The beneficial effects of the technical solution provided by the embodiments of the present application include at least:
[0031] The embodiment of the present application provides a model training method and a multi-scale vehicle and pedestrian detection method. The method first obtains a sample vehicle and pedestrian training set, a sample vehicle and pedestrian annotation training set and a YOLOv8 improved model. The improved method of the YOLOv8 improved model includes replacing the traditional convolution module of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model with a partial convolution module, introducing a lightweight cross-scale feature fusion module CCFM, and introducing a heavy parameter detection head. The partial convolution module can reduce the number of floating-point operations, reduce redundant calculations and memory access, and more effectively extract spatial features. The lightweight cross-scale feature fusion module CCFM can reduce the number of feature map channels, and integrate features of different scales through fusion operations to enhance the model's adaptability to scale changes and the ability to detect small-scale objects. The lightweight of the model is effectively achieved under the condition of very little precision loss. The heavy parameter module can increase the ability to learn features and reduce the amount of parameters and calculations. Then, based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian annotation training set, the YOLOv8 improved model is trained to obtain a lightweight target vehicle and pedestrian detection model. This method can reduce the number of model parameters and calculations while ensuring the detection accuracy of vehicles and pedestrians, and improve the detection effect of the model after it is deployed on an embedded platform. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0033] Figure 1 A method flow chart of the model training method provided in an embodiment of the present application;
[0034] Figure 2 A schematic diagram of the structure of the YOLOv8 improved model in the model training method provided in the embodiment of the present application;
[0035] Figure 3 A schematic diagram of the structure of the Bottleneck structure in the YOLOv8 improved model of the model training method provided in an embodiment of the present application;
[0036] Figure 4 A schematic diagram of the structure of a detection head improved by a heavy parameter module in the YOLOv8 improved model of the model training method provided in an embodiment of the present application;
[0037] Figure 5 A schematic diagram of the structure of the re-parameterization module in the YOLOv8 improved model of the model training method provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0039] In order to make the technical solutions and advantages of the present application more clear, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0040] In the first aspect, the present application embodiment provides a model training method, see Figure 1 , the method comprising:
[0041] Step 101, obtaining a sample vehicle-pedestrian training set, a sample vehicle-pedestrian annotated training set, and a YOLOv8 improved model. The improvement method of the YOLOv8 improved model includes replacing a traditional convolution module of a Bottleneck structure in a C2f module of a backbone network of the YOLOv8 model with a partial convolution module, introducing a lightweight cross-scale feature fusion module CCFM, and introducing a heavy parameter detection head.
[0042] The sample vehicle and pedestrian training set and the sample vehicle and pedestrian annotated training set are the basis for YOLOv8 improved model training. Some convolution modules can reduce the number of floating-point operations, reduce redundant calculations and memory access, and extract spatial features more effectively. The lightweight cross-scale feature fusion module CCFM can reduce the number of feature map channels and integrate features of different scales through fusion operations to enhance the model's adaptability to scale changes and its ability to detect small-scale objects. It effectively realizes the lightweight of the model with little loss of accuracy. The heavy parameter module can increase the ability to learn features and reduce the amount of parameters and calculations.
[0043] In some embodiments, the improved method of the YOLOv8 improved model, such as Figure 2 As shown, specifically including:
[0044] (1) A partial convolution module with a partial ratio of one quarter is used to replace the first convolution module of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model, and a partial convolution module with a partial ratio of one third is used to replace the second convolution module of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model, to obtain the first improved model.
[0045] Using two partial convolution modules ( Figure 2The PConv in the YOLOv8 model (hereinafter referred to as PConv) replaces the first and second convolution modules of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model. The partial convolution module applies the traditional convolution module to only a part of the input channel for spatial feature extraction, while keeping the remaining channels unchanged. If the input and output feature map sizes and number of channels are the same, its floating point calculation times FLOPs are:
[0046]
[0047] Among them, h and w are the height and width of the input feature map, k is the size of the convolution kernel, and c p is the number of channels involved in the calculation in the partial convolution module, c is the number of channels in the traditional convolution module, when the partial ratio is one quarter, that is, c is c p When the FLOPs of the partial convolution module is four times that of the traditional convolution module, it is about one sixteenth of that of the traditional convolution module. Therefore, it can reduce the computational complexity, reduce memory access, and maintain the effect of feature information flow.
[0048] (2) In the neck network of the first improved model, a lightweight cross-scale feature fusion module CCFM is introduced to obtain the second improved model.
[0049] In some embodiments, before obtaining the second improved model, the method further includes changing the number of channels of the input feature map of feature fusion, and modifying the number of channels of the three extracted feature maps to 128.
[0050] For an input image with an input size of 640×640 pixels, the feature fusion part of the YOLOv8 model (i.e., the first improved model) extracts three feature maps of different scales from the backbone network: 80×80, 40×40, and 20×20. The number of channels corresponding to these feature maps are 128, 256, and 512, respectively. The 80×80 feature map captures more detailed feature information, which comes from the feature map obtained after several layers of convolution and downsampling of the input image. It has 128 channels and is mainly used to detect smaller targets; the 40×40 feature map captures medium-detailed information and has 256 channels. It is the result of further downsampling and combines the contextual information of a larger area, which helps detect medium-sized targets; the 20×20 feature map captures global information and has 512 channels. It is obtained after multiple downsamplings. It contains the global semantic information of the image and is suitable for detecting larger targets.
[0051] like Figure 2 and Figure 3As shown in the figure, the improved YOLOv8 model extracts three feature maps of 80×80, 40×40, and 20×20 from the backbone network. All of them first pass through a convolution module consisting of a 1×1 convolution layer with a convolution kernel of 128, a BN layer, and a SiLU activation function, so that the number of channels of the three extracted feature maps becomes 128, unifying the number of channels to facilitate subsequent splicing or addition operations. At the same time, the number of channels of the feature maps of size 40×40 and 20×20 is reduced to half and one-third of the original, which greatly reduces the number of parameters and calculations. A 1×1 convolution module with a convolution kernel of 128 is added after the first C2f module of the backbone network. Although the number of channels of the input and output is the same, the weight of each channel can be redistributed through the 1×1 convolution layer, thereby highlighting more useful features and suppressing irrelevant or redundant features, which helps to improve the quality of feature representation and enhance the discrimination ability of the subsequent detection layer. The BN layer enables the model to stabilize the training process, accelerate convergence, improve generalization ability, and reduce sensitivity to data preprocessing and learning rate. The use of SiLU activation function increases the nonlinear expression ability of the model and improves the complexity and discriminability of features.
[0052] Changing the number of channels of the output feature map of feature fusion, the feature map output by the feature fusion part of the YOLOv8 model has the same size as the input feature map, which are 80×80, 40×40 and 20×20 respectively. The number of channels corresponding to these three feature maps are 128, 256 and 512 respectively. The size of the feature fusion output feature map of the improved YOLOv8 model remains unchanged, and the number of channels is changed to 128, thereby integrating features of different scales through fusion operations to enhance the model's adaptability to scale changes and its ability to detect small-scale objects, effectively achieving the lightweight of the model with little loss of accuracy.
[0053] (3) Based on the second improved model, the detection head is improved by the re-parameter module to obtain the YOLOv8 improved model.
[0054] In some embodiments, Figure 2 and Figure 4 As shown in FIG. 1 , based on the second improved model, the detection head is improved by the re-parameter module to obtain the YOLOv8 improved model, which specifically includes:
[0055] The ConvModule modules of the first two detection branches in the second improved model are replaced by the reparameterized module (RepConv module for short).
[0056] In some embodiments, Figure 4 and Figure 5As shown in the figure, each feature map of the input detection head passes through two branches, and each branch first passes through two convolution modules. The improved detection head, namely the re-parameterized detection head, is first a branch, which passes through two re-parameterized modules consisting of 1×1 convolution, 3×3 convolution, BN layer and activation function, and then becomes two branches, which serve as the input of the convolution layer of each branch respectively, that is, the four traditional convolution modules of the two branches are converted into two re-parameterized modules of one branch.
[0057] In some embodiments, Figure 4 and Figure 5 As shown in the figure, during the training process, the reparameterization module uses a multi-branch structure, such as a combination of convolutional layers with different convolution kernel sizes, which enables the model to better learn complex feature representations. These branches can include, for example, 3×3 convolutional layers, 1×1 convolutional layers, and other auxiliary branches, which can enhance the network's expressive power and perform better than the original two 3×3 convolutional branches. In the inference phase, mathematical methods are used to merge the parameters of the multi-branch structure into the parameters of a single convolutional layer. The calculation formula for the convolutional layer is as follows:
[0058] Conv(x)=W(x)+b
[0059] Among them, W is the convolution kernel, also called the filter or weight matrix, x is the input feature map, and b is a constant value added to the result of the convolution operation, called the bias term.
[0060] In some embodiments, the calculation formula of the BN layer is as follows:
[0061]
[0062] in, γ and β are parameters that need to be learned during training.
[0063] In some embodiments, the convolutional layer and the BN layer are combined and processed according to the following formula:
[0064] Rep(x)=W BN *(W conv *x+b conv )+b BN
[0065] =(W BN *W conv )*x+(W BN *b conv +b BN )
[0066] The merging process generates an equivalent single-branch convolution layer by superimposing the convolution kernels and bias terms of different branches, where W Rep =W BN *Wconv , b Rep =W BN *b conv +b BN The combined single convolutional layer is used in the inference phase, which greatly reduces the computation and storage requirements compared to the multi-branch structure in the training phase. This simplification makes the model more efficient in the inference phase, with faster inference speed and lower computational overhead.
[0067] Step 102: Based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian annotated training set, train the YOLOv8 improved model to obtain a target vehicle and pedestrian detection model.
[0068] Then, based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian annotated training set, the YOLOv8 improved model is trained to obtain a lightweight target vehicle and pedestrian detection model.
[0069] In some embodiments, step 102 may specifically include:
[0070] Step 1021, based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian labeled training set, train the YOLOv8 improved model to obtain the intermediate YOLOv8 improved model.
[0071] Step 1022, in response to the performance of the intermediate YOLOv8 improved model satisfying the training completion condition, the intermediate YOLOv8 improved model satisfying the training completion condition is used as the target vehicle pedestrian detection model.
[0072] In some embodiments, the method further comprises:
[0073] In response to the performance of the intermediate YOLOv8 improved model not meeting the training completion condition, adjusting the training weights and training parameters of the intermediate YOLOv8 improved model until the performance of the intermediate YOLOv8 improved model meets the training completion condition, the training parameters including the initial learning rate, the number of training iterations, and the number of samples in each training batch.
[0074] In some embodiments, the training completion condition is that the mAP of the intermediate YOLOv8 improved model does not increase for 50 consecutive rounds.
[0075] In some embodiments, obtaining a sample vehicle and pedestrian training set includes:
[0076] Obtain the KITTI dataset and divide it into a sample vehicle and pedestrian training set, a sample vehicle and pedestrian verification set, and a sample vehicle and pedestrian test set in a ratio of 7:2:1.
[0077] In some embodiments, each image in the KITTI dataset contains up to 15 vehicles and up to 30 pedestrians.
[0078] In some embodiments, obtaining a sample vehicle and pedestrian labeling training set includes:
[0079] The sample vehicle-pedestrian training set, the sample vehicle-pedestrian verification set and the sample vehicle-pedestrian test set are labeled to obtain the sample vehicle-pedestrian labeled training set, the sample vehicle-pedestrian labeled verification set and the sample vehicle-pedestrian labeled test set, and the format of the label file is converted into a txt label file that meets the requirements of the YOLOv8 improved model. The labeled label categories include cars, trucks, people and cyclists.
[0080] In some embodiments, the training environment can be: Cuda11.8, Pytorch2.0.1 and Python3.9 are built on a server with an RTX 3090 GPU, an Intel(R) Xeon(R) Platinum 8255C CPU, 24GB of video memory, and 43GB of memory.
[0081] In some embodiments, the method further includes: inputting the verification set into the deep learning model of vehicle and pedestrian image recognition detection for verification, optimizing the deep learning model based on the verification results, and ultimately obtaining the deep learning model of vehicle and pedestrian image recognition detection with the best effect.
[0082] In some embodiments, a sample vehicle-pedestrian verification set and a sample vehicle-pedestrian annotated verification set are used to verify the target vehicle-pedestrian detection model.
[0083] Through ablation experiments, the performance indicators of the target vehicle and pedestrian detection model (A+B+C model in Table 1) and the YOLOv8 model (YOLOv8s in Table 1) obtained by the model training method provided in the embodiment of the present application are compared. The A model in Table 1 (i.e., the first improved model in the present application) is a model in which a traditional convolution module with a Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model is replaced with a partial convolution module on the basis of the YOLOv8 model. The A+B model (i.e., the second improved model in the present application) is a model in which a traditional convolution module with a Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model is replaced with a partial convolution module on the basis of the YOLOv8 model, and a lightweight cross-scale feature fusion module CCFM is introduced. The comparison results of the models are shown in Table 1. The mAP of the first improved model increased by 0.4%, Params decreased by 15.3%, GFLOPs decreased by 18.0%, and Weight decreased by 3.5MB; the mAP of the second improved model decreased by 0.5% compared with the YOLOv8 model, Params decreased by 50.5%, GFLOPs decreased by 37.3%, and Weight decreased by 11.3MB; the mAP of the target vehicle and pedestrian detection model (A+B+C model in Table 1) increased by 0.1%, Params decreased by 52.2%, GFLOPs decreased by 41.5%, and Weight decreased by 11.7MB, which significantly improved the performance of the vehicle and pedestrian detection task and made the model easy to deploy on embedded platforms.
[0084] It should be noted that mAP means average precision, Params means the number of model parameters, GFLOPs means the number of floating-point operations, and Weights means volume.
[0085] Table 1 Performance comparison of the target vehicle and pedestrian detection model and the YOLOv8 model
[0086]
[0087] In summary, the model training method provided in the embodiment of the present application can reduce the number of model parameters and calculation amount while ensuring the detection accuracy of vehicles and pedestrians, thereby improving the detection effect of the model after deployment on the embedded platform.
[0088] In a second aspect, the present application provides a multi-scale vehicle and pedestrian detection method, the method comprising:
[0089] Obtaining a target vehicle and pedestrian detection model in the model training method of the first aspect;
[0090] The vehicle and pedestrian training set to be tested is input into the target vehicle and pedestrian detection model to obtain the vehicle and pedestrian detection results. The vehicle and pedestrian detection results include target categories, target bounding box positions and confidence levels. The target categories include cars, trucks, people and cyclists.
[0091] In summary, the multi-scale vehicle and pedestrian detection method provided in the embodiment of the present application can reduce the number of model parameters and the amount of calculation on the basis of ensuring the detection accuracy of vehicles and pedestrians, and improve the detection effect of the model after being deployed on the embedded platform.
[0092] In the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more than two, unless otherwise clearly defined.
[0093] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the present application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only.
[0094] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A model training method, characterized in that: The method comprises: A sample vehicle and pedestrian training set, a sample vehicle and pedestrian annotated training set, and a YOLOv8 improved model are obtained. The improved method of the YOLOv8 improved model includes replacing a traditional convolution module of a Bottleneck structure in a C2f module of a backbone network of the YOLOv8 model with a partial convolution module, introducing a lightweight cross-scale feature fusion module CCFM, and introducing a heavy parameter detection head; Based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian labeled training set, the YOLOv8 improved model is trained to obtain a target vehicle and pedestrian detection model.
2. The model training method according to claim 1, characterized in that: The step of training the YOLOv8 improved model based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian labeled training set to obtain a target vehicle and pedestrian detection model includes: Based on the sample vehicle and pedestrian training set and the sample vehicle and pedestrian labeled training set, training a YOLOv8 improved model to obtain an intermediate YOLOv8 improved model; In response to the performance of the intermediate YOLOv8 improved model satisfying the training completion condition, the intermediate YOLOv8 improved model satisfying the training completion condition is used as the target vehicle pedestrian detection model.
3. The model training method according to claim 1, characterized in that: The improvement methods of YOLOv8 improved model include: The first convolution module of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model is replaced with a partial convolution module with a partial ratio of one quarter, and the second convolution module of the Bottleneck structure in the C2f module of the backbone network of the YOLOv8 model is replaced with a partial convolution module with a partial ratio of one third, to obtain a first improved model; In the neck network of the first improved model, a lightweight cross-scale feature fusion module CCFM is introduced to obtain the second improved model; Based on the second improved model, the detection head is improved by the re-parameter module to obtain the YOLOv8 improved model.
4. The model training method according to claim 3, characterized in that: Before obtaining the second improved model, the number of channels of the input feature map of feature fusion is also changed, and the number of channels of the three extracted feature maps is modified to 128.
5. The model training method according to claim 3, characterized in that: On the basis of the second improved model, the detection head is improved by the re-parameter module to obtain the YOLOv8 improved model, which specifically includes: The ConvModule modules of the first two detection branches in the second improved model are replaced by the reparameterized module RepConv.
6. The model training method according to claim 2, characterized in that: The training completion condition is that the mAP of the intermediate YOLOv8 improved model does not increase for 50 consecutive rounds.
7. The model training method according to claim 1, characterized in that: The step of obtaining a sample vehicle and pedestrian training set includes: A KITTI dataset is obtained, and the KITTI dataset is divided into a sample vehicle and pedestrian training set, a sample vehicle and pedestrian verification set, and a sample vehicle and pedestrian test set in a ratio of 7:2:
1.
8. The model training method according to claim 7, characterized in that: The step of obtaining a sample vehicle and pedestrian labeling training set includes: The sample vehicle-pedestrian training set, the sample vehicle-pedestrian verification set and the sample vehicle-pedestrian test set are labeled to obtain the sample vehicle-pedestrian labeled training set, the sample vehicle-pedestrian labeled verification set and the sample vehicle-pedestrian labeled test set, and the format of the label file is converted into a txt label file that meets the requirements of the YOLOv8 improved model. The labeled label categories include cars, trucks, people and cyclists.
9. The model training method according to claim 2, characterized in that: The method further comprises: In response to the performance of the intermediate YOLOv8 improved model not meeting the training completion condition, adjusting the training weights and training parameters of the intermediate YOLOv8 improved model until the performance of the intermediate YOLOv8 improved model meets the training completion condition, the training parameters including an initial learning rate, a number of training iterations, and a number of samples in each training batch.
10. A multi-scale vehicle and pedestrian detection method, characterized in that: The method comprises: Obtaining a target vehicle and pedestrian detection model in the model training method as described in any one of claims 1 to 9; The vehicle and pedestrian training set to be tested is input into the target vehicle and pedestrian detection model to obtain a vehicle and pedestrian detection result, wherein the vehicle and pedestrian detection result includes a target category, a target bounding box position and a confidence level, and the target categories include cars, trucks, people and cyclists.