Pedestrian target detection method based on multi-scale multi-feature neural network

Through the multi-scale multi-feature neural network method, feature maps of different scales are extracted and fused and refined, the problems of low pedestrian detection accuracy and background occlusion in the prior art are solved, and the accuracy and applicability of pedestrian detection are improved.

CN118942114AInactive Publication Date: 2024-11-12SMART WORKSHOP ENTERPRISE MANAGEMENT (WUXI) CO LTD

Patent Information

Application Number
CN202410996723.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing pedestrian detection technology has low detection accuracy when processing images of different scales, and cannot accurately detect pedestrians when there is too much occlusion in the background.

Method used

A multi-scale multi-feature neural network is used to extract feature maps of different scales through feature pyramid structure networks and fuse them. The feature information is refined in combination with convolution operations, and the cross entropy loss function and smooth L1 loss function are used for model training.

Benefits of technology

The detection ability of targets of different sizes is improved, the identification ability of pedestrian targets is enhanced, especially the detection accuracy of pedestrians with small targets in complex environments, and the generalization ability and training efficiency of the model are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942114A_ABST
    Figure CN118942114A_ABST
Patent Text Reader

Abstract

The invention provides a pedestrian target detection method based on a multi-scale multi-feature neural network, and belongs to the technical field of target detection based on computer vision. The method comprises the following steps: firstly, acquiring pedestrian image data and preprocessing the pedestrian image data; secondly, for the processed data, constructing a multi-scale feature extraction network to extract feature maps of different scales; thirdly, a multi-scale feature fusion network is constructed to fuse feature maps of different scales, so that the network can learn richer feature information, and the ability of the model to detect targets of different sizes is improved; thirdly, constructing a pedestrian target detection network, and inputting the fusion features into the network to obtain a pedestrian target detection result; and finally, designing a loss function, calculating a gradient through a back propagation algorithm to update the weight of the model, and obtaining a final target detection model through sufficient training, and deploying and using the target detection model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian detection, and in particular to a pedestrian target detection method based on a multi-scale and multi-feature neural network. Background Art

[0002] Pedestrian detection technology aims to identify and locate pedestrians from images or video sources. This technology has become an important tool for evaluating whether there are pedestrians in images or video sequences, and marking and displaying them. At present, pedestrian detection technology has been widely used in many fields such as intelligent driving assistance systems, intelligent robotics technology, and intelligent video surveillance, and has played a key role in improving the intelligence level of these systems. Pedestrian detection technology itself is not isolated. It is closely related to tasks such as pedestrian tracking and behavior recognition in video analysis. However, complex factors such as the various postures of pedestrians, occlusions between pedestrians, and occlusions between pedestrians and backgrounds have become key challenges affecting the accuracy of pedestrian detection. Therefore, improving the ability of pedestrian detection technology to detect small target pedestrians in complex environments has important research significance and application prospects.

[0003] The Chinese invention patent with application number CN202210979010.9 discloses a cross-modal adaptive fusion pedestrian target detection system and method for all-weather, which mainly includes two parts: a cross-modal differential information fusion module and a confidence-aware adaptive fusion module. The cross-modal differential information fusion module mainly performs complementary feature enhancement on the feature information of the visible light and infrared modalities extracted by the network, enhances the spatial information of the differential feature maps of the two through global pooling and average pooling operations, and then obtains the fusion feature vectors of each modality through the fully connected layer and Tanh activation function, and then performs feature enhancement expression on the initially extracted visible light and infrared modal features respectively; the confidence-aware adaptive fusion module makes full use of the confidence-aware expression to adaptively weight the features between the enhanced different modalities, so that the network detector can better select reliable modalities for processing and improve the robustness of the detector; finally, the multi-task loss is used to optimize the network model parameters.

[0004] The Chinese invention patent with application number CN202410103120.8 discloses a pedestrian target detection method, device, equipment and medium based on a monitoring scenario, including the following steps: Step 1, real-time acquisition of the current video frame in the target monitoring; Step 2, inputting the current video frame into a pre-trained head point positioning deep learning network model for head point positioning processing, and obtaining at least one pedestrian head point position corresponding to each frame image in the current video frame; Step 3, obtaining the pedestrian bounding box corresponding to each frame image in the current video frame through an improved target detection algorithm; Step 4, optimizing the pedestrian bounding box through each of the pedestrian head point positions to obtain a pedestrian target detection bounding box result, so as to determine the number and position of pedestrians corresponding to the current video frame according to the pedestrian target detection bounding box result.

[0005] The existing technology has the following limitations:

[0006] (1) When the data input to the model are images of different scales, the detection accuracy of the above method is low;

[0007] (2) When the background in the image obscures the pedestrian too much, the above method cannot accurately detect the pedestrian in the image.

[0008] In summary, existing object detection technologies have some challenges and shortcomings in pedestrian detection, and further research and improvement are needed to improve accuracy, efficiency and applicability. Summary of the invention

[0009] In view of the above problems, the first aspect of the present invention proposes a pedestrian target detection method based on a multi-scale and multi-feature neural network. The method first collects image data containing pedestrians and preprocesses them; secondly, for the processed data, a feature pyramid network is designed to extract feature maps of different scales and fuse them, so that the network can learn richer feature information, and the fused feature maps are further refined by convolution operation; finally, a cross entropy loss function and a smoothed L1 loss function are introduced, and the gradient is calculated by a back propagation algorithm to update the weight of the model. After sufficient training, the final target detection model is obtained and deployed for use, including the following processes:

[0010] Step 1, pedestrian data collection and processing: collect image data containing pedestrians, pre-process the original image data, and divide the pre-processed data into a training set and a test set;

[0011] Step 2: Design of multi-scale multi-feature fusion module: By introducing a feature pyramid structure network to process images at multiple scales simultaneously and fuse feature maps of different scales, the model can learn richer feature information.

[0012] Step 3: Design of pedestrian target detection module; using target detection model based on deep convolutional neural network;

[0013] Step 4: train the model; use the test set to test and verify the effect of the trained model, and save the final pedestrian target detection model;

[0014] Step 5: Deploy the model on a hardware platform to detect pedestrians.

[0015] Preferably, the step 1 specifically includes the following process:

[0016] S201, obtaining a real image containing pedestrians, and annotating the obtained images to form a corresponding label data set, wherein the image data containing pedestrians is recorded as {x1, x2, ..., xn}, and the corresponding labels are recorded as {y1, y2, ..., yn};

[0017] S202, preprocessing the image data described in S1, including using Gaussian filtering to smooth the image to improve the quality and clarity of the image data; using rotation, translation, and adjustment of brightness and contrast to perform data enhancement.

[0018] Preferably, the step 2 specifically includes the following process:

[0019] S301, introduces the feature pyramid structure; the feature pyramid structure enables the network to process images at multiple scales simultaneously. Each level of the pyramid is fused with the feature maps of the adjacent levels through top-down upsampling and lateral connections. Assume that Feature l Represents the feature map of the lth layer. The upsampling calculation process is as follows:

[0020] U l =Upsample(Feature l ) (1)

[0022] Among them, U l Represents the upsampled feature map, Upsample() represents the upsampling operation;

[0023] The feature fusion calculation process of the horizontal connection is as follows:

[0024] P l =F l +U (l―1) (2)

[0026] Among them, P l It is the fused feature map. lis the bottom-up l-th layer feature map, U (l-1) It is the result of upsampling the top-down (l-1)th layer feature map;

[0027] S302, further refine the fused feature map by performing a 3×3 convolution operation, so that each layer of the feature pyramid can generate a richer and more adaptable feature representation, thereby improving the performance of the entire network. The calculation process of the convolution operation is as follows:

[0028] R l =Conv 3×3 (P l ) (3)

[0030] Among them, R l It is the final feature map after the 3×3 convolution operation; further refinement can be regarded as a feature mapping process, in which the new value of each pixel in the feature map is obtained by weighted summation of the pixel values ​​in the surrounding neighborhood. The calculation process is as follows:

[0031] R ij =∑ m ∑ n Kernel mn ×Feature i―m,j―n (4)

[0032] Among them, R ij is the pixel value on the refined feature map, Feature i-m,j-n is the pixel value on the original feature map, Kernel mn is the weight of the convolution kernel, and m and n represent the positions of the elements in the convolution kernel.

[0033] Preferably, the step 3 specifically includes the following process:

[0034] The YOLOv8 network structure is selected as the main model structure of the pedestrian target detection module to receive the multi-scale fusion feature map output by the multi-scale multi-feature fusion module. The calculation process is as follows:

[0035] Y = YOLOv8(P l1 ,P l2 ,...,P ln ) (5)

[0036] Where Y is the output of YOLOv8, P li are feature maps of different scales.

[0037] Preferably, step 4 specifically includes the following process:

[0038] S501, define a loss function, select a cross entropy loss function as a classification loss function, and select a smooth L1 loss function as a bounding box regression loss function;

[0039] The calculation formula of the cross entropy loss function is as follows:

[0040]

[0041] Where N is the total number of samples, C is the total number of categories, and y i,c is the one-hot encoding of the true label of sample i, p i,c is the probability that the model predicts that sample i belongs to category c;

[0042] The calculation formula of the smooth L1 loss function is as follows:

[0043]

[0044] Where x is the difference between the predicted value and the true value;

[0045] S502, setting the model termination condition, when the model loss does not change for 10 consecutive times during the training process, the model training is terminated;

[0046] S503, setting the optimizer of the model, selecting Adam as the optimizer for model training, to update the parameters of the model and improve the convergence speed of the model;

[0047] S504, saving the fully trained optimal model for subsequent model deployment.

[0048] Preferably, the step 5 specifically includes the following process:

[0049] S601, converting the trained model into a format compatible with the hardware platform, and loading and initializing the model, including loading model weights, allocating memory space for the model to run, etc.;

[0050] S602, input the image data to be detected, and the model can detect pedestrians in the image. The calculation process is as follows:

[0051] Result=Model(Image) (8)

[0052] Model represents the trained network, Image represents the input image to be detected, and Result represents the result of pedestrian detection.

[0053] The second aspect of the present invention also provides a pedestrian target detection device based on a multi-scale and multi-feature neural network, characterized in that: the device includes at least one processor and at least one memory, and the processor and the memory are coupled; the memory stores a computer execution program of a pedestrian target detection model based on a multi-scale and multi-feature neural network constructed by the construction method described in the first aspect; when the processor executes the computer execution program stored in the memory, the processor executes a pedestrian target detection method based on a multi-scale and multi-feature neural network.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] (1) The feature pyramid structure network processes images at multiple scales simultaneously and fuses feature maps of different scales, enabling the model to learn richer feature information and improving the detection capability of objects of different sizes.

[0056] (2) The multi-scale and multi-feature fusion technology is used to enhance the model’s ability to identify pedestrian targets, especially the detection of small-target pedestrians in complex environments, thereby improving detection accuracy.

[0057] (3) In the data preprocessing stage, Gaussian filtering is used for image smoothing, and data enhancement is performed through methods such as rotation, translation, and adjustment of brightness and contrast, which further improves the generalization ability of the model. In the model training stage, the cross entropy loss function and smooth L1 loss function are selected, and the Adam optimizer is used to improve the efficiency and convergence speed of model training.

[0058] In general, the present invention provides a reliable solution for pedestrian target detection and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is the main step of the present invention.

[0060] Figure 2 This is the overall flow chart of the pedestrian target detection method based on multi-scale and multi-feature neural network.

[0061] Figure 3 Pedestrian detection results.

[0062] Figure 4 This is a simplified structural diagram of a pedestrian target detection device based on a multi-scale and multi-feature neural network in Example 2 of the present invention. DETAILED DESCRIPTION

[0063] The invention will be further described below in conjunction with specific embodiments.

[0064] Embodiment 1:

[0065] The present invention proposes a pedestrian target detection method based on a multi-scale multi-feature neural network. The overall process of the present invention is as follows: Figure 1 As shown in the figure, the overall process of pedestrian target detection method based on multi-scale multi-feature neural network is as follows Figure 2 As shown in the figure, the following key steps are included: first, by collecting image data containing pedestrians and preprocessing them; second, for the processed data, by designing a feature pyramid network to extract feature maps of different scales and fuse them, the network can learn richer feature information, and further refine the fused feature maps by using convolution operations; finally, the cross entropy loss function and the smoothed L1 loss function are introduced, and the gradient is calculated through the back propagation algorithm to update the model weights. After sufficient training, the final target detection model is obtained and deployed.

[0066] 1. Pedestrian data collection and processing, including the following steps:

[0067] S201, obtaining a real image containing pedestrians, and annotating the obtained images to form a corresponding label data set, wherein the image data containing pedestrians is recorded as {x1, x2, ..., xn}, and the corresponding labels are recorded as {y1, y2, ..., yn};

[0068] S202, preprocessing the image data described in S1, including using Gaussian filtering to smooth the image to improve the quality and clarity of the image data; using rotation, translation, and adjustment of brightness and contrast to perform data enhancement.

[0069] 2. Multi-scale and multi-feature fusion module design, including the following steps:

[0070] S301, introduces the feature pyramid structure; the feature pyramid structure enables the network to process images at multiple scales simultaneously. Each level of the pyramid is fused with the feature maps of the adjacent levels through top-down upsampling and lateral connections. Assume that Feature l Represents the feature map of the lth layer. The upsampling calculation process is as follows:

[0071] U l =Upsample(Feature l ) (1)

[0073] Among them, U l Represents the upsampled feature map, Upsample() represents the upsampling operation;

[0074] The feature fusion calculation process of the horizontal connection is as follows:

[0075] P l =Fl +U (l―1) (2)

[0077] Among them, P l It is the fused feature map. l is the bottom-up l-th layer feature map, U (l-1) It is the result of upsampling the top-down (l-1)th layer feature map;

[0078] S302, further refine the fused feature map by performing a 3×3 convolution operation, so that each layer of the feature pyramid can generate a richer and more adaptable feature representation, thereby improving the performance of the entire network. The calculation process of the convolution operation is as follows:

[0079] R l =Conv 3×3 (P l ) (3)

[0081] Among them, R l It is the final feature map after the 3×3 convolution operation; further refinement can be regarded as a feature mapping process, in which the new value of each pixel in the feature map is obtained by weighted summation of the pixel values ​​in the surrounding neighborhood. The calculation process is as follows:

[0082] R ij =∑ m ∑ n Kernel mn ×Feature i―m,j―n (4)

[0084] Among them, R ij is the pixel value on the refined feature map, Feature i-m,j-n is the pixel value on the original feature map, Kernel mn is the weight of the convolution kernel, and m and n represent the positions of the elements in the convolution kernel.

[0085] 3. The design of pedestrian target detection module includes the following steps:

[0086] The YOLOv8 network structure is selected as the main model structure of the pedestrian target detection module to receive the multi-scale fusion feature map output by the multi-scale multi-feature fusion module. The calculation process is as follows:

[0087] Y = YOLOv8(P l1 ,P l2 ,...,P ln ) (5)

[0088] Where Y is the output of YOLOv8, P li are feature maps of different scales.

[0089] 4. Neural network model training, including the following steps:

[0090] S501, define a loss function, select a cross entropy loss function as a classification loss function, and select a smooth L1 loss function as a bounding box regression loss function;

[0091] The calculation formula of the cross entropy loss function is as follows:

[0092]

[0093] Where N is the total number of samples, C is the total number of categories, and y i,c is the one-hot encoding of the true label of sample i, p i,c is the probability that the model predicts that sample i belongs to category c;

[0094] The calculation formula of the smooth L1 loss function is as follows:

[0095]

[0096] Where x is the difference between the predicted value and the true value;

[0097] S502, setting the model termination condition, when the model loss does not change for 10 consecutive times during the training process, the model training is terminated;

[0098] S503, setting the optimizer of the model, selecting Adam as the optimizer for model training, to update the parameters of the model and improve the convergence speed of the model;

[0099] S504, saving the fully trained optimal model for subsequent model deployment.

[0100] 5. Object detection model deployment, including the following steps:

[0101] S601, converting the trained model into a format compatible with the hardware platform, and loading and initializing the model, including loading model weights, allocating memory space for the model to run, etc.;

[0102] S602, input the image data to be detected, and the model can detect pedestrians in the image. The calculation process is as follows:

[0103] Result=Model(Image) (8)

[0104] Model represents the trained network, Image represents the input image to be detected, and Result represents the result of pedestrian detection.

[0105] The following example results are presented for this process:

[0106] In order to verify the effectiveness of the pedestrian target detection method based on a multi-scale multi-feature neural network proposed in the present invention, this embodiment provides the following display results: Figure 3 As shown, the white rectangular box selects all the pedestrians in the picture, and most of the pedestrians' bodies are selected, indicating that the method of the present invention has a high pedestrian target detection accuracy. This embodiment also provides a comparison of the results with the currently existing target detection models Fast R-CNN, SSD and YOLOv7, and the indicator for evaluating the performance of the model is the average accuracy mAP of all categories. The experimental results show that the mAP of Fast R-CNN is 83.4, the mAP of SSD is 86.1, the mAP of YOLOv7 is 86.8, and the mAP of the method of the present invention is 89.2. The method proposed in the present invention achieved the maximum mAP value among all the compared methods, indicating that the method proposed in the present invention has the highest pedestrian target detection accuracy compared with the existing methods, further verifying the effectiveness and practicality of the present invention.

[0107] Embodiment 2:

[0108] like Figure 4 As shown, the present invention also provides a pedestrian target detection device based on a multi-scale multi-feature neural network, the device includes at least one processor and at least one memory, and also includes a communication interface and an internal bus; a computer execution program is stored in the memory; a computer execution program for pedestrian target detection based on a multi-scale multi-feature neural network constructed by the construction method described in Example 1 is stored in the memory; when the processor executes the computer execution program stored in the memory, the processor can execute a pedestrian target detection method based on a multi-scale multi-feature neural network. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk.

[0109] The device may be provided as a terminal, a server or other forms of devices.

[0110] Figure 4 is a block diagram of a device shown as an example. The device may include one or more of the following components: a processing component, a memory, a power component, a multimedia component, an audio component, an input / output (I / O) interface, a sensor component, and a communication component. The processing component generally controls the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component may include one or more processors to execute instructions to complete all or part of the steps of the above method. In addition, the processing component may include one or more modules to facilitate interaction between the processing component and other components. For example, the processing component may include a multimedia module to facilitate interaction between the multimedia component and the processing component.

[0111] The memory is configured to store various types of data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0112] The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0113] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0114] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0115] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A pedestrian target detection method based on a multi-scale and multi-feature neural network, characterized in that: The process includes: Step 1: Pedestrian data collection and processing; Collect pedestrian image data, preprocess the original image data, and divide the preprocessed data into a training set and a test set; Step 2: Design a multi-scale feature extraction network to extract multi-scale features of pedestrian images and obtain rich features of different scales of images; Step 3, designing a multi-scale feature fusion network to fuse the multi-scale features of the pedestrian image obtained in step 2, so that the model can learn richer feature information; Step 4: Design a pedestrian target detection network; use a target detection model based on a deep convolutional neural network to detect the position of pedestrians in pedestrian images; Step 5: train the model; Use the test set to test and verify the effect of the trained model, and save the final pedestrian target detection model; Step 6: Model deployment: Deploy the model on the hardware platform to achieve real-time detection of pedestrians.

2. The pedestrian target detection method based on a multi-scale multi-feature neural network as claimed in claim 1, characterized in that: The step 1 is specifically The process includes: S201, obtain a real image containing pedestrians, and annotate the acquired images to form a corresponding label data set. The image data containing pedestrians is recorded as {x1, x2, ..., x n }, the corresponding labels are recorded as {y1,y2,...,y n }; S202, preprocessing the image data described in S1, including using Gaussian filtering to smooth the image to improve the quality and clarity of the image data; using rotation, translation, and adjustment of brightness and contrast to perform data enhancement.

3. The pedestrian target detection method based on a multi-scale multi-feature neural network as claimed in claim 1, characterized in that: The specific process of step 2 is: The multi-scale feature extraction module includes three upsampling layers of different scales and three channel attention mechanism layers; S301, input the pedestrian image into upsampling layers of different scales, and the upsampling layers of different scales upsample the input pedestrian image respectively to obtain feature maps of different scales. The calculation process is as follows: U l =Upsample l (image) (1) Among them, U l Represents the feature map obtained after the pedestrian image is input into the lth upsampling layer, Upsample l (·) represents the upsampling operation performed by the lth upsampling layer, image is a pedestrian image, l = 1, 2, 3; S302, U l Input channel attention mechanism layer to obtain channel attention weight P l , the calculation process is as follows: P l =σ(MLP(MPool(U l )+APool(U l ))) Where σ(·) is the Sigmoid activation function, MPool(·) and APool(·) represent the maximum pooling and average pooling operations respectively, and MLP(·) is a multi-layer perceptron. l with U l Perform product operation to obtain channel attention features Through the channel attention mechanism layer, the network can effectively capture the local spatial structure information in pedestrian images, enhance the attention to pedestrian features, and suppress unimportant environmental features.

4. The pedestrian target detection method based on a multi-scale multi-feature neural network as claimed in claim 1, characterized in that: The specific process of step 3 is as follows: The multi-scale feature fusion network includes a multi-head attention mechanism layer and a feature mapping layer; S401, the channel attention features obtained in step 2 and Input multiple attention layers in parallel to obtain representations and The fusion feature S1 of the interaction relationship is calculated as follows: Among them, MultiHead(Q, K, V) is the output of the multi-head attention mechanism. In this mechanism, each head generates an output vector, and then these output vectors are connected. Concat is a connection operation. After the connection, the obtained vector is multiplied by a weight matrix W o Perform a linear transformation and map it to the desired output dimension. are the linear projections of the i-th attention head, mapping the query, key, and value to the corresponding subspace; S402, further refine the S1 input feature mapping layer to obtain a richer and more adaptable feature representation S2, thereby improving the performance of the entire network. The j-th element in S2 is calculated by weighted summing the elements in the neighborhood of the j-th element in S1.

5. The pedestrian target detection method based on a multi-scale multi-feature neural network as claimed in claim 1, characterized in that: The specific process of step 4 is as follows: The pedestrian target detection network includes a spatial attention mechanism layer and a YOLOV8 target detection head network; S501, input S2 obtained in step 3 into the spatial attention mechanism layer to obtain the spatial attention weight K, and the specific calculation process is as follows: K=σ(C(MPool(S2)+APool(S2))) Where C(·) represents the convolution operation; K is multiplied by S2 to obtain the spatial attention feature S3; the spatial attention mechanism can help the model locate the position of pedestrians in the image more accurately; S502, input S3 into the YOLOv8 target detection head network to obtain the pedestrian target detection result; the YOLOv8 head network adopts a decoupled head structure to separate the classification and detection heads. This design allows the model to focus more on its respective tasks, thereby improving the positioning accuracy of pedestrians in pedestrian images.

6. The method for pedestrian target detection based on a multi-scale multi-feature neural network as claimed in claim 1, characterized in that: The model training in step 5 specifically includes the following process: S601, define a loss function, select a cross entropy loss function as a classification loss function, and select a smooth L1 loss function as a bounding box regression loss function; The calculation formula of the cross entropy loss function is as follows: Where N is the total number of samples, C is the total number of categories, and y i,c is the one-hot encoding of the true label of sample i, p i,c is the probability that the model predicts that sample i belongs to category c; The calculation formula of the smooth L1 loss function is as follows: Where x is the difference between the predicted value and the true value; S602, setting the model termination condition, when the loss of the model does not change for 10 consecutive times during the training process, the model training is terminated; S603, setting the optimizer of the model, selecting Adam as the optimizer for model training, to update the parameters of the model and improve the convergence speed of the model; S604, saving the fully trained optimal model for subsequent model deployment.

7. The method for pedestrian target detection based on a multi-scale multi-feature neural network as claimed in claim 1, characterized in that: The step 5 specifically includes the following processes for the deployment of the model: S701, converting the trained model into a format compatible with the hardware platform, and loading and initializing the model, including loading model weights, allocating memory space for the model to run, etc.; S702, input the image data to be detected, and the model can detect pedestrians in the image. The calculation process is as follows: Result = Model (Image) Model represents the trained network, Image represents the input image to be detected, and Result represents the result of pedestrian detection.

8. A pedestrian target detection device based on a multi-scale and multi-feature neural network, characterized in that: The device includes at least one processor and at least one memory, the processor and the memory are coupled; the memory stores a computer execution program of a pedestrian target detection model based on a multi-scale and multi-feature neural network constructed by the construction method described in any one of claims 1 to 5; when the processor executes the computer execution program stored in the memory, the processor executes a pedestrian target detection method based on a multi-scale and multi-feature neural network.

Citation Information

Patent Citations

  • All-weather-oriented cross-modal adaptive fusion pedestrian target detection system and method

    CN115410147A

  • Pedestrian target detection method and device based on monitoring scene, equipment and medium

    CN117934607A

  • Small target detection method based on attention mechanism

    CN114202672A

  • Pedestrian detection method of Faster R-CNN network based on improved clustering algorithm

    CN114332921A

  • Segmentation network training method and device, equipment, computer program and medium

    CN114581452A

Cited By

  • Multi-pedestrian target detection method based on YOLOCDG network in complex scene

    CN121330721A