Pedestrian detection method and device in complex environment, equipment and storage medium
By employing adaptive multi-scale convolution, large separable kernel attention mechanism, and content-aware feature recombination upsampling technology, the problems of omission and false detection in pedestrian detection under complex environments are solved, thereby improving the accuracy of pedestrian detection.
Patent Information
- Application Number
- CN202510967612.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-14
- Publication Date
- 2025-11-21
AI Technical Summary
Existing pedestrian detection algorithms are prone to omissions and false detections when detecting pedestrians in complex environments, resulting in inconsistent detection accuracy.
Adaptive multi-scale convolution operation, large separable kernel attention mechanism and content-aware feature recombination upsampling technique are adopted. The adaptive multi-scale convolution operation is used to extract features from the image data to be detected to generate a primary feature map. The large separable kernel attention mechanism is used to enhance the features of key regions, and the content-aware feature recombination upsampling is used to generate a high-resolution feature map, which is then used for pedestrian detection.
It significantly improves the ability to extract pedestrian features and the accuracy of detail reconstruction in complex environments, thereby increasing the accuracy of pedestrian detection.
Smart Images
Figure CN120997874A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of target detection, and particularly relates to a pedestrian detection method and device in a complex environment, equipment and a storage medium. BACKGROUND
[0002] At present, with continuous iteration and update of various target detection models, pedestrian detection has also been developed. However, due to the complex real environment, various clothes of pedestrians and other reasons, the pedestrian detection accuracy in a complex background cannot be stably maintained at a high level. The existing pedestrian detection algorithm is prone to missing and false detection when detecting pedestrians in a complex environment. Therefore, how to improve the accuracy of pedestrian detection is still a problem to be solved.
[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY
[0004] The main purpose of the present application is to provide a pedestrian detection method, device, equipment and storage medium in a complex environment, which aims to solve the technical problem of how to improve the accuracy of pedestrian detection.
[0005] To achieve the above purpose, the present application provides a pedestrian detection method in a complex environment, which comprises:
[0006] Receiving pedestrian image data to be detected, performing feature extraction on the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map;
[0007] According to the large separable kernel attention mechanism, the key region features in the primary feature map are enhanced to generate a fusion feature map;
[0008] According to the content perception feature reorganization up-sampling, the fusion feature map is processed to generate a high-resolution feature map;
[0009] According to the high-resolution feature map, pedestrian detection is performed to obtain a detection result.
[0010] In an embodiment, the step of performing feature extraction on the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map comprises:
[0011] The channel reorganization convolution branch and the spatial perception convolution branch are used to process the pedestrian image data to be detected respectively to obtain a first feature and a second feature;
[0012] The first feature and the second feature are fused to generate a primary feature map.
[0013] In an embodiment, before the channel reorganization convolution branch and the spatial perception convolution branch process the to-be-detected pedestrian image data respectively, the method further comprises:
[0014] obtaining learnable weight parameters;
[0015] adjusting the output proportion of the channel reorganization convolution branch and the spatial perception convolution branch according to the learnable weight parameters.
[0016] In an embodiment, the step of enhancing key region features in the primary feature map according to the large separable kernel attention mechanism to generate a fusion feature map comprises:
[0017] decomposing the primary feature map into a channel feature map and a spatial feature map;
[0018] performing point-wise convolution operation on the channel feature map to generate channel attention weights;
[0019] performing depth separable convolution operation on the spatial feature map to generate spatial attention weights;
[0020] reorganizing the channel feature map and the spatial feature map according to the channel attention weights and the spatial attention weights to generate the fusion feature map.
[0021] In an embodiment, the step of processing the fusion feature map according to content-aware feature reorganization up-sampling to generate a high-resolution feature map comprises:
[0022] performing feature encoding on the fusion feature map to generate soft assignment weight parameters;
[0023] reorganizing feature information within a local receptive field according to the soft assignment weight parameters to obtain a feature reorganization result;
[0024] constructing a high-resolution feature representation through the feature reorganization result to generate the high-resolution feature map.
[0025] In an embodiment, the step of performing pedestrian detection according to the high-resolution feature map to obtain a detection result comprises:
[0026] performing depth separable convolution on the high-resolution feature map to generate a spatial positioning feature vector;
[0027] performing classification convolution on the spatial positioning feature vector to generate a class probability distribution;
[0028] obtaining a detection result according to the spatial positioning feature vector and the class probability distribution.
[0029] In an embodiment, after the step of obtaining a detection result, the method further comprises:
[0030] calculating a confidence score of the detection result;
[0031] when the confidence score is lower than a preset threshold, performing secondary detection until the confidence score meets the requirement.
[0032] In addition, to achieve the above object, the application further provides a pedestrian detection device in a complex environment, which comprises:
[0033] a receiving module, configured to receive pedestrian image data to be detected, and perform feature extraction on the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map;
[0034] a fusion module, configured to enhance key region features in the primary feature map according to a large separable kernel attention mechanism to generate a fusion feature map;
[0035] a generation module, configured to process the fusion feature map according to content-aware feature reorganization up-sampling to generate a high-resolution feature map;
[0036] a detection module, configured to perform pedestrian detection according to the high-resolution feature map to obtain a detection result.
[0037] In addition, to achieve the above object, the application further provides a pedestrian detection device in a complex environment, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the pedestrian detection method in a complex environment as described above.
[0038] In addition, to achieve the above object, the application further provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the pedestrian detection method in a complex environment as described above.
[0039] In addition, to achieve the above object, the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the pedestrian detection method in a complex environment as described above.
[0040] The application provides a pedestrian detection method in a complex environment, the application receives pedestrian image data to be detected, performs feature extraction on the pedestrian image data to be detected through adaptive multi-scale convolution operation, obtains a primary feature map, enhances key region features in the primary feature map according to a large separable kernel attention mechanism, generates a fusion feature map, processes the fusion feature map according to content perception feature reorganization up-sampling, generates a high-resolution feature map, performs pedestrian detection according to the high-resolution feature map, and obtains a detection result.
[0041] In conclusion, the application significantly improves the pedestrian feature extraction capability and detail reconstruction precision in a complex environment through adaptive multi-scale convolution, a large separable kernel attention mechanism and content perception feature reorganization up-sampling, and improves the accuracy of pedestrian detection. BRIEF DESCRIPTION OF DRAWINGS
[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application together with the specification.
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.
[0044] Figure 1 A flowchart is provided for the pedestrian detection method in a complex environment according to Embodiment 1 of the application;
[0045] Figure 2 An improved YOLOv11 network structure diagram is provided for the pedestrian detection method in a complex environment according to Embodiment 1 of the application;
[0046] Figure 3 An ADown network structure is provided for the pedestrian detection method in a complex environment according to Embodiment 1 of the application;
[0047] Figure 4 A CARAFE up-sampling process diagram is provided for the pedestrian detection method in a complex environment according to Embodiment 1 of the application;
[0048] Figure 5 A flowchart is provided for the pedestrian detection method in a complex environment according to Embodiment 2 of the application;
[0049] Figure 6 An LSKA structure diagram is provided for the pedestrian detection method in a complex environment according to Embodiment 2 of the application;
[0050] Figure 7A module structure schematic diagram of a pedestrian detection device in a complex environment for an embodiment of the present application.
[0051] Figure 8 A device structure schematic diagram of a hardware running environment involved in a pedestrian detection method in a complex environment for an embodiment of the present application.
[0052] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION
[0053] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.
[0054] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0055] The main solution of the present application is to receive pedestrian image data to be detected, perform feature extraction on the pedestrian image data to be detected through adaptive multi-scale convolution operation, obtain a primary feature map, enhance key region features in the primary feature map according to a large separable kernel attention mechanism, generate a fusion feature map, process the fusion feature map according to content perception feature reorganization up-sampling, generate a high-resolution feature map, and perform pedestrian detection according to the high-resolution feature map to obtain a detection result.
[0056] At present, with the continuous iteration and update of various target detection models, pedestrian detection has also been developed. However, due to the complexity of the real environment, the diversity of pedestrian clothing, and other reasons, the pedestrian detection accuracy in a complex background cannot be stably maintained at a high level. The existing pedestrian detection algorithm is prone to missing and false detection when detecting pedestrians in a complex environment. Therefore, how to improve the accuracy of pedestrian detection is still a problem to be solved.
[0057] The present application significantly improves the pedestrian feature extraction capability and detail reconstruction accuracy in a complex environment through adaptive multi-scale convolution, large separable kernel attention mechanism and content perception feature reorganization up-sampling, and improves the accuracy of pedestrian detection.
[0058] Based on this, an embodiment of the present application provides a pedestrian detection method in a complex environment, which is described in detail with reference to Figure 1 , Figure 1 A flowchart of a first embodiment of a pedestrian detection method in a complex environment for the present application.
[0059] In the present embodiment, the pedestrian detection method in a complex environment includes steps S10-S40:
[0060] Step S10: receiving to-be-detected pedestrian image data, performing feature extraction on the to-be-detected pedestrian image data through an adaptive multi-scale convolution operation to obtain a primary feature map;
[0061] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a pedestrian detection device in a complex environment, etc. In the following, the pedestrian detection device in a complex environment is taken as an example to describe the embodiment and each of the following embodiments.
[0062] It should be noted that the network architecture of YOLOv11 includes four modules: an input processing module, a feature extraction backbone module, a feature fusion neck module and a prediction head module. The input processing module ensures the standardization of the input data, laying a foundation for subsequent extraction of high-quality features. The feature extraction backbone module uses innovative modules such as C3k2 to greatly improve the computational efficiency. The neck module integrates multi-scale features to enhance the model's perception of important regions. The prediction head module applies lightweight depthwise convolution (DWConv) structure, which significantly improves the computational efficiency while maintaining high performance. The embodiment is an improved pedestrian detection algorithm based on YOLOv11. For reference Figure 2 , Figure 2 The improved YOLOv11 network structure diagram. First, the adaptive downsampling (ADown) convolution proposed by YOLOv9 is used to replace the original Conv convolution, thereby effectively reducing the number of parameters and computational complexity and improving the computational efficiency of the model; second, the large separable kernel attention (LSKA) attention mechanism is integrated into the C2PSA module of YOLOv11 to strengthen the learning and capturing ability of the algorithm for pedestrian features and positioning. The LSKA attention mechanism is based on local context relationships and can accurately capture key regional information in images, thereby improving detection accuracy. Finally, the content-aware reassembly of features (CARAFE) upsampling is used to replace the original upsampling method, which enhances the multi-scale feature expression ability of the model and helps to further improve the detection accuracy.
[0063] It should be noted that the ADown module proposed in the YOLOv9 algorithm is used to replace the traditional Conv module in the embodiment, and the algorithm architecture is improved. The traditional Conv module has limitations in processing multi-scale targets, while the ADown module enhances the feature extraction capability and computational efficiency. The improvement of the embodiment enables the YOLOv11 to accurately detect different scale pedestrian targets while maintaining lightweight in complex scenes, thereby improving the performance of the model in pedestrian detection.
[0064] In one possible manner, the channel reorganization convolution branch and the spatial perception convolution branch are used to process the to-be-detected pedestrian image data respectively to obtain first features and second features; and the first features and the second features are fused to generate a primary feature map. Before the channel reorganization convolution branch and the spatial perception convolution branch are used to process the to-be-detected pedestrian image data, learnable weight parameters are obtained; and the output proportions of the channel reorganization convolution branch and the spatial perception convolution branch are adjusted according to the learnable weight parameters.
[0065] It should be noted that the ADown module itself uses parallel 1x1 and 3x3 convolution branches to realize adaptive fusion of multi-scale features. For reference, Figure 3 , Figure 3 is an ADown network structure. The 1x1 convolution is responsible for channel information reorganization and optimization of feature expression; and the 3x3 convolution is focused on spatial feature extraction and provides rich details. Meanwhile, the embodiment introduces learnable adaptive weight parameters to dynamically adjust the contribution proportions of the two branches: when processing small targets, the 3x3 convolution branch is emphasized to obtain spatial details, and when processing large targets, the 1x1 convolution branch is relied on more to optimize channel information. This design improves the adaptability of the model to different scale targets, balances the computational efficiency and feature expression capability, and reduces the computational overhead.
[0066] Step S20: enhancing key region features in the primary feature map according to the large separable kernel attention mechanism to generate a fusion feature map;
[0067] It should be noted that, in order to effectively improve the feature extraction efficiency of the C2PSA module and maintain a low computational overhead, the attention layer of the C2PSA module is replaced by a large separable kernel attention mechanism (LSKA) in the embodiment, thereby forming a new C2PSA_LSKA module. After this improvement, the C2PSA_LSKA module significantly reduces the computational overhead while maintaining high efficient feature extraction capability, successfully finding a better balance point between performance and efficiency.
[0068] Step S30: processing the fusion feature map according to content-aware feature reorganization up-sampling to generate a high-resolution feature map;
[0069] It should be noted that the CARAFE up-sampling is a feature map up-sampling method specially designed for computer vision tasks in deep learning, and performs well especially in object detection and image segmentation. Therefore, the CARAFE up-sampling is used to replace the original up-sampling method of YOLOv11. Unlike traditional up-sampling methods such as bilinear interpolation or deconvolution, CARAFE up-sampling up-samples feature maps in a content-aware manner, which can better preserve and restore detailed information.
[0070] In a feasible manner, the step of processing the fusion feature map according to the content-aware feature reorganization up-sampling to generate a high-resolution feature map includes: performing feature encoding on the fusion feature map to generate soft assignment weight parameters; reorganizing feature information in a local receptive field according to the soft assignment weight parameters to obtain a feature reorganization result; and constructing a high-resolution feature expression through the feature reorganization result to generate the high-resolution feature map.
[0071] CARAFE uses the content information of the feature map itself to guide the up-sampling process. For reference Figure 4 , Figure 4 is a schematic diagram of the CARAFE up-sampling process. Specifically, the CARAFE up-sampling module first performs feature extraction on the input feature map to generate a set of soft assignment weights related to spatial positions. These weights determine how to aggregate information from low-resolution feature maps to generate high-resolution feature maps. In this way, CARAFE can more accurately reconstruct high-quality feature maps, especially when dealing with objects with complex structures. The advantage of CARAFE up-sampling is that it can significantly improve the quality of the up-sampled feature map while maintaining computational efficiency. It does not rely on fixed interpolation methods, but dynamically generates the parameters needed for up-sampling according to the feature content, so that better results can be obtained in different image content and scenes. In addition, due to its flexibility and efficiency in design, CARAFE up-sampling can be easily integrated into various deep learning models to enhance the model's ability to capture details and improve overall performance.
[0072] Step S40: performing pedestrian detection according to the high-resolution feature map to obtain a detection result.
[0073] It can be understood that by performing pedestrian detection on the high-resolution feature map, the class probability of the detection target can be obtained, so as to determine the pedestrian detection result.
[0074] In a feasible manner, the pedestrian detection according to the high-resolution feature map to obtain a detection result comprises: performing a depth separable convolution on the high-resolution feature map to generate a spatial positioning feature vector; performing a classification convolution on the spatial positioning feature vector to generate a category probability distribution; and obtaining a detection result according to the spatial positioning feature vector and the category probability distribution.
[0075] It can be understood that after the pedestrian detection result is calculated, a confidence score of the detection result can also be calculated; when the confidence score is lower than a preset threshold, secondary detection is performed until the confidence score meets the requirement. Through multiple detections, the accuracy of pedestrian detection is improved.
[0076] The embodiment receives pedestrian image data to be detected, extracts features from the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map, enhances key region features in the primary feature map according to a large separable kernel attention mechanism to generate a fusion feature map, processes the fusion feature map according to content perception feature reorganization upsampling to generate a high-resolution feature map, and detects pedestrians according to the high-resolution feature map to obtain a detection result.
[0077] As can be seen above, the embodiment significantly improves the pedestrian feature extraction capability and detail reconstruction precision in a complex environment through adaptive multi-scale convolution, a large separable kernel attention mechanism, and content perception feature reorganization upsampling, and improves the accuracy of pedestrian detection.
[0078] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above-mentioned first embodiment can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 5 , step S20 further comprises steps S201-S204:
[0079] Step S201: decomposing the primary feature map into a channel feature map and a spatial feature map;
[0080] It can be understood that LSKA (Large Separable Kernel Attention) is a large separable kernel attention mechanism, which is an effective mechanism for improving feature extraction efficiency while considering the calculation cost. Please refer to Figure 6 , Figure 6 is a schematic diagram of the LSKA structure. Since the traditional attention mechanism often limits its performance and actual application due to high calculation complexity in the feature extraction process, the LSKA mechanism adopts the design idea of separable kernel. The attention operation of the large convolution kernel is decomposed, and in the embodiment, the primary feature map is decomposed into a channel feature map and a spatial feature map.
[0081] Step S202: performing a point-by-point convolution operation on the channel feature map to generate channel attention weights;
[0082] It can be understood that the correlation between different channels can be captured by fusing the information between channels through point-by-point convolution (1x1 convolution).
[0083] Step S203: performing a depth separable convolution operation on the spatial feature map to generate spatial attention weights;
[0084] It can be understood that the extraction of spatial features is focused by using operations such as depth separable convolution. This decomposition greatly reduces the computational amount, while retaining the ability to capture long-distance dependencies, and can more efficiently extract the context information and key features in the image.
[0085] Step S204: reorganizing the channel feature map and the spatial feature map according to the channel attention weights and the spatial attention weights to generate the fusion feature map.
[0086] It can be understood that the LSKA mechanism can also dynamically allocate attention weights, and flexibly adjust resource allocation according to the importance of different feature regions, so that key regions are more fully represented.
[0087] In this embodiment, the primary feature map is decomposed into a channel feature map and a spatial feature map; a point-by-point convolution operation is performed on the channel feature map to generate channel attention weights; a depth separable convolution operation is performed on the spatial feature map to generate spatial attention weights; and the channel feature map and the spatial feature map are reorganized according to the channel attention weights and the spatial attention weights to generate the fusion feature map. Through the large separable kernel attention mechanism, the point-by-point convolution and the depth separable convolution can more efficiently extract the context information and key features in the image, thereby improving the accuracy of pedestrian detection.
[0088] The application also provides a pedestrian detection device in a complex environment, please refer to Figure 7 The pedestrian detection device in a complex environment comprises:
[0089] The receiving module 10 is configured to receive pedestrian image data to be detected, and perform feature extraction on the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map.
[0090] The fusion module 20 is configured to enhance key region features in the primary feature map according to a large separable kernel attention mechanism to generate a fusion feature map.
[0091] The generation module 30 is configured to process the fusion feature map according to content-aware feature reorganization up-sampling to generate a high-resolution feature map.
[0092] The detection module 40 is configured to perform pedestrian detection according to the high-resolution feature map to obtain a detection result.
[0093] The embodiment receives pedestrian image data to be detected, extracts features from the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map, enhances key region features in the primary feature map according to a large separable kernel attention mechanism to generate a fusion feature map, processes the fusion feature map according to content-aware feature reorganization up-sampling to generate a high-resolution feature map, and performs pedestrian detection according to the high-resolution feature map to obtain a detection result.
[0094] As can be seen from the above, the embodiment significantly improves the pedestrian feature extraction capability and detail reconstruction accuracy in a complex environment through adaptive multi-scale convolution, a large separable kernel attention mechanism, and content-aware feature reorganization up-sampling, and improves the accuracy of pedestrian detection.
[0095] In an embodiment, the receiving module 10 is further configured to process the pedestrian image data to be detected through a channel reorganization convolution branch and a spatial perception convolution branch respectively to obtain first features and second features, and fuse the first features and the second features to generate a primary feature map.
[0096] In an embodiment, the receiving module 10 is further configured to obtain learnable weight parameters, and adjust the output proportions of the channel reorganization convolution branch and the spatial perception convolution branch according to the learnable weight parameters.
[0097] In an embodiment, the fusion module 20 is further configured to decompose the primary feature map into a channel feature map and a spatial feature map, perform point-by-point convolution operation on the channel feature map to generate channel attention weights, perform depth separable convolution operation on the spatial feature map to generate spatial attention weights, and reorganize the channel feature map and the spatial feature map according to the channel attention weights and the spatial attention weights to generate the fusion feature map.
[0098] In an embodiment, the generation module 30 is further configured to perform feature encoding on the fusion feature map to generate soft assignment weight parameters, reorganize feature information within a local receptive field according to the soft assignment weight parameters to obtain a feature reorganization result, and construct a high-resolution feature expression through the feature reorganization result to generate the high-resolution feature map.
[0099] In an embodiment, the detection module 40 is further configured to perform depth separable convolution on the high-resolution feature map to generate a spatial positioning feature vector, perform classification convolution on the spatial positioning feature vector to generate a class probability distribution, and obtain a detection result according to the spatial positioning feature vector and the class probability distribution.
[0100] In an embodiment, the detection module 40 is further configured to calculate a confidence score of the detection result; when the confidence score is lower than a preset threshold, a secondary detection is performed until the confidence score meets the requirement.
[0101] The pedestrian detection device in a complex environment provided in the present application adopts the pedestrian detection method in a complex environment in the above embodiments, and can solve the technical problem of how to improve the accuracy of pedestrian detection. Compared with the prior art, the pedestrian detection device in a complex environment provided in the present application has the same beneficial effects as the pedestrian detection method in a complex environment provided in the above embodiments, and other technical features in the pedestrian detection device in a complex environment are the same as the features disclosed in the above embodiments, which will not be repeated here.
[0102] The present application provides a pedestrian detection device in a complex environment, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the pedestrian detection method in a complex environment in Embodiment I.
[0103] Reference will be made to the following description Figure 8 which shows a structural schematic diagram of a pedestrian detection device in a complex environment suitable for implementing the embodiments of the present application. The pedestrian detection device in a complex environment in the embodiments of the present application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 8 The pedestrian detection device in a complex environment shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.
[0104] As Figure 8As shown, the pedestrian detection device in complex environment can include a processing device 1001 (for example, a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to programs stored in a ROM (Read Only Memory) 1002 or programs loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the pedestrian detection device in complex environment are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the pedestrian detection device in complex environment to communicate with other devices wirelessly or by wire to exchange data. Although the pedestrian detection device in complex environment with various systems is shown in the figure, it should be understood that all the systems shown are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.
[0105] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are performed.
[0106] The pedestrian detection device in complex environment provided by the present application adopts the pedestrian detection method in complex environment in the above-mentioned embodiments, and can solve the technical problem of how to improve the accuracy of pedestrian detection. Compared with the prior art, the pedestrian detection device in complex environment provided by the present application has the same beneficial effects as the pedestrian detection method in complex environment provided by the above-mentioned embodiments, and other technical features in the pedestrian detection device in complex environment are the same as the features disclosed in the previous embodiment method, which will not be repeated here.
[0107] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.
[0108] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be made by any person skilled in the art within the spirit and scope of the application are intended to be encompassed by the application. The scope of the application is defined by the appended claims.
[0109] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., computer programs) for performing the pedestrian detection method in a complex environment in the above embodiments.
[0110] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any appropriate medium, including but not limited to an electrical wire, an optical cable, an RF (Radio Frequency) cable, etc., or any appropriate combination thereof.
[0111] The above computer readable storage medium can be included in a pedestrian detection device in a complex environment; or can exist independently without being assembled into a pedestrian detection device in a complex environment.
[0112] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the pedestrian detection device in a complex environment, the pedestrian detection device in a complex environment is caused to: receive pedestrian image data to be detected, perform feature extraction on the pedestrian image data to be detected through an adaptive multi-scale convolution operation, obtain a primary feature map, enhance key region features in the primary feature map according to a large separable kernel attention mechanism, generate a fusion feature map, process the fusion feature map according to content perception feature reorganization up-sampling, generate a high-resolution feature map, and perform pedestrian detection according to the high-resolution feature map to obtain a detection result.
[0113] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).
[0114] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations for systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may
[0115] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.
[0116] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the pedestrian detection method in a complex environment described above, and can solve the technical problem of how to improve the accuracy of pedestrian detection. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the pedestrian detection method in a complex environment provided by the above embodiments, and will not be described here.
[0117] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the pedestrian detection method in a complex environment as described above.
[0118] The computer program product provided by the present application can solve the technical problem of how to improve the accuracy of pedestrian detection. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the pedestrian detection method in a complex environment provided by the above embodiments, and will not be described here.
[0119] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields based on the technical concept of the present application, and the contents of the present application specification and drawings are included in the patent protection scope of the present application.
Claims
1. A pedestrian detection method in a complex environment, characterized in that, The method includes: Receive pedestrian image data to be detected, and extract features from the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map; A fused feature map is generated by enhancing key region features in the primary feature map using a large separable kernel attention mechanism. The fused feature map is processed by resampling based on content-aware features to generate a high-resolution feature map; Pedestrian detection is performed based on the high-resolution feature map to obtain the detection results.
2. The method as described in claim 1, characterized in that, The step of extracting features from the pedestrian image data to be detected through adaptive multi-scale convolution operations to obtain a primary feature map includes: The pedestrian image data to be detected is processed by channel recombination convolution branch and spatial awareness convolution branch respectively to obtain the first feature and the second feature; The first feature and the second feature are combined to generate a primary feature map.
3. The method as described in claim 2, characterized in that, Before processing the pedestrian image data to be detected through the channel reconstruction convolution branch and the spatial awareness convolution branch respectively, the method further includes: Obtain learnable weight parameters; The output ratio of the channel recombination convolution branch to the spatially aware convolution branch is adjusted according to the learnable weight parameters.
4. The method as described in claim 1, characterized in that, The step of enhancing key region features in the primary feature map using a large separable kernel attention mechanism to generate a fused feature map includes: The primary feature map is decomposed into a channel feature map and a spatial feature map; Perform pointwise convolution on the channel feature map to generate channel attention weights; The spatial feature map is subjected to a depthwise separable convolution operation to generate spatial attention weights; The channel feature map and the spatial feature map are recombined based on the channel attention weight and the spatial attention weight to generate the fused feature map.
5. The method as described in claim 1, characterized in that, The step of processing the fused feature map by resampling based on content-aware features to generate a high-resolution feature map includes: The fused feature map is feature encoded to generate soft-assignment weight parameters; Based on the soft-assignment weight parameters, feature information is reconstructed within the local receptive field to obtain the feature reconstruction result; A high-resolution feature representation is constructed based on the feature recombination results, and the high-resolution feature map is generated.
6. The method as described in claim 1, characterized in that, The step of performing pedestrian detection based on the high-resolution feature map to obtain the detection result includes: The high-resolution feature map is subjected to depthwise separable convolution to generate spatial localization feature vectors; Perform a classification convolution on the spatial positioning feature vector to generate a category probability distribution; The detection result is obtained based on the spatial positioning feature vector and the category probability distribution.
7. The method as described in claim 1, characterized in that, After the step of obtaining the detection result, the method further includes: Calculate the confidence score of the detection results; When the confidence score is lower than a preset threshold, a second test is performed until the confidence score meets the requirements.
8. A pedestrian detection device in a complex environment, characterized in that, The device includes: The receiving module is used to receive pedestrian image data to be detected, and to extract features from the pedestrian image data to be detected through adaptive multi-scale convolution operation to obtain a primary feature map; The fusion module is used to enhance the key region features in the primary feature map based on a large separable kernel attention mechanism to generate a fused feature map; The generation module is used to process the fused feature map by resampling based on content-aware features to generate a high-resolution feature map; The detection module is used to perform pedestrian detection based on the high-resolution feature map and obtain the detection results.
9. A pedestrian detection device in a complex environment, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the pedestrian detection method in a complex environment as claimed in any one of claims 1 to 7.
10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the pedestrian detection method in a complex environment as described in any one of claims 1 to 7.