Pedestrian detection method and system based on deep learning

Abstract

The invention provides a pedestrian detection method and system based on deep learning, and the method comprises the steps: obtaining scene image data in a to-be-detected scene, and carrying out the preprocessing of the scene image data; performing object shape recognition on the preprocessed scene image data based on a pre-training model, and performing feature extraction on an object shape recognition result based on an attention mechanism; and analyzing a feature extraction result, judging whether a pedestrian exists or not, and identifying and positioning the pedestrian when the pedestrian exists. According to the invention, whether the pedestrian exists or not is accurately and effectively judged, and when the pedestrian exists, the pedestrian is effectively identified and positioned, so that the reliability and the accuracy of pedestrian detection are ensured.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a pedestrian detection method and system based on deep learning. Background Art

[0002] At present, with the development of intelligent monitoring, autonomous driving and other fields, the accuracy and real-time requirements for pedestrian detection are getting higher and higher; pedestrian detection technology plays a vital role in traffic safety, autonomous driving, video surveillance and other fields; accurate and rapid detection of pedestrians is of great significance for ensuring the safety of pedestrians and vehicles, improving traffic efficiency and realizing intelligent urban management;

[0003] However, traditional methods mainly rely on manually designed feature extractors and simple classifiers; these methods usually require a lot of manual intervention to select and adjust features, and their performance in complex scenarios and changing environments is often not ideal, resulting in low detection accuracy, high false positive rate, and reduced reliability.

[0004] Therefore, in order to overcome the above technical problems, the present invention provides a pedestrian detection method and system based on deep learning. Summary of the invention

[0005] The present invention provides a pedestrian detection method and system based on deep learning, which is used to collect scene image data and pre-process the scene image data, thereby providing reliable images for pedestrian detection. Secondly, the pre-processed scene image data is used for object shape recognition through a pre-trained model, and the object shape recognition results are extracted according to the attention mechanism, ensuring that pedestrians in the scene image data can be accurately and effectively detected, avoiding the influence of the environment and obstacles on pedestrian occlusion. Finally, the feature extraction results are analyzed to achieve accurate and effective judgment on whether there are pedestrians, and when there are pedestrians, the pedestrians can be effectively identified and located, ensuring the reliability and accuracy of pedestrian detection.

[0006] The present invention provides a pedestrian detection method based on deep learning, comprising:

[0007] Step 1: Obtain scene image data of the scene to be detected and preprocess the scene image data;

[0008] Step 2: Perform object shape recognition on the preprocessed scene image data based on the pre-trained model, and perform feature extraction on the object shape recognition results based on the attention mechanism;

[0009] Step 3: Analyze the feature extraction results to determine whether there are pedestrians, and identify and locate pedestrians if there are pedestrians.

[0010] Preferably, in a pedestrian detection method based on deep learning, in step 1, scene image data of a scene to be detected is obtained, and the scene image data is preprocessed, including:

[0011] Determine the acquisition angle for capturing images of the scene to be detected based on the pedestrian detection requirements, and determine the target distance and target range at each acquisition angle;

[0012] Obtaining configuration parameters of the camera device based on the acquisition angle, the target distance, and the target range, and configuring the camera device based on the configuration parameters;

[0013] Based on the configuration results, the camera device is controlled to collect images of the scene to be detected at different collection angles to obtain corresponding scene image data, and the image upload channel is opened to permissions based on the collection results;

[0014] Based on the permission opening results, the scene image data under different acquisition angles are uploaded to the main control center in real time for preprocessing according to the image upload channel, and a reminder notification is sent to the main control center simultaneously.

[0015] Preferably, a pedestrian detection method based on deep learning, based on the permission opening result, uploads scene image data at different acquisition angles to the main control center in real time for preprocessing according to the image upload channel, including:

[0016] Obtain the scene image data obtained, and extract the timestamp of uploading the scene image data under different acquisition angles to the main control center;

[0017] The scene image data at different acquisition angles are pre-processed and sorted in sequence based on the order of timestamps, and the values ​​of different pixel points in the scene image data at each acquisition angle are determined in sequence based on the pre-processing sorting results;

[0018] Based on the values ​​of different pixel points, the corresponding pixel value distribution range is obtained, and the light and dark areas of the scene image data are divided based on the pixel value distribution range, and new domain value intervals are respectively allocated to the light and dark areas based on the image contrast enhancement requirements;

[0019] The pixel values ​​of each pixel point in the light and dark areas are respectively parameter mapped in the new domain value range, and the contrast adjustment of the scene image data at each acquisition angle is completed based on the mapping results, and the preprocessing operation is completed based on the contrast adjustment results.

[0020] Preferably, a pedestrian detection method based on deep learning completes a preprocessing operation based on a contrast adjustment result, comprising:

[0021] Acquire preprocessed scene image data and extract the data source of the scene image data;

[0022] Extract key information from the data source to obtain the acquisition angle and acquisition time corresponding to the scene image data, and generate personalized marking symbols for the scene image data based on the acquisition angle and acquisition time;

[0023] The scene image data is marked based on the personalized marking symbols, and the marking results are temporarily cached.

[0024] Preferably, a pedestrian detection method based on deep learning, in step 2, object shape recognition is performed on the preprocessed scene image data based on a pre-trained model, and feature extraction is performed on the object shape recognition result based on an attention mechanism, including:

[0025] Acquire preprocessed scene image data, and input the preprocessed scene image data into a pre-trained model;

[0026] Perform pixel traversal on the preprocessed scene image data based on the input result, and determine a preliminary sensitive image area based on the pixel traversal result, and determine an image distribution range of the preliminary sensitive image area;

[0027] Adding an initial offset to the standard convolution based on prior knowledge, and performing a first object shape recognition on a preliminary sensitive image area based on the added result;

[0028] Iteratively adjusting the initial offset a target number of times based on the first object shape recognition result, and performing object shape recognition a target number of times on the preliminary sensitive image area based on the iterative adjustment result to obtain a first object shape feature map;

[0029] At the same time, a convolution operation of a plane dimension is performed on the first object shape feature map based on the deep convolution in the pre-trained model, and a horizontal layer feature map is input based on the convolution operation of the plane dimension;

[0030] Processing the horizontal feature map point by point based on the point-by-point convolution in the pre-trained model, fusing the vertical feature information of the same position point based on the point-by-point processing result, and obtaining a second object shape feature map based on the fusion result;

[0031] Performing pooling processing on the second object shape feature map based on the attention mechanism, and performing adaptive feature modification on the second object shape feature map in plane dimension and space dimension based on the pooling processing to obtain an object shape recognition result;

[0032] The object shape recognition results are divided into image blocks, and a target weight is assigned to each image block based on the image block division results based on the attention mechanism;

[0033] Based on the allocation result, analysis resources are dynamically allocated to each image block, and features are extracted for each image block based on the analysis resources. The feature extraction results of each image block are summarized to obtain the target feature results.

[0034] Preferably, a pedestrian detection method based on deep learning determines the image distribution range of the preliminary sensitive image area, including:

[0035] The image distribution range of the obtained preliminary sensitive image area is obtained, and when the image distribution range is smaller than a preset range threshold, it is determined to start the small target detection mechanism;

[0036] Perform edge detection on the preliminary sensitive image area based on the small target detection mechanism, extract visual features of the object based on the edge detection results, and obtain shallow features of the preliminary sensitive image area based on the visual features;

[0037] At the same time, the preliminary sensitive image area is processed based on a preset deep neural network to obtain a target feature vector corresponding to the preliminary sensitive image area;

[0038] Based on the mapping dictionary between feature vectors and semantic features, the target feature vector is semantically parsed to obtain the semantic features corresponding to the target feature vector, and the deep features of the preliminary sensitive image area are obtained based on the semantic features;

[0039] The shallow features and deep features of the preliminary sensitive image area are fused at the same position to obtain the final feature map.

[0040] Preferably, in a pedestrian detection method based on deep learning, in step 3, the feature extraction result is analyzed to determine whether there is a pedestrian, including:

[0041] The feature extraction results are obtained, and at the same time, the size ratios and appearance characteristics of different parts of the human body are obtained based on big data, and the feature extraction results are matched with the size ratios and appearance characteristics of different parts of the human body for similarity;

[0042] A corresponding similarity value is obtained based on the similarity matching result, and when the similarity value is greater than a preset similarity threshold, it is determined that a pedestrian exists;

[0043] Otherwise, it is determined that there is no pedestrian.

[0044] Preferably, in a pedestrian detection method based on deep learning, in step 3, when a pedestrian exists, identifying and locating the pedestrian includes:

[0045] When it is determined that there is a pedestrian, the relative position relationship between the pedestrian and the objects in the background environment in the scene to be detected is determined based on the feature extraction result, and whether the objects in the background environment block the pedestrian based on the relative position relationship;

[0046] When there is occlusion, the occluded pedestrians and the unoccluded pedestrians are distinguished and circled, and the position distribution status of different pedestrians is obtained based on the distinction and circle results;

[0047] At the same time, imaging parameters of the image acquisition device are obtained, and coordinate conversion is performed on the positions of the blocked pedestrians and the unblocked pedestrians according to the position distribution state based on the imaging parameters;

[0048] Based on the coordinate transformation results, the actual distances between the blocked pedestrians and the unblocked pedestrians and the image acquisition device are obtained respectively, and the final pedestrian detection result is obtained based on the actual distance and position distribution state.

[0049] The present invention provides a pedestrian detection system based on deep learning, comprising:

[0050] An image acquisition and preprocessing module is used to acquire scene image data of a scene to be detected and preprocess the scene image data;

[0051] A feature extraction module is used to perform object shape recognition on the preprocessed scene image data based on the pretrained model, and to perform feature extraction on the object shape recognition results based on the attention mechanism;

[0052] The pedestrian detection module analyzes the feature extraction results to determine whether there are pedestrians, and identifies and locates pedestrians if there are pedestrians.

[0053] Preferably, a pedestrian detection system based on deep learning, an image acquisition and preprocessing module, comprises:

[0054] A parameter determination unit, used to determine the acquisition angle when acquiring images of the scene to be detected based on the pedestrian detection requirements, and determine the target distance and target range at each acquisition angle;

[0055] A device configuration unit, used to obtain configuration parameters of the camera device based on the acquisition angle, target distance and target range, and configure the camera device based on the configuration parameters;

[0056] Image acquisition and transmission unit, used for:

[0057] Based on the configuration results, the camera device is controlled to collect images of the scene to be detected at different collection angles to obtain corresponding scene image data, and the image upload channel is opened to permissions based on the collection results;

[0058] Based on the permission opening results, the scene image data under different acquisition angles are uploaded to the main control center in real time for preprocessing according to the image upload channel, and a reminder notification is sent to the main control center simultaneously.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. By collecting scene image data and preprocessing the scene image data, reliable images are provided for pedestrian detection. Secondly, the object shape recognition is performed on the preprocessed scene image data through the pre-training model, and the feature extraction of the object shape recognition result is performed according to the attention mechanism, ensuring that the pedestrians in the scene image data can be accurately and effectively detected, avoiding the influence of the environment and obstacles on the pedestrians. Finally, the feature extraction results are analyzed to achieve accurate and effective judgment on whether there are pedestrians, and when there are pedestrians, the pedestrians can be effectively identified and located, ensuring the reliability and accuracy of pedestrian detection.

[0061] 2. By analyzing the requirements for pedestrian detection, the acquisition angle and the target distance and target range at each acquisition angle can be accurately and effectively determined, so that the camera device can be accurately and effectively configured according to the acquisition angle, target distance and target range. Secondly, the configured camera device is controlled to perform corresponding image acquisition to ensure the comprehensiveness and reliability of the collected scene image data. Finally, the collected scene image data is uploaded to the main control center for preprocessing, which provides human protection and convenience for pedestrian detection.

[0062] 3. By processing the pre-trained model with the pre-processed scene image data input value, the preliminary sensitive image area in the scene image data, that is, the corresponding image distribution range, is determined, which provides a reference for determining the initial offset corresponding to the standard convolution. Secondly, the preliminary sensitive image area is identified by the determined offset. At the same time, the first object shape feature map obtained by identifying the preliminary sensitive image area is processed again by deep convolution and point-by-point convolution, and finally the second object shape feature map is effectively acquired, which provides a guarantee for feature extraction. Finally, the second object shape feature map is processed by the attention mechanism to achieve accurate and reliable feature extraction of the object shape, ensuring the accuracy and reliability of pedestrian detection.

[0063] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0066] Figure 1 is a flowchart of a pedestrian detection method based on deep learning in an embodiment of the present invention;

[0067] Figure 2 This is a flowchart of step 1 in a pedestrian detection method based on deep learning in an embodiment of the present invention;

[0068] Figure 3 4 is a structural diagram of a deep learning-based pedestrian detection system in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0070] Embodiment 1:

[0071] This embodiment provides a pedestrian detection method based on deep learning, such as Figure 1 As shown, including:

[0072] Step 1: Obtain scene image data of the scene to be detected and preprocess the scene image data;

[0073] Step 2: Perform object shape recognition on the preprocessed scene image data based on the pre-trained model, and perform feature extraction on the object shape recognition results based on the attention mechanism;

[0074] Step 3: Analyze the feature extraction results to determine whether there are pedestrians, and identify and locate pedestrians if there are pedestrians.

[0075] In this embodiment, the scene to be detected refers to a place where pedestrian detection is required, for example, the movement of pedestrians on the road is detected when a car is driving.

[0076] In this embodiment, the scene image data refers to the final image obtained after image acquisition of the scene to be detected, and is an image used to record the positional relationship between objects and pedestrians in the current environment.

[0077] In this embodiment, preprocessing refers to performing operations such as contrast adjustment on scene image data in order to provide reliable images for pedestrian detection.

[0078] In this embodiment, the pre-trained model is trained in advance and includes deformable convolution and depthwise separable convolution. The purpose is to recognize objects of different shapes in scene image data, thereby improving the accuracy of object recognition.

[0079] In this embodiment, the attention mechanism is a mechanism for focusing on parsing and extracting image information related to pedestrian features in the object shape recognition results, in order to distinguish the main content from the secondary content of the image, thereby achieving effective detection of pedestrians.

[0080] In this embodiment, identification and positioning refers to identifying and determining the position of the pedestrian and the distance between the pedestrian and the image acquisition device when the presence of the pedestrian is determined.

[0081] The working principle and beneficial effects of the above technical solution are: by collecting scene image data and preprocessing the scene image data, reliable images are provided for pedestrian detection; secondly, object shape recognition is performed on the preprocessed scene image data through a pre-trained model, and feature extraction is performed on the object shape recognition results according to the attention mechanism, ensuring that pedestrians in the scene image data can be accurately and effectively detected, avoiding the influence of the environment and obstacles on pedestrian occlusion; finally, the feature extraction results are analyzed to achieve accurate and effective judgment on whether there are pedestrians, and when there are pedestrians, effective identification and positioning of pedestrians are achieved, ensuring the reliability and accuracy of pedestrian detection.

[0082] Embodiment 2:

[0083] Based on Example 1, this embodiment provides a pedestrian detection method based on deep learning, such as Figure 2 As shown, in step 1, scene image data of the scene to be detected is obtained, and the scene image data is preprocessed, including:

[0084] Step 101: determining a collection angle for collecting images of a scene to be detected based on pedestrian detection requirements, and determining a target distance and a target range at each collection angle;

[0085] Step 102: obtaining configuration parameters of the camera device based on the acquisition angle, target distance and target range, and configuring the camera device based on the configuration parameters;

[0086] Step 103: Based on the configuration result, the camera device is controlled to collect images of the scene to be detected at different collection angles to obtain corresponding scene image data, and the image upload channel is opened to permissions based on the collection result;

[0087] Step 104: Based on the permission opening result, the scene image data at different acquisition angles are uploaded to the main control center in real time according to the image upload channel for preprocessing, and a reminder notification is sent to the main control center simultaneously.

[0088] In this embodiment, the pedestrian detection requirements are known in advance and are used to characterize the acquisition angle and corresponding acquisition range that need to be met when performing image acquisition.

[0089] In this embodiment, the target distance and the target range refer to the longitudinal distance and the lateral distance that need to be achieved when performing image acquisition.

[0090] In this embodiment, permission opening refers to starting and link configuring the image upload channel, that is, allowing the collected scene image data to be uploaded through the image upload channel.

[0091] The working principle and beneficial effects of the above technical solution are: by analyzing the requirements for pedestrian detection, the acquisition angle and the target distance and target range at each acquisition angle can be accurately and effectively determined, so that the camera device can be accurately and effectively configured according to the acquisition angle, target distance and target range. Secondly, the configured camera device is controlled to perform corresponding image acquisition to ensure the comprehensiveness and reliability of the collected scene image data. Finally, the collected scene image data is uploaded to the main control center for preprocessing, which provides human protection and convenience for pedestrian detection.

[0092] Embodiment 3:

[0093] Based on Example 2, this embodiment provides a pedestrian detection method based on deep learning, which uploads scene image data at different acquisition angles to the main control center in real time for preprocessing according to the permission opening result and the image upload channel, including:

[0094] Obtain the scene image data obtained, and extract the timestamp of uploading the scene image data under different acquisition angles to the main control center;

[0095] The scene image data at different acquisition angles are pre-processed and sorted in sequence based on the order of timestamps, and the values ​​of different pixel points in the scene image data at each acquisition angle are determined in sequence based on the pre-processing sorting results;

[0096] Based on the values ​​of different pixel points, the corresponding pixel value distribution range is obtained, and the light and dark areas of the scene image data are divided based on the pixel value distribution range, and new domain value intervals are respectively allocated to the light and dark areas based on the image contrast enhancement requirements;

[0097] The pixel values ​​of each pixel point in the light and dark areas are respectively parameter mapped in the new domain value range, and the contrast adjustment of the scene image data at each acquisition angle is completed based on the mapping results, and the preprocessing operation is completed based on the contrast adjustment results.

[0098] In this embodiment, the timestamp refers to the specific time information when the scene image data at different acquisition angles is uploaded to the main control center.

[0099] In this embodiment, the pixel value distribution range refers to the range of pixel values ​​in different areas of the scene image data, which provides a reference for contrast adjustment.

[0100] In this embodiment, the image contrast enhancement requirement is known in advance and is used to perform difference enhancement processing on the pixel values ​​of the bright and dark areas, that is, to increase the pixel values ​​of the pixel points in the bright area and to reduce the pixel values ​​of the pixel points in the dark area, wherein the new domain value interval is the parameter value interval for reassigning the values ​​of different pixel points in the bright and dark areas.

[0101] In this embodiment, performing parameter mapping on the pixel values ​​of each pixel point in the bright and dark areas in the new threshold value interval respectively means re-assigning the pixel values ​​of different pixel points by allocating the new threshold value interval, that is, increasing the image contrast.

[0102] The working principle and beneficial effects of the above technical solution are: by pre-processing the scene image data at different acquisition angles and sorting them in sequence, the pixel values ​​of the pixel points of the scene image data at different acquisition angles are determined according to the sorting results, and the scene image data is divided into light and dark areas according to the obtained pixel values ​​of the pixel points. Finally, the pixel values ​​of different pixel points in the light and dark areas are reassigned according to the division results, so as to achieve effective contrast adjustment of the scene image data, provide reliable images for pedestrian detection, and ensure the accuracy of pedestrian detection.

[0103] Embodiment 4:

[0104] Based on Example 3, this embodiment provides a pedestrian detection method based on deep learning, which completes the preprocessing operation based on the contrast adjustment result, including:

[0105] Acquire preprocessed scene image data and extract the data source of the scene image data;

[0106] Extract key information from the data source to obtain the acquisition angle and acquisition time corresponding to the scene image data, and generate personalized marking symbols for the scene image data based on the acquisition angle and acquisition time;

[0107] The scene image data is marked based on the personalized marking symbols, and the marking results are temporarily cached.

[0108] In this embodiment, the data source refers to the acquisition source of the scene image data and the specific configuration corresponding to the acquisition.

[0109] In this embodiment, the key information refers to a data segment in the data source that can characterize the acquisition angle and acquisition time of the scene image data.

[0110] In this embodiment, the personalized marking symbol is a symbol generated according to the acquisition angle and acquisition time, and is used to distinguish different scene images.

[0111] The working principle and beneficial effects of the above technical solution are: by extracting key information from the data source of different scene image data, the identity information of different scene image data can be effectively determined. Finally, personalized marking symbols are generated according to the obtained identity information, so that different scene image data can be effectively distinguished according to the personalized marking symbols, thereby facilitating the determination of the position and direction of the detected pedestrians and providing convenience for pedestrian detection.

[0112] Embodiment 5:

[0113] Based on Example 1, this embodiment provides a pedestrian detection method based on deep learning. In step 2, object shape recognition is performed on the preprocessed scene image data based on the pretrained model, and feature extraction is performed on the object shape recognition result based on the attention mechanism, including:

[0114] Acquire preprocessed scene image data, and input the preprocessed scene image data into a pre-trained model;

[0115] Perform pixel traversal on the preprocessed scene image data based on the input result, and determine a preliminary sensitive image area based on the pixel traversal result, and determine an image distribution range of the preliminary sensitive image area;

[0116] Adding an initial offset to the standard convolution based on prior knowledge, and performing a first object shape recognition on a preliminary sensitive image area based on the added result;

[0117] Iteratively adjusting the initial offset a target number of times based on the first object shape recognition result, and performing object shape recognition a target number of times on the preliminary sensitive image area based on the iterative adjustment result to obtain a first object shape feature map;

[0118] At the same time, a convolution operation of a plane dimension is performed on the first object shape feature map based on the deep convolution in the pre-trained model, and a horizontal layer feature map is input based on the convolution operation of the plane dimension;

[0119] Processing the horizontal feature map point by point based on the point-by-point convolution in the pre-trained model, fusing the vertical feature information of the same position point based on the point-by-point processing result, and obtaining a second object shape feature map based on the fusion result;

[0120] Performing pooling processing on the second object shape feature map based on the attention mechanism, and performing adaptive feature modification on the second object shape feature map in plane dimension and space dimension based on the pooling processing to obtain an object shape recognition result;

[0121] The object shape recognition results are divided into image blocks, and a target weight is assigned to each image block based on the image block division results based on the attention mechanism;

[0122] Based on the allocation result, analysis resources are dynamically allocated to each image block, and features are extracted for each image block based on the analysis resources. The feature extraction results of each image block are summarized to obtain the target feature results.

[0123] In this embodiment, the preliminary sensitive image area refers to an image area related to pedestrian detection obtained by analyzing the pixels of each pixel point in the scene image data, and is a part of the scene image data.

[0124] In this embodiment, the image distribution range is used to characterize the image size of the preliminary sensitive image area, so as to facilitate further effective detection of different objects in the scene image data according to the image distribution range.

[0125] In this embodiment, the prior knowledge is known in advance and is used to characterize the initial offset added to the standard convolution in different situations. It is obtained in advance by analyzing a large number of images.

[0126] In this embodiment, the standard convolution is trained in advance and is used to process the preliminary sensitive image area to achieve object recognition.

[0127] In this embodiment, the initial offset refers to the degree of deformation of the object to be detected in the preliminary sensitive image area. For example, the standard convolution is a regular box when performing object recognition, and the offset is the degree of enlarging and reducing the regular box. The purpose is to make the final frame fit the object, thereby achieving effective recognition of the object.

[0128] In this embodiment, the first object shape recognition refers to recognizing the shape of the object in the graphic according to the initial offset, in order to update the initial offset according to the recognition result, thereby ensuring the reliability of the final recognition result.

[0129] In this embodiment, the first object shape feature map refers to the result obtained after multiple adjustments to the initial offset, that is, the morphological result obtained after the irregular object in the image is finally recognized.

[0130] In this embodiment, deep convolution is trained in advance and is used to extract planar features in the shape feature map of the first object. In the deep convolution, each convolution kernel is only responsible for processing one channel of the input feature map, rather than processing all channels at the same time like traditional convolution. That is, for each input channel, there is a separate convolution kernel for convolution operation.

[0131] In this embodiment, the horizontal layer feature map refers to the result obtained by performing a convolution operation on the first object shape feature map in the plane dimension through depth convolution, and represents all features presented by the first object shape feature map on the plane channel.

[0132] In this embodiment, the point-by-point convolution is trained in advance, and the point-by-point convolution can fuse, filter or transform the features of each channel obtained by the deep convolution, thereby increasing the diversity and expression ability of the features.

[0133] In this embodiment, fusing the longitudinal feature information of the same position point refers to fusing the transverse plane features and the longitudinal feature information of the same position, in order to ensure the reliability of the features finally obtained.

[0134] In this embodiment, the second object shape feature map refers to a result obtained by processing the first object shape feature map through depthwise convolution and point-by-point convolution.

[0135] In this embodiment, pooling processing refers to downsampling the feature map to reduce the amount of data and calculation by reducing the size of the feature map, including maximum pooling and average pooling. Maximum pooling selects the maximum value in a local area of ​​the feature map as the output value of the area; average pooling calculates the average value of all values ​​in the local area as the output. Pooling processing can extract the main features while maintaining the relative position relationship of the features to a certain extent.

[0136] In this embodiment, adaptive feature modification refers to adjusting the second object shape feature map in the plane dimension and the space dimension respectively through the attention mechanism, in order to ensure the reliability of the final result.

[0137] In this embodiment, the target weight is used to represent the importance of each image block, and the larger the value, the more important it is.

[0138] In this embodiment, the parsing resources are known in advance, and are algorithms required for extracting features from each image block.

[0139] The working principle and beneficial effects of the above technical solution are: by processing the pre-trained model of the pre-processed scene image data input value, the preliminary sensitive image area in the scene image data, that is, the corresponding image distribution range, is determined, which provides a reference for determining the initial offset corresponding to the standard convolution, and secondly, the preliminary sensitive image area is identified by the determined offset. At the same time, the first object shape feature map obtained by identifying the preliminary sensitive image area is processed again by deep convolution and point-by-point convolution, and finally the second object shape feature map is effectively acquired, which provides a guarantee for feature extraction, and finally, the second object shape feature map is processed by the attention mechanism to realize accurate and reliable feature extraction of the object shape, thereby ensuring the accuracy and reliability of pedestrian detection.

[0140] Embodiment 6:

[0141] Based on Example 5, this embodiment provides a pedestrian detection method based on deep learning, which determines the image distribution range of the preliminary sensitive image area, including:

[0142] The image distribution range of the obtained preliminary sensitive image area is obtained, and when the image distribution range is smaller than a preset range threshold, it is determined to start a small target detection mechanism;

[0143] Perform edge detection on the preliminary sensitive image area based on the small target detection mechanism, extract visual features of the object based on the edge detection results, and obtain shallow features of the preliminary sensitive image area based on the visual features;

[0144] At the same time, the preliminary sensitive image area is processed based on a preset deep neural network to obtain a target feature vector corresponding to the preliminary sensitive image area;

[0145] Based on the mapping dictionary between feature vectors and semantic features, the target feature vector is semantically parsed to obtain the semantic features corresponding to the target feature vector, and the deep features of the preliminary sensitive image area are obtained based on the semantic features;

[0146] The shallow features and deep features of the preliminary sensitive image area are fused at the same position to obtain the final feature map.

[0147] In this embodiment, the preset range threshold is set in advance and is used as a basis for determining whether to activate the small target detection mechanism.

[0148] In this embodiment, the small target detection mechanism is a scheme for detecting small targets, that is, when the collected scene image data is far away, the area where the pedestrian appears in the scene image data is small, that is, small target detection is started.

[0149] In this embodiment, the visual feature refers to the contour shape of the object.

[0150] In this embodiment, the shallow features refer to the texture and color distribution of the object.

[0151] In this embodiment, the preset deep neural network is trained in advance.

[0152] In this embodiment, the target feature vector refers to feature information in the preliminary sensitive image region and is used for feature vector representation.

[0153] In this embodiment, the mapping dictionary between feature vectors and semantic features is known in advance and is used to record the corresponding relationship between different feature vectors and semantic features.

[0154] In this embodiment, the semantic features corresponding to the target feature vector refer to the type and position information of each object in the scene image data, that is, the deep features.

[0155] The working principle and beneficial effects of the above technical solution are: by combining multiple means to extract features at different levels, the feature description is made more comprehensive and detailed, and deep features are obtained through semantic analysis to better understand the semantic information of the image content. The fusion of shallow and deep features can obtain richer and more comprehensive feature representation, which is helpful for subsequent analysis and judgment, and effectively improves the ability and effect of feature extraction and analysis of sensitive image areas.

[0156] Embodiment 7:

[0157] Based on Example 1, this embodiment provides a pedestrian detection method based on deep learning. In step 3, the feature extraction result is analyzed to determine whether there is a pedestrian, including:

[0158] The feature extraction results are obtained, and at the same time, the size ratios and appearance characteristics of different parts of the human body are obtained based on big data, and the feature extraction results are matched with the size ratios and appearance characteristics of different parts of the human body for similarity;

[0159] A corresponding similarity value is obtained based on the similarity matching result, and when the similarity value is greater than a preset similarity threshold, it is determined that a pedestrian exists;

[0160] Otherwise, it is determined that there is no pedestrian.

[0161] In this embodiment, the preset similarity threshold is set in advance and is used as a criterion for determining whether a pedestrian exists.

[0162] The working principle and beneficial effects of the above technical solution are: first, obtain the feature extraction results, and then use big data to obtain benchmark information such as size ratios and appearance characteristics of different parts of the human body; compare the extracted features with these benchmark information one by one, and calculate the similarity; compare the similarity value with the preset similarity threshold, if it is greater than the threshold, it is judged that there is a pedestrian, otherwise it is judged that there is no pedestrian, which effectively improves the accuracy and reliability of judging whether there is a pedestrian, provides an accurate basis for subsequent further pedestrian detection and analysis, and helps the entire pedestrian detection system to operate efficiently and accurately.

[0163] Embodiment 8:

[0164] Based on Example 1, this embodiment provides a pedestrian detection method based on deep learning. In step 3, when a pedestrian exists, the pedestrian is identified and located, including:

[0165] When it is determined that there is a pedestrian, the relative position relationship between the pedestrian and the objects in the background environment in the scene to be detected is determined based on the feature extraction result, and whether the objects in the background environment block the pedestrian based on the relative position relationship;

[0166] When there is occlusion, the occluded pedestrians and the unoccluded pedestrians are distinguished and circled, and the position distribution status of different pedestrians is obtained based on the distinction and circle results;

[0167] At the same time, imaging parameters of the image acquisition device are obtained, and coordinate conversion is performed on the positions of the blocked pedestrians and the unblocked pedestrians according to the position distribution state based on the imaging parameters;

[0168] Based on the coordinate transformation results, the actual distances between the blocked pedestrians and the unblocked pedestrians and the image acquisition device are obtained respectively, and the final pedestrian detection result is obtained based on the actual distance and position distribution state.

[0169] The working principle and beneficial effects of the above technical solution are: determining the relative position relationship between pedestrians and background objects through feature extraction, so as to judge whether there is occlusion; in the case of occlusion, distinguishing and circled, clarifying the status of different pedestrians, and then, considering the imaging parameters of the image acquisition device, converting the position distribution state of the pedestrians into actual coordinates, which can more accurately reflect the position of the pedestrians in the real space; finally, the actual distance and position distribution state obtained by the coordinate conversion are used to comprehensively obtain the final pedestrian detection result, including the specific position of the pedestrian and the distance from the acquisition device and other information. Due to the application of more complex actual scenes, the adaptability and practicality of the pedestrian detection method in different environments are improved.

[0170] Embodiment 9:

[0171] This embodiment provides a pedestrian detection system based on deep learning, such as Figure 3 As shown, including:

[0172] An image acquisition and preprocessing module is used to acquire scene image data of a scene to be detected and to preprocess the scene image data;

[0173] A feature extraction module is used to perform object shape recognition on the preprocessed scene image data based on the pretrained model, and to perform feature extraction on the object shape recognition results based on the attention mechanism;

[0174] The pedestrian detection module analyzes the feature extraction results to determine whether there are pedestrians, and identifies and locates pedestrians if there are pedestrians.

[0175] The working principle and beneficial effects of the above technical solution are: by collecting scene image data and preprocessing the scene image data, reliable images are provided for pedestrian detection; secondly, object shape recognition is performed on the preprocessed scene image data through a pre-trained model, and feature extraction is performed on the object shape recognition results according to the attention mechanism, ensuring that pedestrians in the scene image data can be accurately and effectively detected, avoiding the influence of the environment and obstacles on pedestrian occlusion; finally, the feature extraction results are analyzed to achieve accurate and effective judgment on whether there are pedestrians, and when there are pedestrians, effective identification and positioning of pedestrians are achieved, ensuring the reliability and accuracy of pedestrian detection.

[0176] Embodiment 10:

[0177] Based on Example 9, this embodiment provides a pedestrian detection system based on deep learning, an image acquisition and preprocessing module, including:

[0178] A parameter determination unit, used to determine the acquisition angle when acquiring images of the scene to be detected based on the pedestrian detection requirements, and determine the target distance and target range at each acquisition angle;

[0179] A device configuration unit, used to obtain configuration parameters of the camera device based on the acquisition angle, target distance and target range, and configure the camera device based on the configuration parameters;

[0180] Image acquisition and transmission unit, used for:

[0181] Based on the configuration results, the camera device is controlled to collect images of the scene to be detected at different collection angles to obtain corresponding scene image data, and the image upload channel is opened to permissions based on the collection results;

[0182] Based on the permission opening results, the scene image data under different acquisition angles are uploaded to the main control center in real time for preprocessing according to the image upload channel, and a reminder notification is sent to the main control center simultaneously.

[0183] The working principle and beneficial effects of the above technical solution are: by analyzing the requirements for pedestrian detection, the acquisition angle and the target distance and target range at each acquisition angle can be accurately and effectively determined, so that the camera device can be accurately and effectively configured according to the acquisition angle, target distance and target range. Secondly, the configured camera device is controlled to perform corresponding image acquisition to ensure the comprehensiveness and reliability of the collected scene image data. Finally, the collected scene image data is uploaded to the main control center for preprocessing, which provides human protection and convenience for pedestrian detection.

[0184] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A pedestrian detection method based on deep learning, characterized in that: include: Step 1: Obtain scene image data of the scene to be detected and preprocess the scene image data; Step 2: Perform object shape recognition on the preprocessed scene image data based on the pre-trained model, and perform feature extraction on the object shape recognition results based on the attention mechanism; Step 3: Analyze the feature extraction results to determine whether there are pedestrians, and identify and locate pedestrians if there are pedestrians.

2. The pedestrian detection method based on deep learning according to claim 1, characterized in that: In step 1, scene image data of the scene to be detected is obtained, and the scene image data is preprocessed, including: Determine the acquisition angle for capturing images of the scene to be detected based on the pedestrian detection requirements, and determine the target distance and target range at each acquisition angle; Obtaining configuration parameters of the camera device based on the acquisition angle, the target distance, and the target range, and configuring the camera device based on the configuration parameters; Based on the configuration results, the camera device is controlled to collect images of the scene to be detected at different collection angles to obtain corresponding scene image data, and the image upload channel is opened to permissions based on the collection results; Based on the permission opening results, the scene image data under different acquisition angles are uploaded to the main control center in real time for preprocessing according to the image upload channel, and a reminder notification is sent to the main control center simultaneously.

3. The pedestrian detection method based on deep learning according to claim 2, characterized in that: Based on the permission opening results, the scene image data at different acquisition angles is uploaded to the main control center in real time for preprocessing according to the image upload channel, including: Obtain the scene image data obtained, and extract the timestamp of uploading the scene image data under different acquisition angles to the main control center; The scene image data at different acquisition angles are pre-processed and sorted in sequence based on the order of timestamps, and the values ​​of different pixel points in the scene image data at each acquisition angle are determined in sequence based on the pre-processing sorting results; Based on the values ​​of different pixel points, the corresponding pixel value distribution range is obtained, and the light and dark areas of the scene image data are divided based on the pixel value distribution range, and new domain value intervals are respectively allocated to the light and dark areas based on the image contrast enhancement requirements; The pixel values ​​of each pixel point in the light and dark areas are respectively parameter mapped in the new domain value range, and the contrast adjustment of the scene image data at each acquisition angle is completed based on the mapping results, and the preprocessing operation is completed based on the contrast adjustment results.

4. The pedestrian detection method based on deep learning according to claim 3, characterized in that: Based on the contrast adjustment result, preprocessing operations are completed, including: Acquire preprocessed scene image data and extract the data source of the scene image data; Extract key information from the data source to obtain the acquisition angle and acquisition time corresponding to the scene image data, and generate personalized marking symbols for the scene image data based on the acquisition angle and acquisition time; The scene image data is marked based on the personalized marking symbols, and the marking results are temporarily cached.

5. The pedestrian detection method based on deep learning according to claim 1, characterized in that: In step 2, object shape recognition is performed on the preprocessed scene image data based on the pretrained model, and feature extraction is performed on the object shape recognition result based on the attention mechanism, including: Acquire preprocessed scene image data, and input the preprocessed scene image data into a pre-trained model; Perform pixel traversal on the preprocessed scene image data based on the input result, and determine a preliminary sensitive image area based on the pixel traversal result, and determine an image distribution range of the preliminary sensitive image area; Adding an initial offset to the standard convolution based on prior knowledge, and performing a first object shape recognition on a preliminary sensitive image area based on the added result; Iteratively adjusting the initial offset a target number of times based on the first object shape recognition result, and performing object shape recognition a target number of times on the preliminary sensitive image area based on the iterative adjustment result to obtain a first object shape feature map; At the same time, a convolution operation of a plane dimension is performed on the first object shape feature map based on the deep convolution in the pre-trained model, and a horizontal layer feature map is input based on the convolution operation of the plane dimension; Processing the horizontal feature map point by point based on the point-by-point convolution in the pre-trained model, fusing the vertical feature information of the same position point based on the point-by-point processing result, and obtaining a second object shape feature map based on the fusion result; Performing pooling processing on the second object shape feature map based on the attention mechanism, and performing adaptive feature modification on the second object shape feature map in plane dimension and space dimension based on the pooling processing to obtain an object shape recognition result; The object shape recognition results are divided into image blocks, and a target weight is assigned to each image block based on the image block division results based on the attention mechanism; Based on the allocation result, analysis resources are dynamically allocated to each image block, and features are extracted for each image block based on the analysis resources. The feature extraction results of each image block are summarized to obtain the target feature results.

6. The pedestrian detection method based on deep learning according to claim 5, characterized in that: Determine the image distribution range of the preliminary sensitive image area, including: The image distribution range of the obtained preliminary sensitive image area is obtained, and when the image distribution range is smaller than a preset range threshold, it is determined to start a small target detection mechanism; Perform edge detection on the preliminary sensitive image area based on the small target detection mechanism, extract visual features of the object based on the edge detection results, and obtain shallow features of the preliminary sensitive image area based on the visual features; At the same time, the preliminary sensitive image area is processed based on a preset deep neural network to obtain a target feature vector corresponding to the preliminary sensitive image area; Based on the mapping dictionary between feature vectors and semantic features, the target feature vector is semantically parsed to obtain the semantic features corresponding to the target feature vector, and the deep features of the preliminary sensitive image area are obtained based on the semantic features; The shallow features and deep features of the preliminary sensitive image area are fused at the same position to obtain the final feature map.

7. The pedestrian detection method based on deep learning according to claim 1, characterized in that: In step 3, the feature extraction results are analyzed to determine whether there are pedestrians, including: The feature extraction results are obtained, and at the same time, the size ratios and appearance characteristics of different parts of the human body are obtained based on big data, and the feature extraction results are matched with the size ratios and appearance characteristics of different parts of the human body for similarity; A corresponding similarity value is obtained based on the similarity matching result, and when the similarity value is greater than a preset similarity threshold, it is determined that a pedestrian exists; Otherwise, it is determined that there is no pedestrian.

8. The pedestrian detection method based on deep learning according to claim 1, characterized in that: In step 3, when a pedestrian exists, the pedestrian is identified and located, including: When it is determined that there is a pedestrian, the relative position relationship between the pedestrian and the objects in the background environment in the scene to be detected is determined based on the feature extraction result, and whether the objects in the background environment block the pedestrian based on the relative position relationship; When there is occlusion, the occluded pedestrians and the unoccluded pedestrians are distinguished and circled, and the position distribution status of different pedestrians is obtained based on the distinction and circle results; At the same time, imaging parameters of the image acquisition device are obtained, and coordinate conversion is performed on the positions of the blocked pedestrians and the unblocked pedestrians according to the position distribution state based on the imaging parameters; Based on the coordinate transformation results, the actual distances between the blocked pedestrians and the unblocked pedestrians and the image acquisition device are obtained respectively, and the final pedestrian detection result is obtained based on the actual distance and position distribution state.

9. A pedestrian detection system based on deep learning, characterized in that: include: An image acquisition and preprocessing module is used to acquire scene image data of a scene to be detected and preprocess the scene image data; A feature extraction module is used to perform object shape recognition on the preprocessed scene image data based on the pretrained model, and to perform feature extraction on the object shape recognition results based on the attention mechanism; The pedestrian detection module analyzes the feature extraction results to determine whether there are pedestrians, and identifies and locates pedestrians if there are pedestrians.

10. The deep learning-based pedestrian detection system according to claim 9, characterized in that: Image acquisition and preprocessing module, including: A parameter determination unit, used to determine the acquisition angle when acquiring images of the scene to be detected based on the pedestrian detection requirements, and determine the target distance and target range at each acquisition angle; A device configuration unit, used to obtain configuration parameters of the camera device based on the acquisition angle, target distance and target range, and configure the camera device based on the configuration parameters; Image acquisition and transmission unit, used for: Based on the configuration results, the camera device is controlled to collect images of the scene to be detected at different collection angles to obtain corresponding scene image data, and the image upload channel is opened to permissions based on the collection results; Based on the permission opening results, the scene image data under different acquisition angles are uploaded to the main control center in real time for preprocessing according to the image upload channel, and a reminder notification is sent to the main control center simultaneously.

Images