A pedestrian detection method, apparatus, equipment and medium

Abstract

This invention discloses a pedestrian detection method, apparatus, device, and medium. The method includes: acquiring a target image to be detected; inputting the target image into a preset detection network model, the preset detection network model being used to determine preset distribution parameter values ​​corresponding to each pedestrian in the target image, the preset distribution parameter values ​​corresponding to a two-dimensional probability distribution map, the probability distribution boundary of the two-dimensional probability distribution map being the full-body bounding box of the corresponding pedestrian, and the probability peak of the two-dimensional probability distribution map being located at the center of the visible bounding box of the corresponding pedestrian; and determining the pedestrian detection result corresponding to the target image based on the output of the preset detection network model. The technical solution of this invention can solve the problem of misclassification caused by occlusion, improving the accuracy of pedestrian detection.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer technology, and more particularly to a pedestrian detection method, apparatus, device, and medium. Background Technology

[0002] Pedestrian detection, a type of object detection, is used to identify and locate all pedestrians in an image. It is a fundamental prerequisite for pedestrian tracking, pedestrian re-identification, and human attribute analysis, and has wide applications in scenarios such as autonomous driving.

[0003] Currently, existing pedestrian detection methods use network models to detect whether an image contains pedestrians. If so, each pedestrian is located and their position and size information are represented by a rectangular bounding box.

[0004] However, in the process of realizing this invention, the inventors discovered at least the following problems in the prior art:

[0005] In crowded pedestrian scenes with occlusion, the bounding box corresponding to a highly occluded pedestrian will contain a lot of background or other noise information, making the probability value of the network model identifying it as a pedestrian low. This can lead to misclassification of pedestrians and reduce the accuracy of pedestrian detection. Summary of the Invention

[0006] This invention provides a pedestrian detection method, apparatus, device, and medium to address the issue of misclassification caused by occlusion and improve the accuracy of pedestrian detection.

[0007] In a first aspect, embodiments of the present invention provide a pedestrian detection method, comprising:

[0008] Acquire the target image to be detected;

[0009] The target image is input into a preset detection network model, which is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. The probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian. The probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian.

[0010] Based on the output of the preset detection network model, the pedestrian detection result corresponding to the target image is determined.

[0011] Secondly, embodiments of the present invention also provide a pedestrian detection device, comprising:

[0012] The target image acquisition module is used to acquire the target image to be detected.

[0013] The target image input module is used to input the target image into a preset detection network model. The preset detection network model is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. The probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian. The probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian.

[0014] The pedestrian detection result determination module is used to determine the pedestrian detection result corresponding to the target image based on the output of the preset detection network model.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0016] One or more processors;

[0017] a memory for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the pedestrian detection method provided in any embodiment of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the pedestrian detection method provided in any embodiment of the present invention.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] By inputting the target image to be detected into a preset detection network model, the preset detection network model detects pedestrians in the target image and determines the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map, so each pedestrian can be represented by the parameterized two-dimensional probability distribution map. That is, the probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian, and the probability peak of the two-dimensional probability distribution map is located at the center of the visible bounding box of the corresponding pedestrian. At this time, the probability value of the visible area of ​​the pedestrian is higher, and the probability value of the occluded area is lower. Thus, the two-dimensional probability distribution map can more accurately represent and detect pedestrians under occlusion, thereby solving the situation of misclassification caused by occlusion and improving the accuracy of pedestrian detection. Attached Figure Description

[0022] Figure 1 This is a flowchart of a pedestrian detection method provided in an embodiment of the present invention;

[0023] Figure 2This is a flowchart of a pedestrian detection method provided in an embodiment of the present invention;

[0024] Figure 3 This is an example of a preset detection network model involved in the embodiments of the present invention;

[0025] Figure 4 This is a flowchart of a pedestrian detection method provided in an embodiment of the present invention;

[0026] Figure 5 This is an example of a preset detection network model involved in the embodiments of the present invention;

[0027] Figure 6 This is a schematic diagram of the structure of a pedestrian detection device provided in an embodiment of the present invention;

[0028] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0030] Figure 1 This is a flowchart illustrating a pedestrian detection method provided in an embodiment of the present invention. This embodiment is applicable to detecting pedestrians contained in an image, particularly in crowded scenes with occlusions. The method can be executed by a pedestrian detection device, which can be implemented in software and / or hardware and integrated into an electronic device. Figure 1 As shown, the method specifically includes the following steps:

[0031] S110. Obtain the target image to be detected.

[0032] The target image can refer to an image collected for a specific application scenario that requires pedestrian detection. For example, a scene image from an autonomous driving scenario can be used as the target image. In this embodiment, an image collected in a crowded pedestrian scenario can be used as the target image, meaning that the target image contains occluded pedestrians.

[0033] Specifically, this embodiment can acquire target images in real time or acquire pre-acquired target images to meet different pedestrian detection needs.

[0034] S120. Input the target image into the preset detection network model. The preset detection network model is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. The probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian. The probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian.

[0035] The preset detection network model can be a pre-set network model that uses a parameterized two-dimensional probability distribution map to represent pedestrians. In this embodiment, the preset detection network model can be pre-trained based on sample data. The preset distribution parameter values ​​can refer to the parameter values ​​required to determine the two-dimensional probability distribution map. Each pedestrian detected by the preset detection network model can be represented using a two-dimensional probability distribution map. The two-dimensional probability distribution map can be composed of the probability value corresponding to each pixel. The probability value corresponding to each pixel can be used to represent the probability that the pixel belongs to the human body, that is, the importance of the pixel to the entire human body. The full-body bounding box of the pedestrian can refer to the entire human body region. The visible bounding box of the pedestrian can refer to the unobstructed visible area. The full-body bounding box of the pedestrian can be composed of the visible area and the obstructed invisible area. In this embodiment, the full-body bounding box and the visible bounding box can be represented using, but are not limited to, rectangular bounding boxes. The probability distribution boundary of the two-dimensional probability distribution map for each pedestrian is the full-body bounding box of the corresponding pedestrian, and the probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian. At this time, the probability value corresponding to the visible area of ​​the pedestrian is higher, and the probability value corresponding to the occluded invisible area is lower. Thus, the two-dimensional probability distribution map can be used to more accurately characterize and detect pedestrians under occlusion.

[0036] Specifically, the target image is input into a preset detection network model, which can detect each pedestrian in the target image and determine the preset distribution parameter value corresponding to each detected pedestrian. This preset distribution parameter value is then used to reconstruct the corresponding two-dimensional probability distribution map. Compared to directly determining the two-dimensional probability distribution map, indirectly obtaining it by determining the preset distribution parameter value significantly reduces the number of parameters that need to be determined, thus improving detection efficiency.

[0037] For example, the preset distribution parameter value for each pedestrian can refer to the value of the preset parameter corresponding to the two-dimensional Beta distribution. The preset parameter corresponding to the two-dimensional Beta distribution can include: the α parameter and β parameter corresponding to the X-axis, and the α parameter and β parameter corresponding to the Y-axis.

[0038] The two-dimensional beta distribution, also known as the B-distribution, is a continuous probability distribution defined on the interval [0,1] with two parameters: α and β. The preset parameters for the parameterized two-dimensional beta distribution can be determined based on the full-body bounding box and the visible bounding box of a pedestrian. Since the full-body bounding box and the visible bounding box are two-dimensional representations aligned with the coordinate axes, the two-dimensional beta distribution can be characterized using a joint two-dimensional distribution formed by beta distributions independent of the X-axis and Y-axis respectively. In this case, only two sets of parameters need to be determined: the α parameter corresponding to the X-axis. x and β parameter β x And the α parameter α corresponding to the Y-axis. y and β parameter β y This further improves detection efficiency.

[0039] S130. Determine the pedestrian detection result corresponding to the target image based on the output of the preset detection network model.

[0040] Specifically, the preset detection network model can output the preset distribution parameter values ​​corresponding to each detected pedestrian, thereby reconstructing the corresponding two-dimensional probability distribution map based on the preset distribution parameter values ​​corresponding to each pedestrian, so as to more accurately represent the position and occlusion of each pedestrian.

[0041] It should be noted that, compared to the rectangular bounding boxes in existing technologies, the parameterized two-dimensional probability distribution map can more accurately represent the shape and visibility information of pedestrians. Moreover, compared to paired full-body boxes and viewboxes, the parameterized two-dimensional probability distribution map is more like a whole, rather than two separate parts.

[0042] The technical solution of this embodiment involves inputting the target image to be detected into a preset detection network model. The preset detection network model detects pedestrians in the target image and determines the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. Thus, each pedestrian can be represented using the parameterized two-dimensional probability distribution map. That is, the probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian, and the probability peak of the two-dimensional probability distribution map is located at the center of the visible bounding box of the corresponding pedestrian. At this time, the probability value of the visible area of ​​the pedestrian is higher, and the probability value of the occluded area is lower. Therefore, the two-dimensional probability distribution map can more accurately represent and detect pedestrians under occlusion, thereby solving the situation of misclassification caused by occlusion and improving the accuracy of pedestrian detection.

[0043] Figure 2This is a flowchart of a pedestrian detection method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment optimizes the predictive detection network model, specifically, "the preset detection network model includes: a feature extraction sub-model, a bounding box location extraction sub-model, a first pooling sub-model, and a first regression sub-model." The process of pedestrian detection is described in detail using this predictive detection network model. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0044] See Figure 2 The pedestrian detection method provided in this embodiment specifically includes the following steps:

[0045] S210. Obtain the target image to be detected.

[0046] S220. Input the target image into the feature extraction sub-model for feature extraction to obtain the target feature map corresponding to the target image.

[0047] The feature extraction sub-model can consist of convolutional layers, activation layers, and pooling layers, and is used to extract feature information from the target image to obtain the target feature map.

[0048] Specifically, Figure 3 An example of a pre-defined detection network model is given. Figure 3 As shown, in the process of pedestrian detection using a preset detection network model, the target image to be detected is first input into the feature extraction sub-model for feature extraction to obtain the target feature map corresponding to the target image.

[0049] S230. Input the target feature map into the bounding box position extraction sub-model to determine the first full-body bounding box position information and the corresponding first visible bounding box position information of the candidate pedestrian.

[0050] The bounding box location extraction sub-model can be used to extract candidate first full-body bounding boxes and first visible bounding boxes that may belong to pedestrians. In this embodiment, the bounding box location extraction sub-model may include two RPNs (Region Proposal Networks), used to extract each candidate first full-body bounding box belonging to a pedestrian and the first visible bounding box located within the first full-body bounding box. The location information of the first full-body bounding box may include the center coordinates of the first full-body bounding box and the width and height of the full-body bounding box. The location information of the first visible bounding box may include the center coordinates of the first visible bounding box and the width and height of the visible bounding box.

[0051] Specifically, such as Figure 3 As shown, the target feature map is input into the bounding box position extraction sub-model to extract the first full-body bounding box and the first visible bounding box, and the position information of each extracted first full-body bounding box and the corresponding first visible bounding box are determined.

[0052] S240. Input the target feature map, the position information of each first full-body bounding box and the corresponding position information of the first visible bounding box into the first pooling sub-model for pooling operation, and determine the first full-body feature map corresponding to each first full-body bounding box and the first visible feature map corresponding to each first visible bounding box.

[0053] The first pooling sub-model can be, but is not limited to, the ROI Pooling region-of-interest pooling model. This first pooling sub-model can perform max pooling on the target feature map based on the position information of each first full-body bounding box to obtain first full-body feature maps of the same size. Similarly, max pooling can be performed on the target feature map based on the position information of each first visible bounding box to obtain first visible feature maps of the same size.

[0054] Specifically, such as Figure 3 As shown, the first pooling sub-model performs pooling operations on the target feature map based on the input position information of each first full-body bounding box and the corresponding position information of the first visible bounding box, thereby determining the first full-body feature map corresponding to each first full-body bounding box and the first visible feature map corresponding to each first visible bounding box.

[0055] S250. Input each first full-body feature map and first visible feature map into the first regression sub-model, determine the second full-body bounding box and the corresponding second visible bounding box belonging to the pedestrian category, as well as the second full-body bounding box position information and the second visible bounding box position information corresponding to the second visible bounding box, and determine the preset distribution parameter value corresponding to the pedestrian based on each second full-body bounding box position information and the corresponding second visible bounding box position information, and output the preset distribution parameter value.

[0056] The first regression sub-model can be used to predict the confidence score for the pedestrian category, that is, to predict the probability value of the full-body bounding box and the corresponding visible bounding box belonging to the pedestrian category, and to regress to obtain more accurate second full-body bounding box and second visible bounding box position information belonging to the pedestrian category. For example, for each first full-body feature map and the corresponding first visible feature map, it can be determined whether each first full-body bounding box and the corresponding first visible bounding box belongs to the pedestrian category, and the first full-body bounding box and the corresponding first visible bounding box that simultaneously belong to the pedestrian category are used as the second full-body bounding box and the corresponding second visible bounding box, thereby ensuring the accuracy of category detection and avoiding classification errors. At the same time, the position of the second full-body bounding box and the corresponding second visible bounding box is corrected using position offsets to obtain more accurate second full-body bounding box and second visible bounding box position information. For each second full-body bounding box position information and the corresponding second visible bounding box position information, the preset distribution parameter value corresponding to the pedestrian is determined.

[0057] Specifically, such as Figure 3As shown, each first full-body feature map and first visible feature map are input into the first regression sub-model to perform regression with preset distribution parameters, and the preset distribution parameter values ​​corresponding to each pedestrian are output.

[0058] For example, when representing a pedestrian using a parameterized two-dimensional Beta distribution, step S250, "determining the preset distribution parameter values ​​corresponding to the pedestrian based on the position information of each second full-body frame and the corresponding position information of the second visible frame," may include: determining the four boundary parameter values ​​of the pedestrian's full-body frame (up, down, left, and right) based on the position information of the second full-body frame, and determining the four boundary parameter values ​​of the pedestrian's visible frame (up, down, left, and right) based on the corresponding position information of the second visible frame; determining the α and β parameters corresponding to the X-axis based on the left and right boundary parameter values ​​of the full-body frame and the left and right boundary parameter values ​​of the visible frame; and determining the α and β parameters corresponding to the Y-axis based on the top and bottom boundary parameter values ​​of the full-body frame and the top and bottom boundary parameter values ​​of the visible frame.

[0059] The second full-body bounding box position information may include the center coordinates of the second full-body bounding box, as well as its width and height. The second visible bounding box position information may include the center coordinates of the second visible bounding box, as well as its width and height. The top, bottom, left, and right boundary parameter values ​​of the pedestrian's full-body bounding box can be obtained using the bottom left corner coordinates (l... f , t f ) and the coordinates of the upper right corner (r) f b f The full-body bounding box is characterized by its left boundary parameter being l. f The right boundary parameter is r f The lower boundary parameter is t f The upper boundary parameter is b f The four boundary parameters of the visible box (top, bottom, left, and right) can also be obtained using the coordinates of the bottom left corner of the visible box (l). v , t v ) and the coordinates of the upper right corner (r) v b v By characterizing the bounding box, we can see that the left boundary parameter is l. v The right boundary parameter is r v The lower boundary parameter is t v The upper boundary parameter is b v .

[0060] Specifically, based on the transformation relationship between boundary parameters and center position coordinates, the position information of the second full-body bounding box can be converted into the top, bottom, left, and right boundary parameter values ​​of the pedestrian's full-body bounding box, and the position information of the second visible bounding box can be converted into the top, bottom, left, and right boundary parameter values ​​of the pedestrian's visible bounding box. On the X-axis, α corresponding to the X-axis can be determined based on the left and right boundary parameter values ​​of the full-body bounding box and the left and right boundary parameter values ​​of the visible bounding box.x and β x On the Y-axis, α can be determined based on the top and bottom boundary parameter values ​​of the full-body bounding box and the top and bottom boundary parameter values ​​of the visible bounding box. y and β y .

[0061] For example, determining the α and β parameters corresponding to the X-axis based on the left and right boundary parameter values ​​of the full-body bounding box and the visible bounding box may include: determining a first weight value corresponding to each pixel point that varies along the X-axis based on the left and right boundary parameter values ​​of the full-body bounding box and the visible bounding box; determining the mean and variance corresponding to the X-axis based on the first weight value corresponding to each pixel point and the left and right boundary parameter values ​​of the full-body bounding box; and determining the α and β parameters corresponding to the X-axis based on the mean and variance corresponding to the X-axis and the left and right boundary parameter values ​​of the full-body bounding box.

[0062] Specifically, this embodiment assigns different first weight values ​​to the visible and invisible regions, allowing the mean and variance of the distribution to replace the peaks and full width at half maximum (FWHM), thus more conveniently determining the α and β parameters corresponding to the X-axis and improving detection efficiency. For example, the first weight value corresponding to the X-value of each pixel in the visible region corresponding to the visible box can be determined as a first value W. v And determine the first weight value corresponding to the X value of each pixel in the invisible region as the second value W. f And the first value W v Greater than the second value W f For example, W v Equal to 1, W f It equals 0.04. This embodiment can determine the first weight value corresponding to each pixel point x that varies along the X-axis based on the following first weight function f(x):

[0063]

[0064] This embodiment can determine the mean value μ corresponding to the X-axis based on the following formula. x and variance σ x 2 :

[0065]

[0066]

[0067] Normalize the mean and variance corresponding to the X-axis, and determine α corresponding to the X-axis based on the normalized mean and variance. x and β x For example, α corresponding to the X-axis can be determined based on the following formula. x and βx :

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] Where λ is used to maintain α x and β x Constants that are all greater than or equal to 1. This is the normalized mean. ν represents the normalized variance. x These are intermediate parameters.

[0074] For example, determining the α and β parameters corresponding to the Y-axis based on the upper and lower boundary parameter values ​​of the full-body bounding box and the upper and lower boundary parameter values ​​of the visible bounding box may include: determining a second weight value corresponding to each pixel point that varies along the Y-axis based on the upper and lower boundary parameter values ​​of the full-body bounding box and the upper and lower boundary parameter values ​​of the visible bounding box; determining the mean and variance corresponding to the Y-axis based on the second weight value corresponding to each pixel point and the upper and lower boundary parameter values ​​of the full-body bounding box; and determining the α and β parameters corresponding to the Y-axis based on the mean and variance corresponding to the Y-axis and the upper and lower boundary parameter values ​​of the full-body bounding box.

[0075] Specifically, similar to determining the α and β parameters corresponding to the X-axis, by assigning different second weight values ​​to the visible and invisible regions, the mean and variance of the distribution can be used instead of the peaks and full width at half maximum (FWHM), making it easier to determine the α and β parameters corresponding to the Y-axis, thereby improving detection efficiency. For example, the second weight value corresponding to the Y value of each pixel in the visible region corresponding to the visible box can be determined as the first value W. v And the second weight value corresponding to the Y value of each pixel in the invisible region is determined as the second value W. f And the first value W v Greater than the second value W f For example, W v Equal to 1, W f It equals 0.04. This embodiment can determine the second weight value corresponding to each pixel point y that varies along the Y value based on the following second weight function f(y):

[0076]

[0077] This embodiment can determine the mean value μ corresponding to the Y-axis based on the following formula. y and variance σ y 2 :

[0078]

[0079]

[0080] Normalize the mean and variance corresponding to the Y-axis, and determine α corresponding to the Y-axis based on the normalized mean and variance. y and β y For example, α corresponding to the Y-axis can be determined based on the following formula. y and β y :

[0081]

[0082]

[0083]

[0084]

[0085]

[0086] Where λ is used to maintain α y and β y Constants that are all greater than or equal to 1. This is the normalized mean. ν represents the normalized variance. y These are intermediate parameters.

[0087] It should be noted that during the training of the preset detection network model, the training of the first regression sub-model can use anchor boxes as parameters to determine the positional offsets between the predicted box and the anchor box, as well as between the ground truth box and the anchor box. For example, based on the normalization method, the positional offsets between the predicted box and the anchor box, and between the ground truth box and the anchor box, can be defined as follows:

[0088] t x =(xx) a ) / ω a t y =(yy) a ) / h a t ω =log(ω / ω) a ),t h =log(h / h) a )

[0089]

[0090] Among them, t x t y t ω and t h This refers to the center position (x, y), width ω, and height h of the prediction box relative to the center position (x, y) of the anchor box. a ,y a Width ω a and height h a The offset between them. and This refers to the center position (x) of the true bounding box. * ,y * Width ω * and height h * Respectively with the center position of the anchor frame (x) a ,y a Width ω a and height h a The offset between them. 'a' represents the numerical identifier corresponding to the pedestrian category.

[0091] Similarly, the mean μ corresponding to the X-axis x and variance σ x 2 and the mean μ corresponding to the Y-axis y and variance σ y 2 The training of these four shape parameters can also utilize anchor boxes as parameters to determine the shape offsets between the predicted bounding box and the anchor box, as well as between the ground truth bounding box and the anchor box. For example, based on normalization, the shape offsets between the predicted bounding box and the anchor box, and between the ground truth bounding box and the anchor box, can be defined as follows:

[0092]

[0093]

[0094] in, and This refers to the mean μ corresponding to the X-axis of the prediction box. x and variance σ x and the mean μ corresponding to the Y-axis y and variance σ y Respectively with the center position of the anchor frame (x) a ,y a Width ω a and height h a The offset between them. and It refers to the mean value corresponding to the X-axis of the true bounding box. and variance and the mean corresponding to the Y-axis and variance Respectively with the center position of the anchor frame (x) a ,y a Width ω a and height h a The offset between them.

[0095] This embodiment can use SmoothL1 as the loss function to regress the distribution parameters. The loss function L... reg It can be represented as follows:

[0096]

[0097] For class prediction, cross-entropy loss can be used as the classification loss function, where L is the loss function. cls It can be represented as follows:

[0098]

[0099] Where, p i This represents the confidence level predicted by the first regression sub-model. This represents the true target value. This embodiment can utilize the loss function L. reg and loss function L cls Train the preset detection network model until both loss functions reach convergence.

[0100] S260. Based on the output of the preset detection network model, determine the pedestrian detection result corresponding to the target image.

[0101] Specifically, the output α for each pedestrian x β x α y and β y This allows us to determine the specific shape of the two-dimensional Beta probability distribution map, thereby obtaining the visibility characteristics of pedestrians. The probability density function P(x,y) of the two-dimensional Beta probability distribution map can be expressed as:

[0102]

[0103] in, For the normalized x, i.e. For the normalized y, that is C is a normalization factor that ensures the sum of the probability values ​​is 1. For a two-dimensional Beta distribution, the probability value of a pixel within the bounding box of a pedestrian is equal to the product of the probability values ​​of two independent one-dimensional Beta distributions. The probability value of a pixel outside the bounding box of a pedestrian is 0.

[0104] The technical solution of this embodiment, by utilizing a preset detection network model including a feature extraction sub-model, a bounding box location extraction sub-model, a first pooling sub-model, and a first regression sub-model, can accurately detect pedestrians, solve the problem of misclassification caused by occlusion, and improve the accuracy of pedestrian detection.

[0105] Based on the above technical solution, after determining the preset distribution parameter value corresponding to the pedestrian based on the position information of each second full-body frame and the corresponding position information of the second visible frame in step S260, the method may further include: determining the two-dimensional probability distribution map corresponding to each pedestrian based on the preset distribution parameter value corresponding to the detected pedestrian; using a symmetrical KL divergence method, determining the pedestrian distance between any two pedestrians based on each two-dimensional probability distribution map; filtering each detected pedestrian based on the pedestrian distance and a preset distance threshold, and outputting the preset distribution parameter value corresponding to the filtered pedestrian.

[0106] Specifically, the first regression sub-model can reconstruct the corresponding two-dimensional probability distribution map based on the preset distribution parameter values ​​corresponding to each detected pedestrian. For example, the two-dimensional Beta probability distribution map corresponding to the pedestrian can be determined based on the α and β parameters corresponding to the X-axis and the α and β parameters corresponding to the Y-axis. In this embodiment, the pedestrian distance between any two pedestrians can be measured using the symmetrical KL divergence based on the two-dimensional probability distribution maps corresponding to any two pedestrians. For example, the pedestrian distance between any two pedestrians is: Among them, D KL (pq) refers to the divergence value of the two-dimensional probability distribution map corresponding to pedestrian p transformed to the two-dimensional probability distribution map corresponding to pedestrian q, D KL (qp) refers to the divergence value of the two-dimensional probability distribution map corresponding to pedestrian q transformed to the two-dimensional probability distribution map corresponding to pedestrian p. If the pedestrian distance is less than or equal to a preset distance threshold, it indicates that the two pedestrians corresponding to these two two-dimensional probability distribution maps are the same pedestrian. In this case, one of the two-dimensional probability distribution maps can be deleted and filtered, thereby identifying highly overlapping pedestrians and further improving the accuracy of pedestrian detection.

[0107] Figure 4 This is a flowchart of a pedestrian detection method provided by an embodiment of the present invention. Based on the above embodiments, this embodiment further optimizes the predictive detection network model, specifically, "the preset detection network model further includes: a second pooling sub-model, a mask processing sub-model, and a second regression sub-model," and the pedestrian detection process is described in detail using this predictive detection network model. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0108] See Figure 4 The pedestrian detection method provided in this embodiment specifically includes the following steps:

[0109] S410. Obtain the target image to be detected.

[0110] S420. Input the target image into the feature extraction sub-model for feature extraction to obtain the target feature map corresponding to the target image.

[0111] S430. Input the target feature map into the bounding box position extraction sub-model to determine the first full-body bounding box position information and the corresponding first visible bounding box position information of the candidate pedestrian.

[0112] S440. Input the target feature map, the position information of each first full-body bounding box and the corresponding position information of the first visible bounding box into the first pooling sub-model for pooling operation, and determine the first full-body feature map corresponding to each first full-body bounding box and the first visible feature map corresponding to each first visible bounding box.

[0113] S450. Input each first full-body feature map and first visible feature map into the first regression sub-model, determine the second full-body bounding box and the corresponding second visible bounding box belonging to the pedestrian category, as well as the second full-body bounding box position information and the second visible bounding box position information, and determine the preset distribution parameter value corresponding to the pedestrian based on each second full-body bounding box position information and the corresponding second visible bounding box position information.

[0114] S460. Input the target feature map, the position information of each second full-body bounding box and the corresponding position information of the second visible bounding box into the second pooling sub-model for pooling operation, and determine the second full-body feature map corresponding to each second full-body bounding box and the second visible feature map corresponding to each second visible bounding box.

[0115] The second pooling sub-model is similar to the first pooling sub-model. The second pooling sub-model can be, but is not limited to, ROI Pooling. The second pooling sub-model can perform max pooling on the target feature map based on the position information of each second full-body bounding box, obtaining second full-body feature maps of the same size. Similarly, max pooling is performed on the target feature map based on the position information of each second visible bounding box, obtaining second visible feature maps of the same size.

[0116] Specifically, Figure 5 An example of a pre-defined detection network model is given. Figure 5 As shown, the target feature map output by the feature extraction sub-model and the position information of each second full-body bounding box and the corresponding position information of the second visible bounding box output by the first regression sub-model can be further input into the second pooling sub-model. Based on the input position information of each second full-body bounding box and the corresponding position information of the second visible bounding box, a pooling operation is performed on the target feature map to determine the second full-body feature map corresponding to each second full-body bounding box and the second visible feature map corresponding to each second visible bounding box.

[0117] S470. Input the preset distribution parameter values ​​corresponding to the pedestrian, each second full-body feature map, and the second visible feature map into the mask processing sub-model. Based on the preset distribution parameter values ​​corresponding to the pedestrian, determine the first mask image corresponding to the second full-body feature map and the second mask image corresponding to the second visible feature map. Then, use the first mask image to perform attention processing on the second full-body feature map to obtain the processed third full-body feature map. Finally, use the second mask image to perform attention processing on the second visible feature map to obtain the processed third visible feature map.

[0118] The masking sub-model can refer to an attention mechanism network model, used to focus attention on the visible area of ​​the pedestrian to further improve detection accuracy. In this embodiment, the probability value of the visible area in the two-dimensional probability distribution map is higher, and the probability value of the invisible area is lower. Therefore, the two-dimensional probability distribution map can be used as a filter to focus attention on the visible area of ​​the pedestrian. This allows the two-dimensional probability distribution to highlight the features of the visible area while suppressing other noise, enabling the preset detection network model to focus more on discriminative features, achieving more accurate localization and higher reliability. The masking sub-model uses the first mask image to perform attention processing on the second full-body feature map, highlighting the features of the visible area within the full-body bounding box. Using the second mask image to perform attention processing on the second visible feature map makes the features of the visible area more obvious, further improving detection accuracy.

[0119] Specifically, such as Figure 5 As shown, the masking sub-model can perform masking attention processing on the preset distribution parameter values ​​of pedestrians output by the first regression sub-model and the various second full-body feature maps and second visible feature maps output by the second pooling sub-model to obtain the processed third full-body feature map and third visible feature map.

[0120] For example, step S470, "determining the first mask image corresponding to the second full-body feature map and the second mask image corresponding to the second visible feature map based on the preset distribution parameter values ​​corresponding to the pedestrian," may include: determining a corresponding two-dimensional probability distribution map based on the preset distribution parameter values ​​corresponding to the pedestrian; sampling the two-dimensional probability distribution map based on the second full-body feature map to determine the first pixel value corresponding to each pixel in the second full-body feature map, and determining the first mask image based on each first pixel value; sampling the two-dimensional probability distribution map based on the second visible feature map to determine the second pixel value corresponding to each pixel in the second visible feature map, and determining the second mask image based on each second pixel value.

[0121] Specifically, when representing pedestrians using a parameterized two-dimensional Beta distribution, the two-dimensional Beta probability distribution map corresponding to the pedestrian can be determined based on the α and β parameters corresponding to the X-axis and the α and β parameters corresponding to the Y-axis, expressed as: Be(x,y; α... x ,β x ,α y ,β y )=C·Be(x;α x ,β x )·Be(y;α y ,β y Where C is a normalization factor that ensures the sum of probability values ​​is 1. Sampling can be performed from the two-dimensional Beta probability distribution map based on the second full-body feature map. The probability value corresponding to each pixel in the second full-body feature map is used as the first pixel value, i.e., the weight value corresponding to the pixel. A first mask image is generated based on each first pixel value, thus obtaining a first mask image with the same size as the second full-body feature map. Similarly, sampling can be performed from the two-dimensional Beta probability distribution map based on the second visible feature map. The probability value corresponding to each pixel in the second visible feature map is used as the second pixel value, i.e., the weight value corresponding to the pixel. A second mask image is generated based on each second pixel value, thus obtaining a second mask image with the same size as the second visible feature map.

[0122] For example, using the first mask image to perform attention processing on the second full-body feature map to obtain the processed third full-body feature map may include: multiplying the pixel values ​​corresponding to the same pixel point in the first mask image and the second full-body feature map, and using the multiplication result as the third full-body feature map.

[0123] Specifically, the first mask image Be and the second whole-body feature map Z can be multiplied at the same pixel, and the result can be used as the third whole-body feature map Z′, i.e. Z′=Z·Be. This can enhance the features of the visible area and suppress the features of other areas, thereby further improving the accuracy of detection.

[0124] For example, similarly, using the second mask image to perform attention processing on the second visible feature map to obtain the processed third visible feature map may include: multiplying the pixel values ​​corresponding to the same pixel point in the second mask image and the second visible feature map, and using the multiplication result as the third visible feature map, thereby further highlighting the features of the visible area.

[0125] It should be noted that during the training of the prediction and detection network model, the masking sub-model can be trained using KL divergence as the loss function. mask It can be represented as:

[0126] Lmask =∑Be * (x,y)(logBe * (x,y)-logBe(x,y))

[0127] Among them, Be * (x,y) can refer to the actual mask value corresponding to the pixel; Be(x,y) can refer to the mask value corresponding to the pixel in the determined mask image.

[0128] S480. Input the third full-body feature map and the third visible feature map into the second regression sub-model, determine the third full-body bounding box and the corresponding third visible bounding box belonging to the pedestrian's category, as well as the position information of the third full-body bounding box and the position information of the third visible bounding box corresponding to the third full-body bounding box. Based on each third full-body bounding box position information and the corresponding third visible bounding box position information, redetermine the preset distribution parameter value corresponding to the pedestrian, and output the redetermined preset distribution parameter value.

[0129] The second regression sub-model can be used to predict the confidence score for the pedestrian category, that is, to predict the probability value of the full-body bounding box and the corresponding visible bounding box belonging to the pedestrian category, and to regress to obtain more accurate third full-body bounding box and third visible bounding box location information belonging to the pedestrian category. For example, for each third full-body feature map and the corresponding third visible feature map, it can be determined whether each second full-body bounding box and the corresponding second visible bounding box belongs to the pedestrian category, and the second full-body bounding box and the corresponding second visible bounding box that simultaneously belong to the pedestrian category are used as the third full-body bounding box and the corresponding third visible bounding box, thereby further ensuring the accuracy of category detection and avoiding classification errors. At the same time, the position of the third full-body bounding box and the corresponding third visible bounding box is corrected using position offsets to obtain more accurate third full-body bounding box and third visible bounding box location information. For each third full-body bounding box location information and the corresponding third visible bounding box location information, the preset distribution parameter value corresponding to the pedestrian can be re-determined.

[0130] In this embodiment, the method for determining the preset distribution parameter values ​​based on the third full-body frame position information and the corresponding third visible frame position information in the second regression sub-model is the same as the method for determining the preset distribution parameter values ​​based on the second full-body frame position information and the corresponding second visible frame position information in the first regression sub-model. Refer to the relevant descriptions in the above embodiments; they will not be repeated here. For example, when using a parameterized two-dimensional Beta distribution to represent a pedestrian, step S480, "determining the preset distribution parameter values ​​corresponding to the pedestrian based on each third full-body frame position information and the corresponding third visible frame position information," may include: determining the four boundary parameter values ​​of the pedestrian's full-body frame (up, down, left, right) based on the third full-body frame position information, and determining the four boundary parameter values ​​of the pedestrian's visible frame (up, down, left, right) based on the corresponding third visible frame position information; determining the α and β parameters corresponding to the X-axis based on the left and right boundary parameter values ​​of the full-body frame and the left and right boundary parameter values ​​of the visible frame; and determining the α and β parameters corresponding to the Y-axis based on the top and bottom boundary parameter values ​​of the full-body frame and the top and bottom boundary parameter values ​​of the visible frame.

[0131] S490. Based on the output of the preset detection network model, determine the pedestrian detection result corresponding to the target image.

[0132] The technical solution of this embodiment utilizes a preset detection network model, including a feature extraction sub-model, a bounding box location extraction sub-model, a first pooling sub-model, a first regression sub-model, a second pooling sub-model, a mask processing sub-model, and a second regression sub-model, to perform pedestrian detection, which can further improve the accuracy of pedestrian detection.

[0133] The following are embodiments of the pedestrian detection device provided in this invention. This device and the pedestrian detection methods in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the pedestrian detection device, please refer to the embodiments of the above pedestrian detection methods.

[0134] Figure 6 This is a schematic diagram of a pedestrian detection device provided in an embodiment of the present invention. This embodiment is applicable to detecting pedestrians contained in an image, especially in crowded pedestrian scenes with occlusions. Figure 6 As shown, the device specifically includes: a target image acquisition module 610, a target image input module 620, and a pedestrian detection result determination module 630.

[0135] The target image acquisition module 610 is used to acquire the target image to be detected; the target image input module 620 is used to input the target image into a preset detection network model, which is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map, the probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian, and the probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian; the pedestrian detection result determination module 630 is used to determine the pedestrian detection result corresponding to the target image based on the output of the preset detection network model.

[0136] Optionally, the preset distribution parameter value for each pedestrian refers to the value of the preset parameter corresponding to the two-dimensional Beta distribution. The preset parameters corresponding to the two-dimensional Beta distribution include: the α and β parameters corresponding to the X-axis, and the α and β parameters corresponding to the Y-axis.

[0137] Optionally, the preset detection network model includes: a feature extraction sub-model, a bounding box location extraction sub-model, a first pooling sub-model, and a first regression sub-model;

[0138] Accordingly, the target image input module 620 includes:

[0139] The target image input unit is used to input the target image into the feature extraction sub-model for feature extraction and to obtain the target feature map corresponding to the target image.

[0140] The target feature map input unit is used to input the target feature map into the bounding box position extraction sub-model to determine the first full-body bounding box position information and the corresponding first visible bounding box position information of the candidate pedestrian;

[0141] The first position information input unit is used to input the target feature map, the position information of each first full-body bounding box and the corresponding first visible bounding box position information into the first pooling sub-model for pooling operation, and to determine the first full-body feature map corresponding to each first full-body bounding box and the first visible feature map corresponding to each first visible bounding box.

[0142] The first feature map input unit is used to input each first full-body feature map and the first visible feature map into the first regression sub-model, determine the second full-body bounding box and the corresponding second visible bounding box belonging to the pedestrian category, as well as the second full-body bounding box position information and the second visible bounding box position information, and determine the preset distribution parameter value corresponding to the pedestrian based on each second full-body bounding box position information and the corresponding second visible bounding box position information, and output the preset distribution parameter value.

[0143] Optionally, the first regression sub-model is specifically used to: determine the four boundary parameter values ​​of the pedestrian's full-body bounding box (up, down, left, and right) based on the second full-body bounding box position information, and determine the four boundary parameter values ​​of the pedestrian's visible bounding box (up, down, left, and right) based on the corresponding second visible bounding box position information; determine the α and β parameters corresponding to the X-axis based on the two boundary parameter values ​​of the full-body bounding box and the two boundary parameter values ​​of the visible bounding box; and determine the α and β parameters corresponding to the Y-axis based on the two boundary parameter values ​​of the full-body bounding box and the two boundary parameter values ​​of the visible bounding box.

[0144] Optionally, the first regression sub-model is specifically used to: determine the first weight value corresponding to each pixel point that varies along the X-axis based on the left and right boundary parameter values ​​of the whole-body bounding box and the left and right boundary parameter values ​​of the visible box; determine the mean and variance corresponding to the X-axis based on the first weight value corresponding to each pixel point and the left and right boundary parameter values ​​of the whole-body bounding box; and determine the α parameter and β parameter corresponding to the X-axis based on the mean and variance corresponding to the X-axis and the left and right boundary parameter values ​​of the whole-body bounding box.

[0145] Optionally, the first regression sub-model is further specifically used for: after determining the preset distribution parameter value corresponding to each pedestrian based on the position information of each second full-body bounding box and the corresponding position information of the second visible bounding box, determining the two-dimensional probability distribution map corresponding to each pedestrian based on the preset distribution parameter value corresponding to the detected pedestrian; using a symmetrical KL divergence method, determining the pedestrian distance between any two pedestrians based on each two-dimensional probability distribution map; filtering each detected pedestrian based on the pedestrian distance and a preset distance threshold, and outputting the preset distribution parameter value corresponding to the filtered pedestrian.

[0146] Optionally, the preset detection network model also includes: a second pooling sub-model, a masking sub-model, and a second regression sub-model;

[0147] Accordingly, the target image input module 620 also includes:

[0148] The second position information input unit is used to input the target feature map, the position information of each second full-body bounding box and the corresponding position information of the second visible bounding box into the second pooling sub-model for pooling operation, and to determine the second full-body feature map corresponding to each second full-body bounding box and the second visible feature map corresponding to the second visible bounding box.

[0149] The second feature map input unit is used to input the preset distribution parameter values ​​corresponding to the pedestrian, each second full-body feature map, and the second visible feature map into the mask processing sub-model. Based on the preset distribution parameter values ​​corresponding to the pedestrian, it determines the first mask image corresponding to the second full-body feature map and the second mask image corresponding to the second visible feature map. It then uses the first mask image to perform attention processing on the second full-body feature map to obtain the processed third full-body feature map, and uses the second mask image to perform attention processing on the second visible feature map to obtain the processed third visible feature map.

[0150] The third feature map input unit is used to input the third full-body feature map and the third visible feature map into the second regression sub-model, determine the third full-body bounding box and the corresponding third visible bounding box belonging to the pedestrian's category, as well as the position information of the third full-body bounding box and the position information of the third visible bounding box corresponding to the third full-body bounding box. Based on each third full-body bounding box position information and the corresponding third visible bounding box position information, the preset distribution parameter value corresponding to the pedestrian is re-determined, and the re-determined preset distribution parameter value is output.

[0151] Optionally, the mask processing sub-model is specifically used for: determining the corresponding two-dimensional probability distribution map based on the preset distribution parameter values ​​corresponding to the pedestrian; sampling the two-dimensional probability distribution map based on the second full-body feature map to determine the first pixel value corresponding to each pixel in the second full-body feature map, and determining the first mask image based on each first pixel value; sampling the two-dimensional probability distribution map based on the second visible feature map to determine the second pixel value corresponding to each pixel in the second visible feature map, and determining the second mask image based on each second pixel value.

[0152] Optionally, the mask processing sub-model is further specifically used to: multiply the pixel values ​​corresponding to the same pixel point in the first mask image and the second whole-body feature map, and use the multiplication result as the third whole-body feature map.

[0153] The pedestrian detection device provided in this embodiment of the invention can execute the pedestrian detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the pedestrian detection method.

[0154] It is worth noting that in the above embodiments of the pedestrian detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0155] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. Figure 7 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 7 The electronic device 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0156] like Figure 7 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0157] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0158] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0159] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 7 Not shown; usually referred to as a "hard drive"). Although Figure 7 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0160] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0161] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0162] Processing unit 16 executes various functional applications and data processing by running programs stored in system memory 28, such as implementing the steps of a pedestrian detection method provided in this embodiment, the method including:

[0163] Acquire the target image to be detected;

[0164] The target image is input into a preset detection network model. The preset detection network model is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. The probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian. The probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian.

[0165] Based on the output of the preset detection network model, the pedestrian detection result corresponding to the target image is determined.

[0166] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the pedestrian detection method provided in any embodiment of the present invention.

[0167] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements the pedestrian detection method steps provided in any embodiment of the present invention, the method comprising:

[0168] Acquire the target image to be detected;

[0169] The target image is input into a preset detection network model. The preset detection network model is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. The probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian. The probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian.

[0170] Based on the output of the preset detection network model, the pedestrian detection result corresponding to the target image is determined.

[0171] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0172] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0173] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0174] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0175] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0176] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A pedestrian detection method, characterized in that, include: Acquire the target image to be detected; The target image is input into a preset detection network model, which is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. The probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian. The probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian. Based on the output of the preset detection network model, the pedestrian detection result corresponding to the target image is determined; The preset distribution parameter value for each pedestrian refers to the value of the preset parameter corresponding to the two-dimensional Beta distribution. The preset parameter corresponding to the two-dimensional Beta distribution includes: the α parameter and β parameter corresponding to the X-axis, and the α parameter and β parameter corresponding to the Y-axis. The preset detection network model includes: a feature extraction sub-model, a bounding box location extraction sub-model, a first pooling sub-model, and a first regression sub-model; The step of inputting the target image into a preset detection network model includes: The target image is input into the feature extraction sub-model for feature extraction to obtain the target feature map corresponding to the target image; The target feature map is input into the bounding box position extraction sub-model to determine the first full-body bounding box position information and the corresponding first visible bounding box position information for the candidate pedestrian; The target feature map, the position information of each first full-body bounding box and the corresponding position information of the first visible bounding box are input into the first pooling sub-model for pooling operation to determine the first full-body feature map corresponding to each first full-body bounding box and the first visible feature map corresponding to each first visible bounding box. Each of the first full-body feature maps and the first visible feature maps is input into the first regression sub-model to determine the second full-body bounding box and the corresponding second visible bounding box belonging to the pedestrian category, as well as the second full-body bounding box position information and the second visible bounding box position information corresponding to the second visible bounding box. Based on each second full-body bounding box position information and the corresponding second visible bounding box position information, the preset distribution parameter value corresponding to the pedestrian is determined, and the preset distribution parameter value is output.

2. The method according to claim 1, characterized in that, The step of determining the preset distribution parameter value corresponding to the pedestrian based on the position information of each second full-body frame and the corresponding position information of the second visible frame includes: Based on the second full-body frame position information, determine the top, bottom, left, and right boundary parameter values ​​of the pedestrian's full-body frame, and based on the corresponding second visible frame position information, determine the top, bottom, left, and right boundary parameter values ​​of the pedestrian's visible frame. Based on the left and right boundary parameter values ​​of the whole body frame and the left and right boundary parameter values ​​of the visible frame, determine the α and β parameters corresponding to the X-axis; Based on the upper and lower boundary parameter values ​​of the whole-body frame and the upper and lower boundary parameter values ​​of the visible frame, the α and β parameters corresponding to the Y-axis are determined.

3. The method according to claim 2, characterized in that, The determination of the α and β parameters corresponding to the X-axis based on the left and right boundary parameter values ​​of the whole-body bounding box and the left and right boundary parameter values ​​of the visible box includes: Based on the left and right boundary parameter values ​​of the whole body frame and the left and right boundary parameter values ​​of the visible frame, determine the first weight value corresponding to each pixel point that changes along the X-axis. Based on the first weight value corresponding to each pixel and the left and right boundary parameter values ​​of the whole body frame, the mean and variance corresponding to the X-axis are determined. Based on the mean and variance corresponding to the X-axis and the left and right boundary parameter values ​​of the whole body frame, the α parameter and β parameter corresponding to the X-axis are determined.

4. The method according to claim 1, characterized in that, After determining the preset distribution parameter value corresponding to the pedestrian based on the position information of each second full-body frame and the corresponding position information of the second visible frame, the method further includes: Based on the preset distribution parameter values ​​corresponding to the detected pedestrians, a two-dimensional probability distribution map corresponding to each pedestrian is determined. Using a symmetrical KL divergence method, the pedestrian distance between any two pedestrians is determined based on the aforementioned two-dimensional probability distribution maps; Based on the pedestrian distance and the preset distance threshold, each detected pedestrian is filtered, and the preset distribution parameter value corresponding to the filtered pedestrian is output.

5. The method according to any one of claims 1-4, characterized in that, The preset detection network model also includes: a second pooling sub-model, a masking sub-model, and a second regression sub-model; The step of outputting the preset distribution parameter value includes: The target feature map, the position information of each second full-body bounding box and the corresponding position information of the second visible bounding box are input into the second pooling sub-model for pooling operation to determine the second full-body feature map corresponding to each second full-body bounding box and the second visible feature map corresponding to each visible bounding box. The preset distribution parameter values ​​corresponding to the pedestrian, each of the second full-body feature maps and the second visible feature map are input into the mask processing sub-model. Based on the preset distribution parameter values ​​corresponding to the pedestrian, the first mask image corresponding to the second full-body feature map and the second mask image corresponding to the second visible feature map are determined. Attention processing is performed on the second full-body feature map using the first mask image to obtain the processed third full-body feature map. Attention processing is also performed on the second visible feature map using the second mask image to obtain the processed third visible feature map. The third full-body feature map and the third visible feature map are input into the second regression sub-model to determine the third full-body bounding box and the corresponding third visible bounding box belonging to the pedestrian's category, as well as the position information of the third full-body bounding box and the position information of the third visible bounding box. Based on each of the third full-body bounding box position information and the corresponding third visible bounding box position information, the preset distribution parameter value corresponding to the pedestrian is re-determined, and the re-determined preset distribution parameter value is output.

6. The method according to claim 5, characterized in that, The step of determining the first mask image corresponding to the second full-body feature map and the second mask image corresponding to the second visible feature map based on the preset distribution parameter values ​​corresponding to the pedestrian includes: A corresponding two-dimensional probability distribution map is determined based on the preset distribution parameter values ​​corresponding to the pedestrians; Based on the second full-body feature map, the two-dimensional probability distribution map is sampled to determine the first pixel value corresponding to each pixel point in the second full-body feature map, and a first mask image is determined based on each first pixel value; The two-dimensional probability distribution map is sampled based on the second visible feature map to determine the second pixel value corresponding to each pixel point in the second visible feature map, and a second mask image is determined based on each second pixel value.

7. The method according to claim 5, characterized in that, The step of performing attention processing on the second full-body feature map using the first mask image to obtain the processed third full-body feature map includes: The pixel values ​​corresponding to the same pixel point in the first mask image and the second whole-body feature map are multiplied together, and the result of the multiplication is used as the third whole-body feature map.

8. A pedestrian detection device, characterized in that, include: The target image acquisition module is used to acquire the target image to be detected. The target image input module is used to input the target image into a preset detection network model. The preset detection network model is used to determine the preset distribution parameter value corresponding to each pedestrian in the target image. The preset distribution parameter value corresponds to a two-dimensional probability distribution map. The probability distribution boundary of the two-dimensional probability distribution map is the full-body bounding box of the corresponding pedestrian. The probability peak of the two-dimensional probability distribution map is located at the center of the visible box of the corresponding pedestrian. The pedestrian detection result determination module is used to determine the pedestrian detection result corresponding to the target image based on the output of the preset detection network model; The preset distribution parameter value for each pedestrian refers to the value of the preset parameter corresponding to the two-dimensional Beta distribution. The preset parameter corresponding to the two-dimensional Beta distribution includes: the α parameter and β parameter corresponding to the X-axis, and the α parameter and β parameter corresponding to the Y-axis. The preset detection network model includes: a feature extraction sub-model, a bounding box position extraction sub-model, a first pooling sub-model, and a first regression sub-model; the target image input module includes a target image input unit, a target feature map input unit, a first position information input unit, and a first feature map input unit; the target image input unit is used to input the target image into the feature extraction sub-model for feature extraction to obtain the target feature map corresponding to the target image; the target feature map input unit is used to input the target feature map into the bounding box position extraction sub-model to determine the first full-body bounding box position information and the corresponding first visible bounding box position information corresponding to the candidate pedestrian; the first position information input unit is used to input the target feature map, each of the first... The full-body bounding box position information and the corresponding first visible bounding box position information are input into the first pooling sub-model for pooling operation to determine the first full-body feature map and the first visible feature map corresponding to each first full-body bounding box; the first feature map input unit is used to input each first full-body feature map and the first visible feature map into the first regression sub-model to determine the second full-body bounding box and the corresponding second visible bounding box belonging to the pedestrian category, as well as the second full-body bounding box position information and the second visible bounding box position information corresponding to the second visible bounding box, and to determine the preset distribution parameter value corresponding to the pedestrian based on each second full-body bounding box position information and the corresponding second visible bounding box position information, and to output the preset distribution parameter value.

9. An electronic device, characterized in that, The electronic device includes: one or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the pedestrian detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the pedestrian detection method as described in any one of claims 1-7.

Images