Target detection method, device and equipment based on Gaussian mixture model

Abstract

The application provides a target detection method, device and equipment based on a Gaussian mixture model, which can be applied to the field of computer vision and the field of digital image processing. The target detection method based on the Gaussian mixture model comprises: processing a target image comprising a target to be detected to obtain a first density map related to the target to be detected, wherein the first density map conforms to a Gaussian mixture distribution, and the first density map serves as a Gaussian mixture model; performing integral processing on the first density map to obtain a first quantity value of the target to be detected; performing local peak value extraction on the first density map to obtain a second quantity value of the target to be detected; obtaining a target mean value and a target variance of the target to be detected based on the first quantity value, the second quantity value and the first density map; and determining position information of the target to be detected according to the target mean value and the target variance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and digital image processing, and more particularly, to a target detection method, device and equipment based on a mixture Gaussian model. BACKGROUND

[0002] In the field of computer vision and digital image processing, target detection is one of the important research directions. Target detection is widely applied to important fields such as intelligent monitoring, intelligent industrial detection, national security and aerospace. As the core part of the intelligent monitoring algorithm, the target detection algorithm is also the basic task of many computer vision detection tasks, and plays an important role in the subsequent secondary tasks, such as behavior recognition, face recognition, target counting and fine-grained recognition.

[0003] In the process of implementing the present application concept, the inventors have found that the existing target detection algorithm has at least the following problems: the existing target detection algorithm has the problems of missed detection and false detection for small targets in an image. SUMMARY

[0004] Therefore, the present application provides a target detection method, device and equipment based on a mixture Gaussian model.

[0005] One aspect of the present application provides a target detection method based on a mixture Gaussian model, comprising:

[0006] processing a target image including a target to be detected to obtain a first density map related to the target to be detected, wherein the first density map conforms to a mixture Gaussian distribution, and the first density map serves as the mixture Gaussian model;

[0007] integrating the first density map to obtain a first quantity value of the target to be detected;

[0008] extracting a local peak value of the first density map to obtain a second quantity value of the target to be detected;

[0009] obtaining a target mean and a target variance of the target to be detected based on the first quantity value, the second quantity value and the first density map;

[0010] determining position information of the target to be detected according to the target mean and the target variance.

[0011] According to the embodiment of the present application, the integrating the first density map to obtain the first quantity value of the target to be detected comprises:

[0012] filtering the first density map to obtain a second density map;

[0013] Integrating the second density map to obtain a first quantity value of the target to be detected.

[0014] According to an embodiment of the present application, the local peak extraction on the first density map to obtain a second quantity value of the target to be detected comprises:

[0015] The local peak extraction on the second density map to obtain a second quantity value of the target to be detected.

[0016] According to an embodiment of the present application, the obtaining of the target mean and target variance of the target to be detected based on the first quantity value, the second quantity value and the first density map comprises:

[0017] The first mean and first variance of the target to be detected are obtained based on the first quantity value and the second density map;

[0018] In the case that the first variance meets the preset condition, the first mean is taken as the target mean and the first variance is taken as the target variance;

[0019] In the case that the first variance does not meet the preset condition, the target mean and target variance of the target are obtained based on the second quantity value and the second density map.

[0020] According to an embodiment of the present application, the obtaining of the first mean and first variance of the target to be detected based on the first quantity value and the second density map comprises:

[0021] A first data group with a quantity of the first quantity value is randomly extracted from the second density map;

[0022] The first data is processed by using a mixed Gaussian model analysis algorithm to obtain the first mean and the first variance.

[0023] According to an embodiment of the present application, the obtaining of the target mean and target variance of the small target to be detected based on the second quantity value and the second density map in the case that the first variance does not meet the preset condition comprises:

[0024] A second data group with a quantity of the second quantity value is randomly extracted from the second density map;

[0025] The second data is processed by using a mixed Gaussian model analysis algorithm to obtain the target mean and the target variance.

[0026] According to an embodiment of the present application, the filtering of the first density map to obtain a second density map comprises:

[0027] For the pixel value at each pixel point of the first density map, the pixel value at the pixel point less than the preset value is set to zero to obtain a second density map.

[0028] According to the embodiment of the present application, the processing of the target image including the target to be detected to obtain the first density map related to the target to be detected comprises:

[0029] The target image is feature-extracted by using a deep convolutional network to obtain a target feature map;

[0030] The target feature map is up-sampled to obtain a high-resolution feature map;

[0031] The high-resolution feature map is resolution-reconstructed by using a resolution reconstruction network to obtain the first density map.

[0032] Another aspect of the present application provides a target detection device based on a Gaussian mixture model, comprising:

[0033] A first density map obtaining module is configured to process a target image including a target to be detected to obtain a first density map related to the target to be detected, wherein the first density map conforms to a Gaussian mixture distribution, and the first density map serves as the Gaussian mixture model;

[0034] A first quantity value obtaining module is configured to integrate the first density map to obtain a first quantity value of the target to be detected;

[0035] A second quantity value obtaining module is configured to extract a local peak value of the first density map to obtain a second quantity value of the target to be detected;

[0036] A target mean and variance obtaining module is configured to obtain a target mean and a target variance of the target to be detected based on the first quantity value, the second quantity value and the first density map;

[0037] A position information determining module is configured to determine position information of the target to be detected according to the target mean and the target variance.

[0038] Another aspect of the present application provides an electronic device, comprising:

[0039] One or more processors;

[0040] A memory configured to store one or more instructions,

[0041] When the one or more instructions are executed by the one or more processors, the one or more processors implement the method as described above.

[0042] According to the embodiment of the present application, by processing the target image including the target to be detected, a first density map related to the target to be detected is obtained, wherein the first density map conforms to a mixed Gaussian distribution, the first density map is taken as a mixed Gaussian model, a mixed Gaussian model related to the target to be detected is obtained, then the first density map is integrated to obtain a first quantity value of the target to be detected, the first quantity value of the target to be detected is obtained according to the pixel-level feature information related to the target to be detected, the local peak value of the first density map is extracted to obtain a second quantity value of the target to be detected, the second quantity value of the target to be detected is obtained according to the regional feature information related to the target to be detected, then the target mean and the target variance of the target to be detected can be obtained based on the first quantity value, the second quantity value and the first density map, and the position information of the target to be detected is determined according to the target mean and the target variance, the mixed Gaussian model related to the target to be detected is analyzed by using the first quantity value of the target to be detected obtained according to the pixel-level feature information related to the target to be detected and the second quantity value of the target to be detected obtained according to the regional feature information related to the target to be detected, the target mean and the target variance of the target to be detected are obtained, and the position information of the target to be detected is determined according to the target mean and the target variance, the recognition precision of the position information of the target to be detected is improved, the recognition rate of the small target is improved, and the false detection rate and the missed detection rate caused by using one kind of feature information are effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0043] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application taken in conjunction with the accompanying drawings, in which:

[0044] Figure 1 A scene diagram of application of the target detection method and device based on the mixed Gaussian model according to the embodiment of the present application is schematically shown;

[0045] Figure 2 A flowchart of the target detection method based on the mixed Gaussian model according to the embodiment of the present application is schematically shown;

[0046] Figure 3 A structure block diagram of the target detection device based on the mixed Gaussian model according to the embodiment of the present application is schematically shown; and

[0047] Figure 4 A block diagram of the electronic device suitable for the above target detection method based on the mixed Gaussian model according to the embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0048] Embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood, however, that the description that follows is merely exemplary and is not intended to limit the scope of the application. In the following detailed description of embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that one or more embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring the concepts of the present application.

[0049] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "includes" and tautological expressions thereof, such as "including," "includes," "include," "contains," "containing," and so forth, mean "comprising."

[0050] All terms used herein including technical and scientific terms have the same meanings as commonly understood by one of ordinary skill in the art unless otherwise defined herein. It should be noted that the terms used herein are merely specific examples for the described embodiments and should not be limiting to the present application.

[0051] In the case where expressions similar to "at least one of A, B, and C, etc." are used, it generally should be interpreted to include any of one, all, or a combination thereof. In the case where expressions similar to "at least one of A, B, or C, etc." are used, it generally should be interpreted to include any of one, all, or a combination thereof. In the case where expressions similar to "at least one of A, B, and C, etc." are used, it generally should be interpreted to include any of one, all, or a combination thereof. In the case where expressions similar to "at least one of A, B, or C, etc." are used, it generally should be interpreted to include any of one, all, or a combination thereof.

[0052] Based on the existing target detection algorithm, there are missing detection and false detection problems in detecting small targets in an image, and the present application provides a target detection method, device and equipment based on a mixed Gaussian model.

[0053] Embodiments of the present application provide a target detection method based on a mixed Gaussian model, comprising:

[0054] The target image including the target to be detected is processed to obtain a first density map related to the target to be detected, wherein the first density map conforms to a Gaussian mixture distribution and serves as a Gaussian mixture model; the first density map is integrated to obtain a first quantity value of the target to be detected; local peak extraction is performed on the first density map to obtain a second quantity value of the target to be detected; based on the first quantity value, the second quantity value, and the first density map, the target mean and target variance of the target to be detected are obtained; the location information of the target to be detected is determined based on the target mean and target variance.

[0055] According to an embodiment of the present invention, by processing a target image including a target to be detected, a first density map related to the target to be detected is obtained. The first density map conforms to a Gaussian mixture distribution. Using the first density map as a Gaussian mixture model, a Gaussian mixture model related to the target to be detected is obtained. Then, the first density map is integrated to obtain a first quantity value of the target to be detected, thus realizing the determination of the first quantity value of the target to be detected based on pixel-level feature information related to the target. Local peak extraction is performed on the first density map to obtain a second quantity value of the target to be detected, thus realizing the determination of the second quantity value of the target to be detected based on regional feature information related to the target. Furthermore, the first quantity value, the second quantity value, and the first... The density map is used to obtain the target mean and target variance of the target to be detected. The location information of the target to be detected is determined based on the target mean and target variance. This is achieved by using the first quantitative value of the target to be detected, obtained from the pixel-level feature information related to the target to be detected, and the second quantitative value of the target to be detected, obtained from the regional feature information related to the target to be detected, to analyze the Gaussian mixture model related to the target to be detected, obtain the target mean and target variance of the target to be detected, and determine the location information of the target to be detected based on the target mean and target variance. This improves the recognition accuracy of the location information of the target to be detected, thereby improving the recognition rate of small targets and effectively reducing the false detection and false negative rates caused by relying on only one type of feature information.

[0056] Figure 1 The illustration shows an application scenario of the target detection method and apparatus based on the Gaussian mixture model according to an embodiment of the present invention.

[0057] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0058] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0059] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0060] The server 105 can be a server providing various services, such as a background management server providing support for websites browsed by the user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0061] It should be noted that the target detection method based on the Gaussian mixture model provided by the embodiments of the present application can generally be executed by the server 105. Correspondingly, the target detection device based on the Gaussian mixture model provided by the embodiments of the present application can generally be arranged in the server 105. The target detection method based on the Gaussian mixture model provided by the embodiments of the present application can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Correspondingly, the target detection device based on the Gaussian mixture model provided by the embodiments of the present application can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0062] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above description is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks, and servers.

[0063] Figure 2 A flowchart of the target detection method based on the Gaussian mixture model according to the embodiments of the present application is schematically shown.

[0064] As Figure 2 shown, the method includes operations S201-S205.

[0065] In operation S201, a target image including a target to be detected is processed to obtain a first density map related to the target to be detected, where the first density map conforms to a mixed Gaussian distribution, and the first density map serves as a mixed Gaussian model.

[0066] According to an embodiment of the present application, the target to be detected can be a small target, and the size of the small target can be selected according to the number of pixels of the small target in the target image. For example, the size of the small target can be less than 10 pixels (px), 20 px, or 30 px in length and width, or less than 10 px in length and width and less than 2 in aspect ratio, or less than 20 px in length and width and less than 1.5 in aspect ratio, or less than 30 px in length and width and less than 3 in aspect ratio. Embodiments of the present application do not limit the specific size of the small target, and the size of the small target can be selected according to actual conditions.

[0067] According to an embodiment of the present application, the target image can include a plurality of targets to be detected, and the number of targets to be detected can be 2, 20, 200, etc. Embodiments of the present application do not limit the number of targets to be detected, and the number of targets to be detected can be selected according to actual conditions.

[0068] According to an embodiment of the present application, for each target to be detected in the plurality of targets to be detected, the first density map includes data corresponding to each target to be detected, respectively, and the data corresponding to each target to be detected respectively conforms to a Gaussian distribution. Therefore, the first density map conforms to a mixed Gaussian distribution, and the first density map can serve as a mixed Gaussian model.

[0069] According to an embodiment of the present application, a related resolution recovery algorithm can be used to perform resolution recovery on the target image, so that the resolution of the target image meets the requirements of a preset target image resolution. Then, a related feature extraction algorithm can be used to perform feature extraction on the target image that meets the requirements of the preset target image resolution to obtain a feature map. Finally, a convolutional neural network can be used to process the feature map to obtain the first density map.

[0070] According to an embodiment of the present application, the preset target image resolution can be 200 px, 500 px, or 1000 px, etc. The related resolution recovery algorithm can be an interpolation method (nearest neighbor interpolation, bilinear interpolation, etc.) in an upsampling method, transpose convolution (also known as deconvolution), and de-pooling method, etc. Embodiments of the present application do not limit the preset target image resolution and the related resolution recovery algorithm, and they can be selected according to actual conditions.

[0071] According to an embodiment of the present application, the relevant feature extraction algorithm may, for example, be AlexNet or Visual Geometry Group (VGG16 or VGG19) in a deep convolutional neural network model, and the present application is not limited to the feature extraction algorithm, which may be selected according to actual conditions.

[0072] In operation S202, the first density map is integrated to obtain a first quantity value of the target to be detected.

[0073] According to an embodiment of the present application, the first density map is integrated to obtain a first quantity value of the target to be detected, so that the first quantity value of the target to be detected is obtained according to the pixel-level feature information related to the target to be detected.

[0074] In operation S203, the first density map is subjected to local peak extraction to obtain a second quantity value of the target to be detected.

[0075] The first density map is subjected to local peak extraction to obtain a second quantity value of the target to be detected, so that the second quantity value of the target to be detected is obtained according to the regional feature information related to the target to be detected.

[0076] According to an embodiment of the present application, the first quantity value and the second quantity value of the target to be detected are both the quantity of the target to be detected included in the target image, only the way of obtaining the first quantity value and the second quantity value is different.

[0077] In operation S204, based on the first quantity value, the second quantity value and the first density map, a target mean value and a target variance of the target to be detected are obtained.

[0078] According to an embodiment of the present application, since the first density map includes information corresponding to a plurality of targets to be detected, and the first density map is a mixture Gaussian model, the quantity of the target to be detected can be solved by analyzing the mixture Gaussian model.

[0079] According to an embodiment of the present application, in the process of analyzing the mixture Gaussian model, a data space for analyzing the mixture Gaussian model needs to be determined, the data in the data space can be extracted from the first density map, and the quantity of the extracted data can be determined according to the first quantity value and the second quantity value. In the case of obtaining the data space, the mean value and the variance of the mixture Gaussian model can be solved by using a suitable mixture Gaussian model analysis algorithm according to the data space, the mean value of the mixture Gaussian model is the target mean value of the target to be detected, and the variance of the mixture Gaussian model is the target variance of the target to be detected.

[0080] In operation S205, the position information of the target to be detected is determined according to the target mean value and the target variance.

[0081] According to an embodiment of the present application, the position information of the target to be detected comprises a center point coordinate and a bounding box information of the target to be detected. The target mean value of the target to be detected corresponds to the center point coordinate of the target to be detected, and the variance of the target to be detected corresponds to the bounding box information of the target to be detected.

[0082] According to the target detection method based on the Gaussian mixture model provided by the embodiment of the present application, the first quantity value of the target to be detected is obtained according to the pixel-level feature information related to the target to be detected, the second quantity value of the target to be detected is obtained according to the regional feature information related to the target to be detected, then the Gaussian mixture model related to the target to be detected is analyzed by using the first quantity value of the target to be detected obtained according to the pixel-level feature information related to the target to be detected and the second quantity value of the target to be detected obtained according to the regional feature information related to the target to be detected, the target mean value and the target variance of the target to be detected are obtained, and the position information of the target to be detected is determined according to the target mean value and the target variance, thereby improving the recognition accuracy of the position information of the target to be detected, improving the recognition rate of the small target, and effectively reducing the false detection rate and the missed detection rate caused by using one kind of feature information.

[0083] According to the embodiment of the present application, for S202 as shown in the figure, Figure 2 The operation of integrating the first density map to obtain the first quantity value of the target to be detected can include the following operations:

[0084] Filtering the first density map to obtain a second density map;

[0085] Integrating the second density map to obtain the first quantity value of the target to be detected.

[0086] According to the embodiment of the present application, the position of each pixel point of the first density map is a probability value, and the first density map is a probability distribution density map. The first density map is filtered to obtain a second density map, and the second density map is also a probability distribution density map.

[0087] According to the embodiment of the present application, the first density map can be filtered by using a threshold filtering method to obtain a second density map.

[0088] According to the embodiment of the present application, filtering the first density map to obtain a second density map includes:

[0089] For the pixel value at each pixel point of the first density map, the pixel value at the pixel point less than the preset value is set to zero to obtain a second density map.

[0090] According to an embodiment of the present application, the threshold filtering of the first density map is implemented by setting the pixel value of each pixel point in the first density map to zero if the pixel value is less than a preset value, so that the second density map only includes the probability value meeting the preset value.

[0091] According to an embodiment of the present application, the specific operation of filtering the first density map by using the threshold filtering method can be represented by formula (1).

[0092] D'(x, y) = {D(x, y) | D(x, y) > T} (1)

[0093] Wherein, D(x, y) represents the first density map, D'(x, y) is the second density map after filtering, and T is the preset value.

[0094] According to an embodiment of the present application, the preset value T can be 10 -2 , 10 -3 , 10 -4 or 10 -5 . The embodiment of the present application does not limit the preset value T, which can be selected according to the actual situation.

[0095] According to an embodiment of the present application, in the process of constructing the first density map generation network training data set, each target to be detected in the target image is blurred by a Gaussian convolution kernel to a Gaussian distribution with a probability sum of 1. Therefore, after the first density map generation network is trained and the first density map is obtained by using the first density map generation network, the first density map can be directly filtered to obtain the second density map, and then the integral operation is performed on the second density map, so that the number of the target to be detected in the current second density map can be directly obtained.

[0096] According to an embodiment of the present application, the specific operation of performing the integral processing on the second density map to obtain the first number value of the target to be detected can be represented by formula (2).

[0097] M I = {(x, y) ∈ D'(x, y) | I(D'(x, y))} (2)

[0098] Wherein, I represents the integral operation on the second density map D'(x, y), and M I represents the first number value of the target to be detected obtained by performing the integral processing on the second density map.

[0099] According to an embodiment of the present application, the first number value of the target to be detected is obtained by performing the integral processing on the second density map, so that the first number value of the target to be detected is obtained based on the pixel-level feature information related to the target to be detected.

[0100] According to an embodiment of the present application, for example,Figure 2 Operation S203, as shown, extracts local peaks from the first density map to obtain a second quantity value of the target to be detected, and may include the following operations:

[0101] Local peaks are extracted from the second density map to obtain the second quantity value of the target to be detected.

[0102] According to an embodiment of the present invention, the data corresponding to each of the multiple targets to be detected in the second density map conforms to a Gaussian distribution. The probability distribution of each Gaussian distribution is characterized by a high probability at the middle point and a decreasing probability towards the surrounding area. Therefore, each Gaussian distribution can be regarded as a whole. Local peaks are extracted from the second density map through a local peak extraction algorithm, and the number of local peaks obtained is the second quantity value of the target to be detected.

[0103] According to an embodiment of the present invention, the specific operation of extracting local peaks from the second density map to obtain the second quantity value of the target to be detected can be represented by formula (3).

[0104] M P ={(x,y)∈D′(x,y)|Peak(D′(x,y))} (3)

[0105] Where Peak represents the local peak extraction algorithm for extracting local peaks from the second density map D′(x,y), and M P The second quantity value of the target to be detected is obtained by extracting local peaks from the second density map.

[0106] According to an embodiment of the present invention, the local peak extraction algorithm can not only obtain the number of targets to be detected, but also the coordinates of the center point of the targets to be detected.

[0107] According to an embodiment of the present invention, local peak extraction is performed on the second density map to obtain a second quantity value of the target to be detected, thereby realizing the second quantity value of the target to be detected based on regional feature information related to the target to be detected.

[0108] According to embodiments of the present invention, for example, Figure 2 Operation S204, as shown, obtains the target mean and target variance of the target to be detected based on the first quantity value, the second quantity value, and the first density map. It may include the following operations:

[0109] Based on the first quantity value and the second density map, the first mean and the first variance of the target to be detected are obtained;

[0110] If the first variance meets the preset conditions, the first mean is taken as the target mean and the first variance is taken as the target variance.

[0111] In a case where the first variance does not meet the preset condition, a target mean and a target variance of the target are obtained based on the second quantity value and the second density map.

[0112] According to an embodiment of the present application, the first variance corresponds to the bounding box information of the target to be detected, and the bounding box information of the target to be detected can be the actual length of the length and width of the target to be detected or the pixel number of the length and width of the target to be detected in the target image. The preset condition can be, for example, that the length and the width are both less than 10px, 20px or 30px, that the length and the width are both less than 10px and the aspect ratio is less than 2, that the length and the width are both less than 20px and the aspect ratio is less than 1.5, or that the length and the width are both less than 30px and the aspect ratio is less than 3.

[0113] According to an embodiment of the present application, in a case where the first variance does not meet the preset condition, it is indicated that the deviation between the first quantity value and the quantity value of the target to be detected is large. Therefore, the target mean and the target variance of the target are obtained based on the second quantity value and the second density map, so as to improve the calculation accuracy of the target mean and the target variance. Since the position information of the target to be detected is determined based on the target mean and the target variance, the recognition accuracy of the position information of the target to be detected can be improved subsequently.

[0114] According to an embodiment of the present application, the first mean and the first variance of the target to be detected are obtained based on the first quantity value and the second density map, including:

[0115] A first data group with a quantity of the first quantity value is randomly extracted from the second density map.

[0116] The first data is processed by using a Gaussian mixture model analysis algorithm to obtain the first mean and the first variance.

[0117] According to an embodiment of the present application, the Gaussian mixture model analysis algorithm includes a Gaussian mixture model-Expectation-Maximization algorithm (GMM-EM).

[0118] According to an embodiment of the present application, the second density map meets the Gaussian mixture model, a first data group with a quantity of the first quantity value is randomly extracted from the second density map, and the first data represents a data space for analyzing the Gaussian mixture model, wherein each data in the first data is greater than 0.

[0119] According to an embodiment of the present application, the probability of each data in the data space constituted by the first data is not 0, and each data includes feature information. The data space constituted by the first data can be represented by formula (4).

[0120] Ω={(x n , y nD'(x i , y i ) > 0} (4)

[0121] wherein i = 0, 1, 2, 3...n, j = 0, 1, 2, 3...n, (x n , y n ) represents the pixel coordinate position of each data in the first data in the second density map, Ω represents the data space formed by the first data, D'(x i , y j ) represents the probability value corresponding to the pixel coordinate position (x i , y j ) in the second density map.

[0122] According to the embodiment of the present application, in the case that the first variance does not meet the preset condition, the target mean and the target variance of the small target to be detected are obtained based on the second quantity value and the second density map, comprising:

[0123] randomly extracting a group of second data with a second quantity value from the second density map;

[0124] processing the second data by using a mixed Gaussian model analysis algorithm to obtain the target mean and the target variance.

[0125] According to the embodiment of the present application, the mixed Gaussian model analysis algorithm comprises GMM-EM.

[0126] According to the embodiment of the present application, the second density map meets the mixed Gaussian model, a group of second data with a second quantity value is randomly extracted from the second density map, and the second data represents the data space used for analyzing the mixed Gaussian model, wherein each data in the second data is greater than 0.

[0127] According to the embodiment of the present application, the probability of each data in the data space formed by the second data is not 0, and each data comprises feature information. The data space formed by the second data can also be represented by formula (4), wherein (x n , y n ) represents the pixel coordinate position of each data in the second data in the second density map, Ω represents the data space formed by the second data, D'(x i , y j ) represents the probability value corresponding to the pixel coordinate position (x i , y j ) in the second density map.

[0128] According to the embodiment of the present application, the feature information represented by the data at all pixel coordinate positions in the first data or the second data covers complete feature information of the entire Gaussian mixture distribution. Therefore, when the first data or the second data is obtained, the first data or the second data can be processed by using a Gaussian mixture model solving algorithm to obtain the target mean and the target variance.

[0129] According to the embodiment of the present application, for the operation S201 as shown in the figure, Figure 2 The operation of processing the target image including the target to be detected to obtain the first density map related to the target to be detected can include the following operations.

[0130] The target image is subjected to feature extraction by using a deep convolutional network to obtain a target feature map.

[0131] The target feature map is subjected to up-sampling to obtain a high-resolution feature map.

[0132] The high-resolution feature map is subjected to resolution reconstruction by using a resolution reconstruction network to obtain the first density map.

[0133] According to the embodiment of the present application, the target image can be subjected to feature extraction by using the VGG19 network with less parameters and high precision to obtain the target feature map.

[0134] According to the embodiment of the present application, the target feature map can be subjected to up-sampling by using the bilinear interpolation algorithm to obtain the high-resolution feature map.

[0135] According to the embodiment of the present application, the resolution reconstruction network includes three layers, and each layer includes a convolutional layer and an activation layer.

[0136] According to the embodiment of the present application, the resolution reconstruction network includes three layers, and each layer includes a convolutional layer and an activation layer, which can reduce the parameters of the resolution reconstruction network.

[0137] According to the embodiment of the present application, the high-resolution feature map is subjected to resolution reconstruction by using the resolution reconstruction network to obtain the first density map, which can realize probability distribution fine-tuning of the high-resolution feature map, so that the first density map converges to the label density map.

[0138] According to the embodiment of the present application, the target image is subjected to feature extraction by using a deep convolutional network to obtain a target feature map, the target feature map is subjected to upsampling to obtain a high-resolution feature map, and finally, the high-resolution feature map is subjected to resolution reconstruction by using a resolution reconstruction network to obtain a first density map, so that a pixel-level probability distribution density map is obtained, the first density map has high precision, and the information in the first density map is used to solve the number of the to-be-detected target included in the first density map, so that the high-precision first density map can indirectly improve the precision of detecting the number of the to-be-detected target.

[0139] It should be noted that, unless it is explicitly stated that there is an execution sequence between different operations or there is an execution sequence in the technical implementation of different operations, the execution sequence of multiple operations can not be distinguished, and multiple operations can be executed simultaneously.

[0140] Figure 3 The structure block diagram of the target detection device based on the Gaussian mixture model according to the embodiment of the present application is schematically shown.

[0141] As shown in Figure 3 The target detection device based on the Gaussian mixture model 300 includes a first density map obtaining module 310, a first number value obtaining module 320, a second number value obtaining module 330, a target mean and variance obtaining module 340, and a position information determining module 350.

[0142] The first density map obtaining module 310 is configured to process a target image including a to-be-detected target to obtain a first density map related to the to-be-detected target, wherein the first density map conforms to a Gaussian mixture distribution, and the first density map is used as a Gaussian mixture model.

[0143] The first number value obtaining module 320 is configured to perform integral processing on the first density map to obtain a first number value of the to-be-detected target.

[0144] The second number value obtaining module 330 is configured to perform local peak value extraction on the first density map to obtain a second number value of the to-be-detected target.

[0145] The target mean and variance obtaining module 340 is configured to obtain a target mean and a target variance of the to-be-detected target based on the first number value, the second number value, and the first density map.

[0146] The position information determining module 350 is configured to determine position information of the to-be-detected target according to the target mean and the target variance.

[0147] According to the embodiment of the present application, the first number value obtaining module includes a second density map obtaining submodule and a first number value obtaining submodule.

[0148] The second density map obtaining submodule is configured to filter the first density map to obtain a second density map.

[0149] The first quantity value obtaining submodule is configured to perform integral processing on the second density map to obtain a first quantity value of the target to be detected.

[0150] According to an embodiment of the present application, the second quantity value obtaining module comprises a second quantity value obtaining submodule.

[0151] The second quantity value obtaining submodule is configured to perform local peak value extraction on the second density map to obtain a second quantity value of the target to be detected.

[0152] According to an embodiment of the present application, the target mean and variance obtaining module comprises a first mean and variance obtaining submodule, a target mean and variance first obtaining submodule, and a target mean and variance second obtaining submodule.

[0153] The first mean and variance obtaining submodule is configured to obtain a first mean and a first variance of the target to be detected based on the first quantity value and the second density map.

[0154] The target mean and variance first obtaining submodule is configured to, in a case where the first variance meets a preset condition, take the first mean as a target mean and take the first variance as a target variance.

[0155] The target mean and variance second obtaining submodule is configured to, in a case where the first variance does not meet the preset condition, obtain a target mean and a target variance of the target based on the second quantity value and the second density map.

[0156] According to an embodiment of the present application, the first mean and variance obtaining submodule comprises a first data extraction unit and a first mean and variance obtaining unit.

[0157] The first data extraction unit is configured to randomly extract a group of first data with a quantity of the first quantity value from the second density map.

[0158] The first mean and variance obtaining unit is configured to process the first data by using a Gaussian mixture model analysis algorithm to obtain the first mean and the first variance.

[0159] The Gaussian mixture model analysis algorithm comprises a maximum expectation algorithm of a Gaussian mixture model.

[0160] According to an embodiment of the present application, the target mean and variance second obtaining submodule comprises a second data extraction unit and a target mean and variance obtaining unit.

[0161] The second data extraction unit is configured to randomly extract a group of second data with a quantity of the second quantity value from the second density map.

[0162] The target mean and variance obtaining unit is configured to process the second data by using a Gaussian mixture model analysis algorithm to obtain a target mean and a target variance.

[0163] The Gaussian mixture model analysis algorithm comprises a Gaussian mixture model expectation maximization algorithm.

[0164] According to an embodiment of the present application, the second density map obtaining submodule comprises a second density map obtaining unit.

[0165] The second density map obtaining unit is configured to set a pixel value of a pixel point less than a preset value to zero to obtain a second density map.

[0166] The first density map obtaining module comprises a target feature map obtaining submodule, a high-resolution feature map obtaining submodule, and a first density map obtaining submodule.

[0167] The target feature map obtaining submodule is configured to perform feature extraction on the target image by using a deep convolutional network to obtain a target feature map.

[0168] The high-resolution feature map obtaining submodule is configured to perform up-sampling on the target feature map to obtain a high-resolution feature map.

[0169] The first density map obtaining submodule is configured to perform resolution reconstruction on the high-resolution feature map by using a resolution reconstruction network to obtain a first density map, wherein the resolution reconstruction network comprises three layers, and each layer comprises a convolutional layer and an activation layer.

[0170] Any one or more of the modules, submodules, units, and sub-units according to the embodiments of the present application, or at least part of the functions of any one or more of the modules, submodules, units, and sub-units, can be implemented in one module. Any one or more of the modules, submodules, units, and sub-units according to the embodiments of the present application can be split into multiple modules for implementation. Any one or more of the modules, submodules, units, and sub-units according to the embodiments of the present application can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or any other reasonable hardware or firmware that can be integrated or packaged with a circuit, or implemented in any one of software, hardware, and firmware or in an appropriate combination of any one or more of the three. Alternatively, one or more of the modules, submodules, units, and sub-units according to the embodiments of the present application can be at least partially implemented as computer program modules that can perform corresponding functions when the computer program modules are run.

[0171] For example, any of the first density map obtaining module 310, the first quantity value obtaining module 320, the second quantity value obtaining module 330, the target mean and variance obtaining module 340, and the position information determining module 350 can be combined in one module / unit / sub-unit, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the function of one or more of the modules / units / sub-units can be combined with at least part of the function of other modules / units / sub-units, and implemented in one module / unit / sub-unit.

[0172] According to an embodiment of the present application, at least one of the first density map obtaining module 310, the first quantity value obtaining module 320, the second quantity value obtaining module 330, the target mean and variance obtaining module 340, and the position information determining module 350 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, etc. in hardware or firmware, or in any one of the three implementation ways of software, hardware and firmware, or in a proper combination of any of the three. Alternatively, at least one of the first density map obtaining module 310, the first quantity value obtaining module 320, the second quantity value obtaining module 330, the target mean and variance obtaining module 340, and the position information determining module 350 can be at least partially implemented as a computer program module which can perform the corresponding function when the computer program module is run.

[0173] It should be noted that the target detection device based on the Gaussian mixture model in the embodiments of the present application corresponds to the target detection method based on the Gaussian mixture model in the embodiments of the present application, and will not be described here.

[0174] Figure 4 A block diagram of an electronic device suitable for the above-mentioned target detection method based on the Gaussian mixture model according to an embodiment of the present application is schematically shown. Figure 4 The electronic device shown is only an example and should not limit the function and use range of the embodiments of the present application.

[0175] As Figure 4As shown, the electronic device 400 according to an embodiment of the present application includes a processor 401 which can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 402 or a program loaded into a random access memory (RAM) 403 from a storage section 408. The processor 401 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 401 can also include an on-board memory for cache use. The processor 401 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present application.

[0176] In the RAM 403, various programs and data required for the operation of the electronic device 400 are stored. The processor 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. The processor 401 performs various operations of the method processes according to embodiments of the present application by executing the programs in the ROM 402 and / or the RAM 403. Note that the programs can also be stored in one or more memories other than the ROM 402 and the RAM 403. The processor 401 can also perform various operations of the method processes according to embodiments of the present application by executing the programs stored in the one or more memories.

[0177] According to an embodiment of the present application, the electronic device 400 can further include an input / output (I / O) interface 405 which is also connected to the bus 404. The system 400 can further include one or more of the following components connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as necessary. A removable recording medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 410 as necessary, so that a computer program read out therefrom is installed in the storage section 408 as necessary.

[0178] According to an embodiment of the present application, the method flow according to the embodiments of the present application can be implemented as a computer software program. For example, the embodiments of the present application include a computer program product comprising a computer program carrying computer program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 409, and / or installed from the detachable medium 411. When the computer program is executed by the processor 401, the above-described functions defined in the system / apparatus according to the embodiments of the present application are performed. According to an embodiment of the present application, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0179] The present application also provides a computer readable storage medium, which can be included in the apparatus / system / device described in the above embodiments, or exist separately without being assembled into the apparatus / system / device. The above computer readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present application.

[0180] According to an embodiment of the present application, the computer readable storage medium can be a non-volatile computer readable storage medium. For example, it can include but is not limited to a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0181] For example, according to an embodiment of the present application, the computer readable storage medium can include one or more memories of the ROM 402 and / or the RAM 403 described above and / or one or more memories other than the ROM 402 and the RAM 403.

[0182] The embodiments of the present application also include a computer program product comprising a computer program containing program code for executing the method provided by the embodiments of the present application, which program code, when the computer program product is run on an electronic device, causes the electronic device to implement the target detection method based on a Gaussian mixture model provided by the embodiments of the present application.

[0183] When the computer program is executed by the processor 401, the above-described functions defined in the system / apparatus according to the embodiments of the present application are performed. According to an embodiment of the present application, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0184] In one embodiment, the computer program can be tangibly embodied in a non-transitory machine-readable storage medium. In another embodiment, the computer program can be tangibly embodied in a signal that can be downloaded and installed by a communication interface 409, and / or installed from a removable medium 411. The computer program can include program code that can be transmitted over a network, including, but not limited to, wireless, wired, or any suitable combination thereof, or any suitable combination of the foregoing. The computer program can be downloaded and installed by a communication interface 409, and / or installed from a removable medium 411.

[0185] According to an embodiment of the present application, program code that implements the application embodiments provided by the present application can be written in any combination of one or more programming languages, including a high-level procedural and / or object-oriented programming language, and / or an assembly / machine language. Program code can execute entirely on a user's computing device, partly on the user's device, as a stand-alone software package, partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0186] The computer program product of the present application can be a computer program product comprising a computer readable storage medium and a computer program mechanism embedded in the computer readable storage medium. Such computer program product can further include a computer readable storage medium and program means for causing a processor or other programmable processing apparatus to function in a particular manner, such that the computer program mechanism that can be executed by such a processor or processing apparatus cause the processor or processing apparatus to implement the functions of the system, method and computer program product of the present application. The computer program product can be intended for use in one or more systems or a processor-controlled device and can cause the system or device to carry out a predetermined activity, thereby effectively transforming the computer program product and processor or processing apparatus into a particular machine.

[0187] The above-described embodiments of the application have been described in order to allow a clear and consistent understanding of the application. Subsequently, it will be apparent to those skilled in the art that various modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the present application various modifications and variations of the present application can be made. It is also to be understood that the present application can be used in various applications. Any such modifications and variations, as well as others, will be considered to fall within the scope of the present application as defined by the appended claims.

Claims

1. A target detection method based on a Gaussian mixture model, comprising: The target image including the target to be detected is processed to obtain a first density map related to the target to be detected, wherein the first density map conforms to a Gaussian mixture distribution. The first density map is integrated to obtain the first quantity value of the target to be detected; Local peak extraction is performed on the first density map to obtain a second quantity value of the target to be detected; A set of first data with a quantity equal to the first quantity value is randomly selected from the second density map, wherein the second density map is obtained by filtering the first density map; The first data is processed using a Gaussian mixture model analytical algorithm to obtain the first mean and the first variance; If the first variance meets the preset conditions, the first mean is taken as the target mean and the first variance is taken as the target variance. If the first variance does not meet the preset conditions, the target mean and target variance of the target to be detected are obtained based on the second quantity value and the second density map. The location information of the target to be detected is determined based on the target mean and the target variance.

2. The method according to claim 1, wherein, The step of integrating the first density map to obtain the first quantity value of the target to be detected includes: The first density map is filtered to obtain the second density map; The second density map is integrated to obtain the first quantity value of the target to be detected.

3. The method according to claim 2, wherein, The step of extracting local peak values ​​from the first density map to obtain the second quantity value of the target to be detected includes: Local peak extraction is performed on the second density map to obtain the second quantity value of the target to be detected.

4. The method according to claim 1, wherein, When the first variance does not meet the preset conditions, obtaining the target mean and target variance of the target to be detected based on the second quantity value and the second density map includes: A set of second data with a quantity equal to the second quantity value is randomly selected from the second density map; The second data is processed using a Gaussian mixture model analytical algorithm to obtain the target mean and the target variance.

5. The method according to claim 2, wherein, The step of filtering the first density map to obtain the second density map includes: For each pixel value in the first density map, the pixel values ​​of the pixels that are less than a preset value are set to zero to obtain the second density map.

6. The method according to claim 1, wherein, The step of processing the target image including the target to be detected to obtain a first density map related to the target to be detected includes: The target image is used to extract features using a deep convolutional network to obtain a target feature map; The target feature map is upsampled to obtain a high-resolution feature map; The high-resolution feature map is reconstructed using a resolution reconstruction network to obtain the first density map.

7. A target detection device based on a Gaussian mixture model, comprising: The first density map obtaining module is used to process a target image including the target to be detected to obtain a first density map related to the target to be detected, wherein the first density map conforms to a Gaussian mixture distribution; The first quantity value acquisition module is used to perform integral processing on the first density map to obtain the first quantity value of the target to be detected; The second quantity value acquisition module is used to extract local peaks from the first density map to obtain the second quantity value of the target to be detected. The target mean and variance acquisition module is used to obtain the target mean and target variance of the target to be detected based on the first quantity value, the second quantity value and the first density map; The target mean and variance acquisition module includes a first mean and variance acquisition submodule, a first target mean and variance acquisition submodule, and a second target mean and variance acquisition submodule; The first mean and variance acquisition submodule is used to obtain the first mean and first variance of the target to be detected based on the first quantity value and the second density map, wherein the second density map is obtained by filtering the first density map; The first submodule for obtaining the target mean and variance is used to take the first mean as the target mean and the first variance as the target variance when the first variance meets the preset conditions. The second submodule for obtaining the target mean and variance is used to obtain the target mean and target variance of the target to be detected based on the second quantity value and the second density map when the first variance does not meet the preset conditions. The first mean and variance acquisition submodule includes a first data extraction unit and a first mean and variance acquisition unit; The first data extraction unit is used to randomly extract a set of first data with a first quantity value from the second density map; The first mean and variance obtaining unit is used to process the first data using a Gaussian mixture model analytical algorithm to obtain the first mean and the first variance; The location information determination module is used to determine the location information of the target to be detected based on the target mean and the target variance.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more instructions. When the one or more instructions are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 6.

Images