Multi-feature fusion pedestrian detection method, device, equipment and medium

Abstract

The present invention relates to the field of image processing technology, and in particular to a multi-feature fused pedestrian detection method, device, equipment and medium. The method comprises: based on a predetermined discretized triangular wave function, generating a first triangular wave signal function for feature extraction of an image block with an odd width and a second triangular wave signal function for feature extraction of an image block with an even width; obtaining a set size of a target image and an image block; wherein the image block is used to divide the target image into a number of small blocks; based on the set size of the image block, determining a corresponding triangular wave signal function, so as to extract features of the target image using the triangular wave signal function to obtain target features; after fusing the target features with pre-extracted Haar features, inputting the target features into a classifier for pedestrian detection. This scheme can overcome the disadvantage of the discontinuity of Haar features, effectively improve the feature value of the fused features, and improve the accuracy of pedestrian detection.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular to a multi-feature fusion pedestrian detection method, device, equipment and medium. Background Art

[0002] In the fields of intelligent video surveillance, intelligent transportation, smart home defect detection, etc., pedestrians are often detected and identified. Usually, some features can be used for detection and identification, such as the well-known HARR (Haar) feature, HOG (histogram of gradients) feature, LBP (local binary pattern) feature, and spectral features based on Cosine transform, etc.

[0003] Although deep learning can automatically learn features after its rise, many experiments have shown that adding traditional features can still improve the performance and robustness of deep learning methods. In addition, for all occasions with low computing resources and high power consumption requirements, the use of traditional features still has its application scenarios.

[0004] Most traditional pedestrian detection methods use Haar (HARR) features for feature extraction, and then input the Haar (HARR) features into a classifier to achieve pedestrian detection.

[0005] However, the Haar feature uses a rectangular wave to modulate the image block and only uses the difference between the sum of pixels in two regions. Therefore, for some continuously changing areas, the traditional pedestrian detection method lacks more efficient features.

[0006] Therefore, a new multi-feature fusion pedestrian detection method is urgently needed. Summary of the invention

[0007] In order to solve the problem that the existing pedestrian detection methods have low detection accuracy due to the lack of more efficient features for continuously changing areas, the embodiments of the present invention provide a multi-feature fusion pedestrian detection method, device, equipment and medium.

[0008] In a first aspect, an embodiment of the present invention provides a multi-feature fusion pedestrian detection method, the method comprising:

[0009] Based on a predetermined discretized triangular wave function, generating a first triangular wave signal function for extracting features from an image block having an odd width and a second triangular wave signal function for extracting features from an image block having an even width;

[0010] Obtaining a target image and a set size of an image block; wherein the image block is used to divide the target image into a plurality of small blocks;

[0011] Based on the set size of the image block, a corresponding triangular wave signal function is determined, so as to extract features of the target image using the triangular wave signal function to obtain target features;

[0012] After the target features are fused with the pre-extracted Haar features, they are input into a classifier for pedestrian detection.

[0013] In a second aspect, an embodiment of the present invention further provides a multi-feature fusion pedestrian detection device, the device comprising:

[0014] A generating unit, configured to generate, based on a predetermined discretized triangular wave function, a first triangular wave signal function for extracting features from an image block having an odd width and a second triangular wave signal function for extracting features from an image block having an even width;

[0015] An acquisition unit, used for acquiring a target image and a set size of an image block; wherein the image block is used for dividing the target image into a plurality of small blocks;

[0016] an extraction unit, configured to determine a corresponding triangular wave signal function based on a set size of the image block, so as to extract features of the target image using the triangular wave signal function to obtain target features;

[0017] The detection unit is used to fuse the target feature with the pre-extracted Haar feature and input it into a classifier for pedestrian detection.

[0018] In a third aspect, an embodiment of the present invention further provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of this specification is implemented.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any embodiment of this specification.

[0020] The embodiments of the present invention provide a multi-feature fused pedestrian detection method, device, equipment and medium. By defining a triangular wave signal function, the target features of the target image are extracted using the triangular wave signal function. Since the triangular wave waveform is continuous, the target features are fused with the pre-extracted Haar features and then input into the classifier. This can overcome the disadvantage of the discontinuity of the Haar features, effectively improve the feature value of the fused features, and improve the accuracy of pedestrian detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. The drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of a multi-feature fusion pedestrian detection method provided by an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of horizontal triangular wave image characteristics provided by an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of longitudinal triangular wave image characteristics provided by an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of a 45-degree triangle wave image characteristic provided by an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of a 135-degree triangle wave image characteristic provided by an embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of triangular waves at three frequencies provided by an embodiment of the present invention;

[0028] Figure 7 is a hardware architecture diagram of a computing device provided by an embodiment of the present invention;

[0029] Figure 8 It is a structural diagram of a multi-feature fusion pedestrian detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] The specific implementation of the above concept is described below.

[0032] Please refer to Figure 1 , an embodiment of the present invention provides a pedestrian detection method based on multi-feature fusion, the method comprising:

[0033] Step 100, based on a predetermined discretized triangular wave function, generating a first triangular wave signal function for extracting features from an image block with an odd width and a second triangular wave signal function for extracting features from an image block with an even width;

[0034] Step 102, obtaining a target image and a set size of an image block; wherein the image block is used to divide the target image into a plurality of small blocks;

[0035] Step 104, based on the set size of the image block, determine the corresponding triangular wave signal function, and use the triangular wave signal function to extract features of the target image to obtain target features;

[0036] Step 106: After the target feature is fused with the pre-extracted Haar feature, it is input into a classifier for pedestrian detection.

[0037] In an embodiment of the present invention, a triangular wave signal function is defined to utilize the triangular wave signal function to extract target features of a target image. Since the triangular wave waveform is continuous, the target features are fused with the pre-extracted Haar features and then input into a classifier. This can overcome the disadvantage of the discontinuity of the Haar features, effectively improve the feature value of the fused features, and improve the accuracy of pedestrian detection.

[0038] For step 100:

[0039] In some implementations, step 100 may include:

[0040] Determine a constant positive basis function to generate a discretized triangular wave function;

[0041] Based on the discretized triangular wave function, a first triangular wave signal function is generated for extracting features from an image block with an odd width.

[0042] In the embodiment of the present invention, the first triangular wave signal function for extracting features from an image block with an odd width is generated in the following manner:

[0043] The basic function that determines the constant positive is:

[0044]

[0045] In the formula, t is the function independent variable;

[0046] Generate a discretized triangular wave function with a period of 2N:

[0047]

[0048] In the formula, n is the sampling point and 2N is the period;

[0049] Generate the first triangular wave signal function for feature extraction of an image block with a height of M and a width of 2*N-1:

[0050]

[0051] Where n is the sampling point, m is the row number of the image block, and N is half of the period.

[0052] In the embodiment of the present invention, the second triangular wave signal function used for feature extraction of an image block with an even width is:

[0053]

[0054] Where n is the sampling point, m is the row number of the image block, and N is half of the period.

[0055] In this embodiment, a first triangular wave signal function for extracting features from image blocks with odd widths and a second triangular wave signal function for extracting features from image blocks with even widths are defined to generate more efficient features for continuously changing areas, thereby improving detection accuracy.

[0056] Regarding step 102 and step 104:

[0057] In feature extraction, in order to reduce computing resource usage and power consumption requirements, the target image needs to be divided into small blocks for feature extraction, namely image blocks. Therefore, in step 102, the set size of the target image and the image block needs to be obtained.

[0058] In some implementations, step 104 may include:

[0059] Based on a set size of the image block, determining whether a width in the set size is an odd number;

[0060] If yes, then obtain the first triangle wave signal function;

[0061] If not, then obtain the second triangle wave signal function;

[0062] The characteristic angle of the triangular wave signal function is determined, so as to use the acquired triangular wave signal function to extract the features of the target image according to the characteristic angle to obtain the target features.

[0063] In this embodiment, the set size of the image block is different, and different triangular wave signal functions need to be selected for feature extraction. In this embodiment, the characteristic angle of the triangular wave signal function can be as follows: Figure 2 The horizontal triangle wave image signal shown can be as follows Figure 3 The vertical triangle wave image signal shown in FIG. 1 can also be Figure 4 and Figure 5The 45-degree feature and the 135-degree triangle wave image signal shown are used to extract the features of the target image and obtain the target features.

[0064] In some implementations, in step 104, “using the triangular wave signal function to extract features from the target image to obtain target features” may include the following steps S1-S8:

[0065] S1, based on a set size of the image block, determining whether a width in the set size is an odd number;

[0066] S2, if yes, obtain the first triangular wave signal function;

[0067] S3, if not, obtaining a second triangular wave signal function;

[0068] S4, defining a frequency characteristic of the triangular wave signal based on the obtained triangular wave signal function;

[0069] S5, determining N' frequencies for performing row transformation on the target image acquired in real time and M' frequencies for performing column transformation on the target image acquired in real time;

[0070] S6, generating triangular wave signals at N' frequencies for row conversion and triangular wave signals at M' frequencies for column conversion based on the triangular wave signal function;

[0071] S7, for each image block of each target image acquired in real time, executing: using triangular wave signals at N' frequencies to perform triangular wave modulation on each row of each image block in the current target image, to obtain initial frequency spectrum features of each image block;

[0072] S8, using triangular wave signals at M' frequencies, triangular wave modulate each column of the initial spectrum feature of each image block, splicing the image spectrum features of each image block after modulation, and obtaining the image spectrum features of the target image as the target feature.

[0073] In this embodiment, step S4 may specifically include:

[0074] Using the obtained triangle wave signal function, a discrete periodic triangle wave signal function is generated:

[0075]

[0076] Take a finite number of cycles to generate a periodic triangular wave signal:

[0077]

[0078] In the formula, 2N is the periodic variable, L is the total number of cycles of the triangular wave signal, m is the dimension of the frequency, and f mrepresents the frequency, k is the cycle number, and n is the sampling point.

[0079] By setting different frequencies f m , we get triangular wave signals h of different frequencies m (n). Figure 6 The window width N is 40, the period variable 2N is 2, and the frequency f m Schematic diagrams of triangular wave waveforms are 1, 2 and 4. It can be seen that the triangular wave signal waveform is continuous. Compared with the Haar feature that uses a rectangular wave to modulate the image block, the rectangular wave waveform is discontinuous. The triangular wave signal waveform of this embodiment is continuous. Then, after the target feature is fused with the pre-extracted Haar feature and input into the classifier, the disadvantage of the discontinuous Haar feature can be overcome, the feature value of the fused feature can be effectively improved, and the accuracy of pedestrian detection can be improved.

[0080] For discrete image signals, the M-dimensional frequency characteristics of the triangular wave signal (m=1,...,M) are defined as:

[0081] q m =Σq(t)h m (t), m=1,...,M

[0082] Where q(t) is the image signal, h m (t) is a periodic triangular wave signal, m is the dimension of frequency, and M is the maximum dimension of frequency.

[0083] In steps S5-S8, each row and column of the image block is modulated in the following manner:

[0084]

[0085] In the formula, is a triangular wave signal at N' frequencies used for row transformation, I is an image block, M is the number of rows of the image block, N is the number of columns of the image block, F' is the initial spectrum feature corresponding to the image block, is a triangular wave signal at M' frequencies used for column transformation, i, j, k, l are variables, and F is the image spectrum feature corresponding to the image block.

[0086] It can be understood that in this embodiment, feature extraction is performed on each image block in the target image to obtain the image spectrum feature corresponding to each image block, and then the image spectrum feature of the target image is integrated and generated as the target feature.

[0087] Regarding step 106:

[0088] In some implementations, step 106 may include:

[0089] The target features are fused with the pre-extracted Haar features and then input into several weak classifiers respectively to obtain the classification results output by each weak classifier;

[0090] Based on the accuracy weight of each weak classifier, the classification results of each weak classifier are weighted and summed to obtain the pedestrian detection result output by the strong classifier; wherein the strong classifier is formed by combining the weak classifiers.

[0091] In this embodiment, a series of target features of the collected image and the fusion features of the Haar features are respectively input into a number of weak classifiers, and these weak classifiers are combined to form a strong classifier. As long as the detection accuracy of each weak classifier is strictly greater than 50%, a strong classifier with high accuracy can be formed. In this embodiment, by using triangular wave signals to extract more efficient target features for continuously changing areas, and then fusing them with Haar features, the disadvantage of discontinuity of Haar features can be overcome, and the detection accuracy of weak classifiers can be effectively improved. If the detection accuracy of each weak classifier is improved a little, a strong classifier with super high accuracy can be formed by weighted summation.

[0092] like Figure 7 , Figure 8 As shown, the embodiment of the present invention provides a pedestrian detection device with multi-feature fusion. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 7 FIG. 1 is a hardware architecture diagram of a computing device in which a multi-feature fusion pedestrian detection device is provided in an embodiment of the present invention. Figure 7 In addition to the processor, memory, network interface, and non-volatile memory shown in the figure, the computing device in which the device is located in the embodiment may also generally include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 8 As shown, as a device in a logical sense, the CPU of the computing device in which it is located reads the corresponding computer program in the non-volatile memory into the memory and runs it. This embodiment provides a multi-feature fusion pedestrian detection device, the device comprising:

[0093] A generating unit 801 is used to generate a first triangular wave signal function for extracting features from an image block with an odd width and a second triangular wave signal function for extracting features from an image block with an even width based on a predetermined discretized triangular wave function;

[0094] The acquisition unit 802 is used to acquire the target image and the set size of the image block; wherein the image block is used to divide the target image into a plurality of small blocks;

[0095] The extraction unit 803 is used to determine the corresponding triangular wave signal function based on the set size of the image block, so as to extract the features of the target image by using the triangular wave signal function to obtain the target features;

[0096] The detection unit 804 is used to fuse the target feature with the pre-extracted Haar feature and input it into the classifier for pedestrian detection.

[0097] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a multi-feature fusion pedestrian detection device. In other embodiments of the present invention, a multi-feature fusion pedestrian detection device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0098] The information interaction, execution process and other contents between the units in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0099] An embodiment of the present invention further provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a multi-feature fusion pedestrian detection method in any embodiment of the present invention is implemented.

[0100] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a multi-feature fusion pedestrian detection method in any embodiment of the present invention.

[0101] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0102] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0103] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0104] It should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0105] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0106] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0107] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian detection method based on multi-feature fusion, It is characterized in that include: Based on a predetermined discretized triangular wave function, generating a first triangular wave signal function for extracting features from an image block having an odd width and a second triangular wave signal function for extracting features from an image block having an even width; Obtaining a target image and a set size of an image block; wherein the image block is used to divide the target image into a plurality of small blocks; Based on the set size of the image block, a corresponding triangular wave signal function is determined, so as to extract features of the target image using the triangular wave signal function to obtain target features; After fusing the target features with the pre-extracted Haar features, the features are input into a classifier for pedestrian detection; Based on a predetermined discretized triangular wave function, generating a first triangular wave signal function for extracting features from an image block with an odd width, comprising: Determine a constant positive basis function to generate a discretized triangular wave function; Based on the discretized triangular wave function, generating a first triangular wave signal function for extracting features from an image block with an odd width; The first triangular wave signal function for feature extraction of image blocks with odd width is generated as follows: The basic function that determines the constant positivity is: In the formula, t is the function independent variable; Generate a discretized triangular wave function with a period of 2N: In the formula, n is the sampling point and 2N is the period; Generate the first triangular wave signal function for feature extraction of an image block with a height of M and a width of 2*N-1: Where n is the sampling point, m is the row number of the image block, and N is half of the period; The second triangular wave signal function used to extract features from image blocks with even width is: Where n is the sampling point, m is the row number of the image block, and N is half of the period.

2. The method according to claim 1, It is characterized in that The determining of the corresponding triangular wave signal function based on the set size of the image block, and using the triangular wave signal function to extract features of the target image to obtain target features, includes: Based on a set size of the image block, determining whether a width in the set size is an odd number; If yes, obtaining the first triangular wave signal function; If not, obtaining the second triangular wave signal function; The characteristic angle of the triangular wave signal function is determined, so as to use the acquired triangular wave signal function to perform feature extraction on the target image according to the characteristic angle to obtain target features.

3. The method according to claim 1, It is characterized in that After fusing the target features with the pre-extracted Haar features, the features are input into a classifier for pedestrian detection, including: The target feature is fused with the pre-extracted Haar feature and then input into several weak classifiers respectively to obtain the classification result output by each weak classifier; Based on the accuracy weight of each weak classifier, the classification results of each weak classifier are weighted and summed to obtain the pedestrian detection result output by the strong classifier; wherein the strong classifier is formed by combining the weak classifiers.

4. A multi-feature fusion pedestrian detection device, used to implement the method as claimed in any one of claims 1 to 3, It is characterized in that including a generating unit configured to generate a first triangular wave signal function for feature extraction of an image block with an odd width and a second triangular wave signal function for feature extraction of an image block with an even width based on a pre-determined discretized triangular wave function; an obtaining unit configured to obtain a target image and a set size of an image block, wherein the image block is used to divide the target image into a plurality of small blocks; an extracting unit configured to determine a corresponding triangular wave signal function based on the set size of the image block, and use the triangular wave signal function to perform feature extraction on the target image to obtain target features; a detecting unit configured to input the target features into a classifier for pedestrian detection after fusing the target features with pre-extracted Haar features.

5. A computing device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method according to any one of claims 1-3 is implemented.

6. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed on a computer, the computer is caused to execute the method according to any one of claims 1-3.

Images