A method for speaker recognition detection based on image processing

Abstract

The present application relates to a kind of speaker recognition detection methods based on image processing, comprising the following steps: obtaining identification detection image, and image gray scale is corrected;The gradient value of each pixel 8 directions of image is calculated, and gradient image is obtained;Gradient image is segmented into several cells, and cell is slid window, and the feature extraction and classification of sliding window HOG are carried out;The feature vector of the detection window obtained is input into first level SVM classifier to carry out face classification detection;Face coordinate data position information is cached;After screening and merging, the position information of each other crossing is input into second level SVM classifier, and the classification detection of speaker face is carried out, after completion, the position information classified as speaker face is saved, and speaker face is framed on original image by frame program, so that it is displayed on screen, and the real-time processing of speaker recognition detection is realized.The method can visually observe the speaker, and effectively improves the recognition accuracy of speaker.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision speaker detection, and particularly relates to a speaker recognition detection method based on image processing. BACKGROUND

[0002] Speaker recognition detection technology has appeared as early as the 1980s, and until now, the most mainstream research direction is still to start from the audio signal to carry out speaker recognition detection in various complex scenes. There are three major research directions in terms of speech signals, one is speech signal parameterization, the mainstream technologies of which include extracting Mel Frequency Cepstral Coefficients (MFCCs), first-order derivative, second-order derivative and other audio features; two is pattern matching, extracting inherent features and performing template matching; three is scoring method, which compares the embedding of the speaker to be tested with the embedding obtained in the training stage, and calculates the similarity of the two through a specific formula, so as to achieve the effect of recognition detection.

[0003] At present, speaker recognition through audio signals is mainly applied to criminal investigation, intelligent customer service, smart home, certificate anti-counterfeiting and other aspects, and the existing information field such as voice unlocking of smart phones also adopts this technology. However, there are still some defects in identifying people through audio, for example, in a monitoring video, if the person is far away from the monitoring or other factors cause there is no sound in the video, which greatly reduces the accuracy of speaker recognition detection. Therefore, a speaker recognition detection method combining image processing technology is needed to solve the problem of low accuracy of speaker recognition detection. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a speaker recognition detection method based on image processing, which can visually observe the speaker and effectively improve the recognition accuracy of the speaker.

[0005] In order to achieve the above purpose, the technical scheme of the present application is:

[0006] A speaker recognition detection method based on image processing, comprising the following steps:

[0007] Step S1, the FPGA acquires the recognition detection image, and gray scales the image, and then performs Gamma correction on the image;

[0008] Step S2, the FPGA obtains 8-neighbor pixels and 16-neighbor pixels of the image through the buffered image pixel data, calculates the gradient values of 8 directions of the image according to the neighbor pixels, and takes the maximum amplitude value in the 8 directions and the corresponding direction as the gradient of the pixel point, to obtain a gradient image;

[0009] Step S3, the FPGA divides the gradient image into several cells, and performs sliding window on all cells with a detection window of 36*36 pixels, and performs HOG feature extraction and classification on the sliding window;

[0010] Step S4, the SVM software processes the detection window through a parallel pipeline, inputs the feature vector of each detection window obtained into a first-level SVM classifier module for face classification detection, and inputs the position information of each detection window into a non-maximum suppression module to remove redundant position information with low confidence and screen and merge the mutually intersecting position information, and then caches the position information of the face coordinate data;

[0011] Step S5, the mutually intersecting position information after screening and merging is input into a second-level SVM classifier module for speaker face classification detection, the position information classified as a speaker face is directly saved after classification, and then a frame drawing program is used to frame the speaker face on the original image, and finally the speaker face is displayed on the screen, so that real-time processing of speaker recognition detection is realized.

[0012] Preferably, in the step S1, the acquiring of the recognition detection image includes: the recognition detection system acquires the image through a camera or other devices, reads the image acquired by the camera or other devices through the FPGA to obtain an input image, adjusts the pixel range of the input image to 800*600 through an image scaling module, and takes the input image as the recognition detection image.

[0013] The image Gamma correction includes: normalizing the image through the FPGA, compensating the pixel points of the normalized image, and denormalizing the calculated compensation pixel points; the compensation calculation formula of the pixel points of the normalized image is: I=V γ , wherein V is the brightness of the normalized image, γ is a correction parameter, and I is the brightness of the normalized image after correction.

[0014] Preferably, in the step S2, the 8 directions of the image are 0 degrees, 26 degrees, 45 degrees, 64 degrees, 90 degrees, 116 degrees, 135 degrees and 153 degrees respectively, and the gradient value calculation formulas of the 8 directions are respectively:

[0015] G0(x,y)=1 / 2[H(x+2,y)+H(x+1,y)-H(x-1,y)-H(x-2,y)]

[0016] G 26 (x,y)=H(x+2,y-1)-H(x-2,y+1)

[0017] G 45(x, y) = 1 / 2[H(x+2, y-2) + H(x+1, y-1) - H(x-1, y+1) - H(x-2, y+2)]

[0018] G 64 (x, y) = H(x+1, y-2) - H(x-1, y+2)

[0019] G 90 = 1 / 2[H(x, y-2) + H(x, y-1) - H(x, y+1) - H(x, y+2)]

[0020] G 116 (x, y) = H(x-1, y-2) - H(x+1, y+2)

[0021] G 135 (x, y) = 1 / 2[H(x-2, y-2) + H(x-1, y-1) - H(x+1, y+1) - H(x+2, y+2)]

[0022] G 153 = H(x-2, y-1) - H(x+2, y+1)

[0023] wherein H(x, y) represents the pixel gray value at position (x, y), the number 2 represents the 16-neighborhood of the image, the number 1 represents the 8-neighborhood of the image; G0(x, y) represents the gradient value in the 0-degree direction, G 26 (x, y) represents the gradient value in the 26-degree direction, G 45 (x, y) represents the gradient value in the 45-degree direction, G 64 (x, y) represents the gradient value in the 64-degree direction, G 90 (x, y) represents the gradient value in the 90-degree direction, G 116 (x, y) represents the gradient value in the 116-degree direction, G 135 (x, y) represents the gradient value in the 135-degree direction, G 153 (x, y) represents the gradient value in the 153-degree direction.

[0024] Preferably, in the step S3, the detection window HOG feature extraction and classification comprises the following steps:

[0025] In step S31, the gradient image is divided into a plurality of cells with a size of 3*3 pixels, and the gradient value and direction of the corresponding pixels of each cell are read, and the gradient values of the 8 directions of each cell are respectively counted into the gradient histogram statistics, to obtain a plurality of feature vectors with a length of 8 one-dimensional vectors;

[0026] Step S32, each 4 cells are grouped into a block in size of 2*2, the gradient histogram of each cell in each block is concatenated, and contrast normalization processing is performed to obtain a VL feature descriptor of each block;

[0027] Step S33, the feature descriptors of all blocks in each detection window are concatenated to obtain a feature vector of each detection window.

[0028] Preferably, in step S4, the face classification detection specifically comprises the following steps:

[0029] Step S41, training of two linear SVM classifiers of face and non-face is performed through a first-level linear SVM classifier to obtain a first-level linear SVM classifier model, cross-validation and adjustment of super parameters of the first-level linear SVM classifier model are performed, optimized parameters of the two first-level linear SVM classifier models are obtained, and the optimized parameters of the two first linear SVM classifier models are respectively stored in a first ROM;

[0030] Step S42, a feature vector of each detection window as a face is judged as a face and a non-face through an SVM decision classification function according to the optimized parameters of the first linear SVM classifier model, and a window coordinate of the face is saved, and a specific judgment formula is as follows:

[0031] f(x) = sign(w1x1 + b1) where sign(w1x1 + b1) = sign(y1), f(x) is an SVM classifier, sign is a sign function, x1 is a feature vector extracted as a face, y1 is an input of the sign function, when a calculated value is greater than 0, the face is judged, otherwise the non-face is judged.

[0032] Preferably, in step S4, the redundant position information with low confidence is removed, a threshold T (T ∈ (0, 1)) is selected as a similarity size of two windows; when the threshold is greater, the two windows are judged as similar, the window with low confidence is removed, and a face frame coordinate is obtained, and face coordinate data is cached.

[0033] Preferably, in step S5, the speaker face classification detection specifically comprises the following steps:

[0034] Step S51, training of two linear SVM classifiers of speaker face and non-speaker face is performed through a second-level linear SVM classifier, optimal super parameters during training are obtained through cross-validation, the optimal super parameters are inputted for optimization, parameters of an optimized second linear SVM classifier model are obtained, and the parameters of the optimized second linear SVM classifier model are respectively stored in a second ROM.

[0035] Step 52, according to the optimized two second linear SVM classifier model parameters, the feature vector of all cells in the detection window is judged as a speaker face and a non-speaker face by the SVM decision classification function, and the coordinates of the window of the speaker face are saved, and the specific judgment formula is:

[0036] f(x)''=sign(w2x2+b2) wherein, sign(w2x2+b2)=sign(y2), f(x)'' is an SVM classifier, sign is a sign function, x2 is the feature vector extracted as a speaker face, y2 is the input of the sign function, When the calculated value is greater than 0, it is judged as a face, otherwise it is judged as a non-face.

[0037] The technical effects and advantages of the present application are:

[0038] 1. The speaker recognition detection method based on image processing provided by the present application can visually observe the speaker and effectively avoid the problem that it is difficult to perform speaker recognition detection because there is no sound in the video due to the long distance between the speaker and the monitor or other factors, thereby effectively improving the recognition accuracy of the speaker.

[0039] 2. The speaker recognition detection method based on image processing provided by the present application calculates the gradient values of 8 directions of the center pixel point by using 24 points in the 16-neighborhood of the pixel point, and records the direction with the largest amplitude value as the direction of the pixel gradient, thereby avoiding the calculation of the arctangent value and reducing the complexity.

[0040] 3. The speaker recognition detection method based on image processing provided by the present application adopts a two-stage linear classifier, first uses a first-stage linear SVM classifier for face detection on the images collected by the camera, and then uses a second-stage linear SVM classifier for speaker face classification on the detected face, thereby greatly saving the resources and time occupied by the second-stage linear SVM classifier for speaker face detection and speeding up the hardware processing speed. DETAILED DESCRIPTION

[0041] Figure 1 is a flowchart of the speaker recognition detection method based on image processing of the present application;

[0042] Figure 2 is a flowchart of the speaker recognition detection method based on image processing of the present application;

[0043] Figure 3is a flow chart of HOG feature extraction and classification of images of the present application;

[0044] Figure 4 is a flow chart of speaker face detection of the present application;

[0045] Figure 5 is a 16-neighborhood map of the center pixel (x, y) of a gradient image of the present application;

[0046] Figure 6 is a pixel position relationship map used for 8 gradient direction calculation of a gradient image of the present application;

[0047] Figure 7 is a relationship diagram among a sliding window, a cell and a block during feature extraction of a detection window of the present application;

[0048] Figure 8 is a step diagram of SVM classification detection of a detection window of the present application. DETAILED DESCRIPTION

[0049] The embodiments given below in combination with the accompanying drawings further illustrate the present application.

[0050] Referring to Figure 1 An image processing-based speaker recognition detection method shown in the figure comprises the following steps:

[0051] Step S1, the FPGA acquires a recognition detection image, and gray scales the image, then performs Gamma correction on the image, and buffers the pixel data of the corrected image;

[0052] Step S2, the FPGA obtains 8-neighborhood pixels and 16-neighborhood pixels of the image through the pixel data of the corrected image, calculates gradient values of 8 directions of the image according to the neighborhood pixels, takes the maximum amplitude value and the corresponding direction among the 8 directions as the gradient of the pixel, and obtains a gradient image;

[0053] Step S3, the FPGA divides the gradient image into a plurality of cells, performs sliding window on all cells with a 36*36 pixel detection window, performs HOG feature extraction and classification on the sliding window, and obtains a feature vector of each detection window;

[0054] Step S4, the SVM software processes the detection window in a parallel pipeline manner, inputs the obtained feature vector of each detection window into a first-level SVM classifier module for face classification detection, inputs the position information of each detection window into a non-maximum suppression module, removes redundant position information with low confidence and filters and combines mutually intersecting position information, and then buffers the face coordinate data position information;

[0055] Step S5, the mutual cross position information after the screening is combined is input into the second level SVM classifier module, and the speaker face classification detection is performed, the position information classified as the speaker face is directly saved after the classification is completed, and the non-speaker face position information is deleted;

[0056] Step S6, the position information classified as the speaker face is input into the frame module, and the speaker face is marked on the original color image through the frame module, and then output to display on the screen, and the real-time processing of the speaker face recognition detection is completed.

[0057] Embodiments

[0058] A speaker recognition detection method based on image processing is implemented as follows:

[0059] Step S1, the FPGA obtains the recognition detection image, and the image is grayed, then the image is Gamma corrected, and the corrected image pixel data is cached.

[0060] In specific implementation, the recognition detection image is obtained by the recognition detection system through the camera acquisition or the image transmission of other devices, the input image is obtained by reading the image of the camera or other devices through the FPGA, and in order to adapt the size of the face, the pixel range of the input image is adjusted to 800*600 through the image scaling module, and the input image is taken as the recognition detection image.

[0061] In specific implementation, the image Gamma correction includes normalizing the image through the FPGA, compensating the pixel points of the normalized image, and denormalizing the calculated compensation pixel points; the compensation calculation formula of the pixel points of the normalized image is: I=V γ , wherein V is the brightness of the normalized image, γ is the correction parameter, and I is the brightness of the corrected normalized image.

[0062] In specific implementation, the recognition detection image is grayed and Gamma corrected, which can weaken the influence of light and the like on the recognition detection image, and can overcome the nonlinear response of the human visual system to the brightness of the recognition detection image.

[0063] In specific implementation, the resolution of the image acquired by the camera or transmitted by other devices is generally high (such as 1080P, that is, 1920*1080), and in order to adapt the size of the face for recognition detection, the pixels of the input recognition detection image are adjusted to be within 800*600 through the scaling module.

[0064] In implementation, the FPGA is implemented by using the look-up table (LUT) method, the value of V∈[0:255] is normalized by using the MATLAB software, then the compensation pixel point is calculated by using the pixel point compensation calculation formula of the normalized image, finally the compensation pixel point is denormalized, and the compensation pixel point is saved in the coe file.

[0065] Further, the coe file can be initialized to the ROM of the FPGA, and the gray value of the image is taken as the address, and when the subsequent processing is performed, the FPGA reads the corresponding value stored in the ROM.

[0066] In step S2, the 8-neighborhood pixels and 16-neighborhood pixels of the image are obtained by using the corrected image pixel data, the gradient values of 8 directions of the image are calculated according to the neighborhood pixels, the maximum amplitude value in the 8 directions and the corresponding direction are taken as the gradient of the pixel point, and the gradient image is obtained.

[0067] In implementation, the gradient values of 8 directions of the image are calculated, so that the gradient image is obtained, the influence of the light can be further weakened, and the definition of the image is improved.

[0068] In implementation, referring to FIG. 1, Figure 6 As shown in FIG. 1, the 8 directions of the image are 0 degree, 26 degree, 45 degree, 64 degree, 90 degree, 116 degree, 135 degree and 153 degree respectively, and the gradient value calculation formulas of the 8 directions are respectively:

[0069] G0(x,y)=1 / 2[H(x+2,y)+H(x+1,y)-H(x-1,y)-H(x-2,y)]

[0070] G 26 (x,y)=H(x+2,y-1)-H(x-2,y+1)

[0071] G 45 (x,y)=1 / 2[H(x+2,y-2)+H(x+1,y-1)-H(x-1,y+1)-H(x-2,y+2)]

[0072] G 64 (x,y)=H(x+1,y-2)-H(x-1,y+2)

[0073] G 90 =1 / 2[H(x,y-2)+H(x,y-1)-H(x,y+1)-H(x,y+2)]

[0074] G 116 (x,y)=H(x-1,y-2)-H(x+1,y+2)

[0075] G 135 (x,y) = 1 / 2[H(x-2,y-2) + H(x-1,y-1) - H(x+1,y+1) - H(x+2,y+2)]

[0076] G 153 = H(x-2,y-1) - H(x+2,y+1)

[0077] wherein H(x,y) represents the pixel gray value at position (x,y), the number 2 represents the 16-neighborhood of the image, and the number 1 represents the 8-neighborhood of the image; G0(x,y) represents the gradient value in the 0-degree direction, G 26 (x,y) represents the gradient value in the 26-degree direction, G 45 (x,y) represents the gradient value in the 45-degree direction, G 64 (x,y) represents the gradient value in the 64-degree direction, G 90 (x,y) represents the gradient value in the 90-degree direction, G 116 (x,y) represents the gradient value in the 116-degree direction, G 135 (x,y) represents the gradient value in the 135-degree direction, G 153 (x,y) represents the gradient value in the 153-degree direction.

[0078] In implementation, the gradient values in the 0-degree, 45-degree, 90-degree and 135-degree directions of the gray image are obtained by averaging the gradient values in the 8-neighborhood and 16-neighborhood of the 0-degree direction, the 8-neighborhood and 16-neighborhood of the 45-degree direction, the 8-neighborhood and 16-neighborhood of the 90-degree direction, and the 8-neighborhood and 16-neighborhood of the 135-degree direction, respectively.

[0079] In implementation, referring to Fig. 1, the image pixel data is a matrix, the number of rows and the number of columns of which are both 5, and the center of the matrix is the center pixel (x,y) of the image. The 8-neighborhood pixels of the center pixel (x,y) are connected with the center pixel (x,y), and the 16-neighborhood pixels of the center pixel (x,y) are connected with the 8-neighborhood pixels. Figure 5

[0080] In implementation, the gradient values in the 8 directions of the center pixel are calculated by using the 24 pixels in the 16-neighborhood of the center pixel, and the direction with the largest amplitude value is recorded as the direction of the gradient of the pixel, so that the calculation of the arctangent value is avoided, and the complexity is reduced.

[0081] ​Step S3, the FPGA divides the gradient image into a plurality of cells, and performs sliding window on all cells with a detection window of 36*36 pixels, and performs HOG feature extraction and classification on the sliding window to obtain a feature vector of each detection window. The specific number of cells into which the gradient image is divided is 144.

[0082] In specific implementation, the sliding window HOG feature extraction and classification includes the following steps:

[0083] The gradient image is divided into 144 cells with a size of 3*3 pixels, the gradient values of 8 directions of each cell are calculated, and the gradient values of 8 directions of each cell are respectively counted into the gradient histogram statistics to obtain 144 one-dimensional vectors with a length of 8.

[0084] In specific implementation, every 4 cells are grouped into a block with a size of 2*2, the gradient histograms of each cell in each block are concatenated, and contrast normalization processing is performed to obtain a dimension feature descriptor of each block.

[0085] In specific implementation, 144 cells are taken as a block with an interval of 6 pixels in the horizontal and vertical directions, which can ensure that the blocks do not overlap; and every 4 cells are grouped into a block, so that a detection window can be composed of 36 blocks, and the number of dimension feature descriptors of each block is 4*8=32.

[0086] In specific implementation, the feature descriptors of all blocks in each detection window are concatenated to obtain a feature vector of each detection window. The number of dimension feature descriptors in each detection window is 36*32=1152.

[0087] Step S4, the SVM software processes the detection window through a parallel pipeline, inputs the obtained feature vector of each detection window into a first-level SVM classifier module for face classification detection, inputs the position information of each detection window into a non-maximum suppression module, removes redundant position information with low confidence and filters and merges the position information that intersects with each other, and then caches the position information of the face coordinate data;

[0088] In specific implementation, the face classification detection specifically includes the following steps:

[0089] The first-level linear SVM classifier is trained through two linear SVM classifiers of face and non-face, and the optimal hyperparameters during training are obtained through cross-validation, the optimal hyperparameters are input for optimization, the parameters of the optimized first-level linear SVM classifier model are obtained, and the parameters of the optimized first linear SVM classifier model are respectively stored in the first ROM.

[0090] In implementation, the parameters of the first linear SVM classifier model are normal vector W1 and bias b1 respectively.

[0091] In implementation, the test image set with known ground-truth (containing face and non-face) is read by MATLAB software, and the generalization of the trained SVM model is tested by SVM performance indicators (such as accuracy, recall, F1, etc.).

[0092] In implementation, according to the parameters of the two first linear SVM classifier models optimized, the feature vector of each detection window is judged as face or non-face by the SVM decision classification function, and the specific judgment formula is:

[0093] f(x) = sign(w1x1 + b1) where sign(w1x1 + b1) = sign(y1), f(x) is the SVM classifier, sign is the sign function, x1 is the feature vector extracted as face, y1 is the input of the sign function, When the calculated value is greater than 0, it is judged as face, otherwise it is judged as non-face.

[0094] In implementation, the feature vector of each detection window is multiplied by the first level SVM parameter normal vector w1, and then the bias b1 is added, and finally the judgment module is used for judgment. When the calculated value is greater than 0, it is judged as face, otherwise it is judged as non-face.

[0095] In implementation, referring to Figure 8 As shown in the figure, 1152-dimensional feature descriptors in each detection window are divided into 6 segments according to each column block of the sliding window, with 192-dimensional feature descriptors in each segment, and are respectively stored in 6 BLOCK RAMs, buffering a sliding window size. When sliding through 6 pixels for the first time, the 36x6 pixel gradient histogram output by the sliding window is multiplied and added with the SVM in the first segment ROM, and the sum is saved. When sliding again, the 36x6 pixel gradient histogram output is multiplied and added with the SVM in the first segment ROM and the second segment ROM. The size of each 6-pixel sliding is multiplied and added with the next new SVM and all the previously used SVMs in turn. Each multiplication and addition sum is saved separately according to the corresponding SVM segment.

[0096] In implementation, the parameter of the normal vector w1 of the SVM is a 1152-dimensional feature descriptor (i.e. a first column vector containing 1152 elements), and the feature vector extracted by each sliding window is also a 1152-dimensional feature descriptor. When making a decision, the 1152-dimensional feature descriptor extracted needs to be multiplied by the 1152-dimensional feature descriptor of the normal vector w1. Since the feature vector extracted by the sliding window is spliced from the feature vectors of the blocks in the detection window in the order from left to right and from top to bottom, the parameter of the normal vector w1 of the SVM also corresponds to the order of the blocks in the sliding window from left to right and from top to bottom.

[0097] In implementation, the sliding window slides from left to right by 6 pixels each time, and 6 blocks from top to bottom appear. At this time, the 6x32=192-dimensional feature descriptor values of the six blocks need to be multiplied by the corresponding w1 parameter. Therefore, the parameter of the normal vector w1 needs to be processed in advance, i.e. the order of the normal vector w1 is modified to correspond to the order of the blocks in the sliding window from top to bottom and from left to right. The modified parameter of the normal vector w1 is that every 192 elements are the parameter of the normal vector w1 corresponding to the feature descriptor values of a column of blocks (i.e. 6 blocks) in the sliding window.

[0098] In implementation, 6 blocks in a column appear each time the sliding window slides by 6 pixels. Therefore, the modified parameter of the normal vector w1 is stored in 6 segments of 192 elements each.

[0099] In implementation, when the first 36x36 detection window is horizontally slid, the first data of the sum saved in each segment is taken, the six sums are added, and a bias is added. When the sum is greater than 0, it is determined that there is a face. In turn, the six sums of the corresponding window in each segment are added and a bias is added to determine whether the sum is greater than 0 each time a window is slid. When the detection window is slid once, the process is repeated from the next detection window. In implementation, see the description of FIG. 6. Figure 8

[0100] In implementation, the removal of redundant position information with low confidence is that a threshold T (T∈(0,1)) is selected as the similarity of the two windows. When the similarity is greater than the threshold, it is determined that the two windows are similar, and the window with low confidence is removed, so as to obtain the face frame coordinates. The face coordinate data is cached.

[0101] Step S5: The position information obtained by screening and merging is input into a second-level SVM classifier module for speaker face classification and detection. After classification, the position information classified as a speaker face is saved, and the position information of non-speaker faces is deleted.

[0102] In implementation, the speaker face classification and detection specifically includes the following steps. ​

[0103] The training of the second-level SVM classifier is performed by a linear SVM classifier for speaker face and non-speaker face, and the optimal hyperparameters during the training are obtained by cross-validation, the optimal hyperparameters are input for optimization, the parameters of the optimized second linear SVM classifier model are obtained, and the parameters of the optimized second linear SVM classifier model are respectively stored in the second ROM.

[0104] In specific implementation, the parameters of the second linear SVM classifier model are respectively a normal vector W2 and a bias b2.

[0105] According to the parameters of the optimized second linear SVM classifier model, the feature vectors of all cells in the detection window that are speaker faces are judged as speaker faces and non-speaker faces by an SVM decision classification function, and the coordinates of the window of the speaker faces are saved, and the specific judgment formula is:

[0106] f(x)″=sign(w2x2+b2) wherein, sign(w2x2+b2)=sign(y2), f(x)″ is an SVM classifier, sign is a sign function, x2 is a feature vector extracted as a speaker face, y2 is an input of the sign function, When the calculated value is greater than 0, it is judged as a speaker face, otherwise it is judged as a non-speaker face.

[0107] In specific implementation, by adopting two-level linear classifiers, first, a first-level linear SVM classifier is adopted for face detection on the images collected by the camera, and second, a second-level linear SVM classifier is adopted for speaker face detection on the detected faces, and after the first-level linear SVM classifier, the second-level linear SVM classifier for speaker face detection can greatly save the occupied resources and time, and speed up the hardware processing speed.

[0108] Step S6, after the judgment, the non-maximum suppression is no longer needed, the position information of the classified speaker faces is directly saved, the position information of the non-speaker faces is discarded, the position information of the classified speaker faces is input into a framing module, the framing module marks the speaker faces on the original color image, and then outputs the speaker faces on the screen for display, completing the real-time processing of speaker face recognition detection.

[0109] In specific implementation, the coordinate information of the sliding window with speaker faces in the detection window obtained after the non-maximum suppression is taken as an address, all cells in the sliding window are read from the gradient histogram of all cells in the detection window as feature vectors, the feature vectors are multiplied by the second-level SVM parameter normal vector W2 and added, and then the bias b2 is added, the result is passed through the sign(y2) sign function, and when greater than zero, it is judged as a speaker face, thereby realizing speaker face detection.

[0110] Specifically, the speaker recognition detection method based on image processing provided by the present application is suitable for an intelligent meeting scene; a camera is used to collect a meeting scene image, a speaker face detection model is used to detect a speaker face, and the camera can automatically track a meeting speaker in real time according to the position of the speaker.

[0111] Specifically, the speaker recognition detection method based on image processing provided by the present application is suitable for sound source positioning; a speaker face detection model is used to locate a speaker and enlarge the image of the speaker alone, so that an image of a larger scene can be obtained. Multiple speakers are located, and the image of the speaker is automatically switched.

[0112] Specifically, the speaker recognition detection method based on image processing provided by the present application is suitable for a concert, an award ceremony and the like; a camera can be used to detect the real-time image of a speaker on a stage and a singer in real time, and the position of the speaker can be tracked automatically, so that the image of the speaker can be projected on a large screen in real time.

[0113] Specifically, the speaker recognition detection method based on image processing provided by the present application is suitable for a surgery scene; it is strictly prohibited to speak near an operating room during surgery, and a speaker detection module is used to facilitate the capture of a speaker by a camera, so that the speaker can be stopped conveniently.

[0114] Specifically, the speaker recognition detection method based on image processing provided by the present application is suitable for a language dysfunction population; in some scenes, such as a school for the deaf and mute, a speaker cannot make a sound due to language dysfunction, and it is inconvenient to seek help, such as in a room and a corridor without people, and the like. When the speaker opens his mouth to speak, a camera can obtain the image of the speaker, and the image is fed back to a medical staff or a duty staff, so that the speaker can be rescued or helped in time.

[0115] The above only describes the preferred embodiments of the present application, and it should be noted that, for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A method for speaker recognition detection based on image processing, characterized in that: The method comprises the following steps: Step S1, the FPGA acquires a recognition detection image, and the image is grayed, then the image is subjected to Gamma correction, and the pixel data of the corrected image is cached; Step S2, the FPGA obtains 8-neighbor pixels and 16-neighbor pixels of the image through the pixel data of the corrected image, calculates gradient values of 8 directions of the image according to the neighbor pixels, takes the maximum amplitude value and the corresponding direction in the 8 directions as the gradient of the pixel point, and obtains a gradient image; Step S3, the FPGA divides the gradient image into a plurality of cells, and performs sliding window on all cells with a detection window of 36*36 pixels, and extracts HOG features and classifies the sliding window to obtain a feature vector of each detection window; Step S4, the SVM software processes the detection window in a parallel pipeline manner, inputs the feature vector of each detection window obtained into a first-level SVM classifier module for face classification detection, inputs the position information of each detection window into a non-maximum suppression module, removes redundant position information with low confidence and filters and combines the position information intersecting with each other, and then caches the position information of the face coordinates; Step S5, the position information intersecting with each other after filtering and combining is input into a second-level SVM classifier module for speaker face classification detection, and after the classification is completed, the position information classified as the speaker face is directly saved, and the position information of the non-speaker face is deleted; Step S6, the position information classified as the speaker face is input into a frame drawing module, the speaker face is marked on the original color image through the frame drawing module, and then the image is output on the screen for display, thereby completing real-time processing of speaker face recognition detection.

2. The method according to claim 1, wherein the method comprises: In the step S1, the recognition detection image is acquired by the recognition detection system through a camera or other devices, the input image is obtained by reading the image transmitted by the camera or other devices through the FPGA, the pixel range of the input image is adjusted to 800*600 through an image scaling module, and the image is taken as the recognition detection image; The image gamma correction comprises normalizing the image by the FPGA, compensating the pixel points of the normalized image, and finally denormalizing the calculated compensation pixel points; the compensation calculation formula of the pixel points of the normalized image is: I = V γ wherein V is the brightness of the normalized image, γ is the correction parameter, and I is the brightness of the corrected normalized image.

3. The method of claim 1, wherein the method further comprises: In the step S2, the 8 directions of the image are 0 degrees, 26 degrees, 45 degrees, 64 degrees, 90 degrees, 116 degrees, 135 degrees and 153 degrees respectively, and the gradient value calculation formulas of the 8 directions are as follows: G0(x,y)=1 / 2[H(x+2,y)+H(x+1,y)-H(x-1,y)-H(x-2,y)] G 26 (x, y) = H(x + 2, y - 1) - H(x - 2, y + 1) G 45 (x,y) = 1 / 2[H(x+2,y-2) + H(x+1,y-1) - H(x-1,y+1) - H(x-2,y+2)] G 64 (x, y) = H(x + 1, y - 2) - H(x - 1, y + 2) G 90 = 1 / 2 [H(x, y - 2) + H(x, y - 1) - H(x, y + 1) - H(x, y + 2)] G 116 (x,y) = H(x - 1,y - 2) - H(x + 1,y + 2) G 135 (x,y) = 1 / 2 [H(x-2,y-2) + H(x-1,y-1) - H(x+1,y+1) - H(x+2,y+2)] G 153 = H(x - 2, y - 1) - H(x + 2, y + 1) where H(x, y) represents the pixel gray value at position (x, y), the number 2 represents the 16-neighborhood of the image, the number 1 represents the 8-neighborhood of the image; G0(x, y) represents the gradient value in the 0-degree direction, G 26 (x, y) represents the gradient value in the 26-degree direction, G 45 (x, y) represents the gradient value in the 45-degree direction, G 64 (x, y) represents the gradient value in the 64-degree direction, G 90 (x, y) represents the gradient value in the 90-degree direction, G 116 (x, y) represents the gradient value in the 116-degree direction, G 135 (x, y) represents the gradient value in the 135-degree direction, G 153 (x, y) represents the gradient value in the 153-degree direction.

4. The method according to claim 3, wherein the method comprises: In the step S3, the HOG feature extraction and classification of the detection window comprises the following steps: Step S31, the gradient image is divided into a plurality of cells with a size of 3*3 pixels, the gradient value and direction of the corresponding pixel of each cell are read, and the gradient values of the 8 directions of each cell are respectively counted into the gradient histogram statistics to obtain a plurality of one-dimensional vectors with a length of 8; Step S32, each 4 cells are combined into a block with a size of 2*2, the gradient histograms of each cell in each block are concatenated, and contrast normalization processing is performed to obtain a dimension feature descriptor of each block; Step S33, concatenating the feature descriptors of all the blocks in each detection window to obtain a feature vector of each detection window.

5. The method of claim 3, wherein the method further comprises: The face classification detection in step S4 specifically includes the following steps: Step S41, training two linear SVM classifiers of face and non-face through the first level linear SVM classifier, and obtaining the optimal hyperparameters during the training through cross-validation, taking the optimal hyperparameters as input to optimize, obtaining the parameters of the optimized first level linear SVM classifier model, and storing the parameters of the optimized first linear SVM classifier model in the first ROM respectively; Step S42, judging face and non-face through the SVM decision classification function according to the parameters of the optimized first linear SVM classifier model, and saving the coordinates of the window of the detection window judged as face, and the specific judgment formula is: f(x) = sign(w1x1 + b1) where, sign(w1x1 + b1) = sign(y1), f(x) is a SVM classifier, sign is a sign function, x1 is a feature vector extracted to be a human face, y1 is an input of the sign function, When the calculated value is greater than 0, it is determined to be a human face, otherwise it is determined to be a non-human face.

6. The method of claim 1, wherein the method further comprises: In step S4, the redundant position information with low confidence is removed, a threshold T (T ∈ (0, 1)) is selected as the similarity of two windows; when it is greater than the threshold, it is judged that the two windows are similar, and the window with low confidence is removed, so as to obtain the face frame coordinates, and the face coordinate data is cached.

7. The method of claim 5, wherein the method further comprises: In step S5, the speaker face classification detection specifically includes the following steps: Step S51, training two linear SVM classifiers of speaker face and non-speaker face through the second level linear SVM classifier, and obtaining the optimal hyperparameters during the training through cross-validation, taking the optimal hyperparameters as input to optimize, obtaining the parameters of the optimized second linear SVM classifier model, and storing the parameters of the optimized second linear SVM classifier model in the second ROM respectively; Step 52, judging speaker face and non-speaker face through the SVM decision classification function according to the parameters of the two optimized second linear SVM classifier models, and the specific judgment formula is: f(x)" = sign(w2x2 + b2) where, sign(w2x2 + b2) = sign(y2), f(x)" is a SVM classifier, sign is a sign function, x2 is a feature vector extracted to be a face of a speaker, y2 is an input of the sign function, When the calculated value is greater than 0, it is determined to be a face of a speaker, otherwise it is determined to be a non-speaker face.

Images