Prison inspection method and device

Abstract

The present invention relates to the field of sound collection, and specifically discloses a prison inspection method and device. It includes constructing a prison space coordinate system by modeling the prison structure, locating the relative position of the microphone in the prison space coordinate system, obtaining the sound signal collected by the microphone and performing first-order filtering and noise reduction processing to obtain the noise-reduced sound signal, extracting MFCC features from the noise-reduced sound signal, and analyzing through CNN model processing to obtain separated human voice signals and / or environmental sound signals and sound classification results. If there is a human voice in the sound classification result, voiceprint recognition is performed to obtain a suspicious person, and the sound source is located according to the relative position of the microphone, the separated human voice signal and / or environmental sound signal, the sound source position is determined, and the timeline and event line are connected in series to generate the prison events of the day and output an automated inspection report. The present invention accurately locates, predicts and traces back, and improves the efficiency of prison discipline inspection.

Description

Technical Field

[0001] The invention relates to the field of sound collection, and in particular to a prison inspection method and device. Background Art

[0002] In the prior art, as disclosed in "201810858358.6", the prison fight identification device and method mentions that the melody features of the previous fight audio are first extracted and stored in the fight audio database, and then a rotating camera with an audio acquisition module is installed in the prison. The audio acquisition module collects the audio in the prison and extracts the feature audio in segments. The extracted feature audio is compared with the feature audio in the fight audio database. If the similarity reaches a preset threshold range, it can be considered that there is a fight in the prison, and then an alarm is issued to notify the prison guards to patrol and stop the prisoners from fighting, so as to ensure the quietness of the prison and the safety of the prisoners.

[0003] Traditional inspection technology, firstly, relies on camera points to collect information, which cannot guarantee the comprehensiveness of information collection. Secondly, the collected sound processing is only preliminary feature extraction for comparison, which is not perfect and may lead to high error rate of sound comparison. Thirdly, it only compares single sounds, and when similar cases occur again, it cannot trace back historical events for comparison. In addition, it is impossible to accurately locate the location of the incident, so prison guards cannot arrive in time to deal with it. Summary of the invention

[0004] In view of the above problems, an object of the present invention is to provide a prison inspection method.

[0005] A second object of the present invention is to provide a prison inspection device.

[0006] The first technical solution adopted by the present invention is: S1, constructing a prison space coordinate system by modeling the prison structure, and locating the relative position of the microphone in the prison space coordinate system; S2, obtaining the sound signal collected by the microphone and performing first-order filtering and noise reduction processing to obtain the noise-reduced sound signal; S3, extracting MFCC features from the noise-reduced sound signal, processing and analyzing it through a CNN model, obtaining a separated human voice signal and / or environmental sound signal and a sound classification result; S4, if there is a human voice in the sound classification result, performing voiceprint recognition on the separated human voice signal to obtain a suspicious person; S5, locating the sound source according to the relative position of the microphone, the separated human voice signal and / or environmental sound signal, and determining the sound source position; S6, using the suspicious person, the sound classification result, the sound source position and the sound source occurrence time to confirm the event, connect the timeline and the event line in series, generate the prison event of the day and output an automated inspection report.

[0007] Optionally, the S1 includes:

[0008] S11, performing spatial modeling based on the prison structure to obtain a prison spatial model;

[0009] S12, establishing a spatial coordinate system according to the prison space model, and dividing the space according to the space;

[0010] S13. Based on the divided space, establish room coordinates for each room and locate the relative position of the microphone in the space coordinate system.

[0011] Optionally, also include:

[0012] S7. Analyze the historical inspection data to determine the characteristics of the suspicious person;

[0013] S8. Predict the event based on the personnel characteristics, and if it is likely to develop into a serious event, send a warning notification to the prison guard.

[0014] Optionally, S2 includes:

[0015] S21, obtaining the sound signal collected by the microphone, and converting it into a frequency domain signal through FFT;

[0016] S22, based on the frequency domain signal, processing it through a low-pass first-order filter to obtain a processed signal;

[0017] S23, performing inverse FFT on the processed signal to convert it into a time domain signal to obtain the noise-reduced sound.

[0018] Optionally, the S3 includes:

[0019] S31, extracting the MFCC features of the noise-reduced sound to obtain prison sound features;

[0020] S32. Determine the sound classification in the prison scene based on the prison sound features, construct and train a six-layer CNN model, and obtain human voice signals and / or environmental sound signals separated according to the sound classification in the prison scene.

[0021] Optionally, the S31 includes:

[0022] S311, performing data preprocessing on the noise-reduced sound to obtain a preprocessed signal;

[0023] S312, performing fast Fourier transform on the preprocessed signal to obtain a spectrum diagram;

[0024] S313, mapping the spectrogram to a Mel-scale spectrogram using overlapping triangular windows whose center frequencies are distributed on a Mel-scale, to obtain a mapped Mel-scale spectrogram;

[0025] S314, performing a logarithmic operation on the mapped Mel spectrum to obtain a Mel logarithmic power spectrum;

[0026] S315, performing discrete cosine transform on the Mel log power spectrum to obtain MFCC features.

[0027] Optionally, the six-layer CNN model includes six convolutional layers, two fully connected layers and one output layer connected in sequence.

[0028] The six convolutional layers are divided into three groups, each of which uses a 3×3 receptive field and a convolution operation with a stride of 1×1;

[0029] The first group consists of the first and second convolutional layers, each using 32 kernels, and a maximum pooling layer with a stride of 2×2 and a dropout layer after the second convolutional layer;

[0030] The second group consists of the third and fourth convolutional layers, each using 64 kernels, and a maximum pooling layer with a stride of 2×2 and a dropout layer after the fourth convolutional layer;

[0031] The third group consists of the fifth and sixth convolutional layers, each using 128 kernels, and a maximum pooling layer with a stride of 2×2 and a dropout layer after the sixth convolutional layer;

[0032] The two fully connected layers are each composed of 1024 neurons, forming a classifier;

[0033] The output layer is composed of multiple neurons, corresponding to multiple sound classifications in training.

[0034] Optionally, the S4 includes:

[0035] S41, if the human voice exists in the sound classification result, use the ECAPA-TDNN scheme to perform prison-wide screening and small room matching;

[0036] S42. Determine the TOP5 scoring personnel based on the matching results, and increase the weighted scores based on the personnel's actions in the environment. The scoring formula is:

[0037] S=0.68*T+0.2*R+0.12F

[0038] In the formula, s is the total score, T is the algorithm score, R indicates whether the person exists in the room where the microphone is located, and F indicates whether the person exists on the current floor.

[0039] Optionally, the S31 includes:

[0040] The MFCC features of the denoised sound are extracted based on the following formula:

[0041]

[0042] Where M is the number of triangular filters, X m is the logarithmic energy of the mth log Mel-spectrogram, and c is the index of the cepstral coefficient.

[0043] The second technical solution adopted by the present invention is: a prison inspection device, comprising:

[0044] A microphone positioning module, used to construct a prison space coordinate system by modeling the prison structure, and locate the relative position of the microphone in the prison space coordinate system;

[0045] A noise reduction module, used to obtain the sound signal collected by the microphone and perform first-order filtering and noise reduction processing to obtain a noise-reduced sound signal;

[0046] A sound separation module is used to extract MFCC features from the noise-reduced sound signal, process and analyze it through a CNN model, and obtain a separated human voice signal and / or environmental sound signal and a sound classification result;

[0047] A voiceprint recognition module, configured to perform voiceprint recognition on the separated human voice signal to identify a suspicious person if there is a human voice in the sound classification result;

[0048] A sound source confirmation module, used to locate the sound source according to the relative position of the microphone, the separated human voice signal and / or the ambient sound signal, and determine the position of the sound source;

[0049] The report generation module is used to confirm the event by using the suspicious person, the sound classification result, the sound source location and the sound source occurrence time, connect the timeline and the event line, generate the prison event of the day and output an automated inspection report.

[0050] Beneficial effects of the above technical solution:

[0051] It can accurately identify the location of the incident in real time, allowing prison guards to quickly check and handle it; it can also trace back key disciplinary violations. After a disciplinary violation occurs, the entire life cycle of the problem can be traced through daily records. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a prison inspection method provided by one embodiment of the present invention;

[0053] Figure 2 A schematic diagram of the structure of a prison inspection device provided by one embodiment of the present invention;

[0054] Figure 3Establish the coordinates of the enclosed space in the prison and the schematic diagram of the pickup location;

[0055] Figure 4 This is a schematic diagram of the noise reduction process of the sound signal;

[0056] Figure 5 Schematic diagram of the six-layer CNN model structure. DETAILED DESCRIPTION

[0057] The following detailed description of the embodiments of the present invention is further described in detail in conjunction with the accompanying drawings and examples. The detailed description of the following embodiments and the accompanying drawings are used to exemplarily illustrate the principles of the present invention, but cannot be used to limit the scope of the present invention, that is, the present invention is not limited to the preferred embodiments described, and the scope of the present invention is defined by the claims.

[0058] In the description of the present invention, it should be noted that, unless otherwise specified, “plurality” means two or more than two; the terms “first”, “second”, etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance; for ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0059] Embodiment 1

[0060] An embodiment of the present invention provides a prison inspection method, such as Figure 1 As shown, S1, construct a prison space coordinate system by modeling the prison structure, and locate the relative position of the microphone in the prison space coordinate system; S2, obtain the sound signal collected by the microphone and perform first-order filtering and noise reduction processing to obtain the noise-reduced sound signal; S3, extract MFCC features from the noise-reduced sound signal, process and analyze it through the CNN model, and obtain separated human voice signals and / or environmental sound signals and sound classification results; S4, if there is a human voice in the sound classification result, perform voiceprint recognition on the separated human voice signal to obtain a suspicious person; S5, locate the sound source according to the relative position of the microphone, the separated human voice signal and / or environmental sound signal, and determine the sound source position; S6, use the suspicious person, the sound classification result, the sound source position and the sound source occurrence time to confirm the event, connect the timeline and the event line, generate the prison event of the day and output an automated inspection report.

[0061] S1. Constructing a prison space coordinate system by modeling the prison structure, and locating the relative position of the microphone in the prison space coordinate system.

[0062] The present invention completes the establishment of closed space coordinates in the prison through spatial modeling, and then performs spatial segmentation of the room corridor. Establish room coordinates and locate the relative position of the microphone. Establish complete coordinates based on the prison, set up the prison origin (0, 0, 0), and obtain several bounded spaces according to the room segmentation in the figure. The origin coordinates of room A (x1, y1, z1) are (a, b, c), where the length, width and height are m1, m2, and m3, then the bounded interval of x1 is a≤x1≤m1, the bounded interval of y1 is b≤y1≤m2, and the bounded interval of z1 is b≤z1≤m3. Figure 3 As shown, including a schematic diagram of the pickup positioning.

[0063] It should be noted that two fixed microphones are installed in a monitoring space for minimum positioning. The spatial positioning method is based on the bounded space cut out of the room. By dividing the bounded space, the spatial range can be utilized to a greater extent. In the case of limited microphones, the monitoring effect of the microphones can be maximized, reducing costs while reducing dependence on microphones when locating events. The spatial division method is used for positioning, and the reduction of space makes positioning more accurate.

[0064] S11, performing spatial modeling based on the prison structure to obtain a prison spatial model;

[0065] S12, establishing a spatial coordinate system according to the prison space model, and dividing the space according to the space;

[0066] S13. Based on the divided space, establish room coordinates for each room and locate the relative position of the microphone in the space coordinate system.

[0067] Through prison modeling and the establishment of spatial coordinates, the location of the incident can be accurately identified in real time, allowing prison guards to quickly arrive at the scene and conduct inspections and handle the incident in a timely manner. At the same time, the prison structure is made transparent, showing the prison structure more intuitively and better for subsequent processing.

[0068] S2. Acquire the sound signal collected by the microphone and perform first-order filtering and noise reduction processing to obtain a noise-reduced sound signal.

[0069] S21, obtaining the sound signal collected by the microphone, and converting it into a frequency domain signal through FFT;

[0070] S22, based on the frequency domain signal, processing it through a low-pass first-order filter to obtain a processed signal;

[0071] S23, performing inverse FFT on the processed signal to convert it into a time domain signal to obtain the noise-reduced sound.

[0072] First, we convert the collected sound signal into a frequency domain signal through FFT (Fast Fourier Transform).

[0073] The formula is:

[0074] X(k)=∑n=0N-1x(n)WNnk

[0075] In the formula: X(k) represents the signal in the frequency domain; x(n) represents the signal in the time domain; WNnk represents a complex number, which is WNnk=exp(-j2πkn / N); j is an imaginary number; k represents the number of movements; N represents the size of the data to be converted.

[0076] Then it is processed through a low-pass first-order filter to eliminate the influence of low-frequency and small-band noise.

[0077] The formula is:

[0078] Y(n)=aX(n)+(1-a)Y(n-1)

[0079] Where: Y(n) is the output value of this filter; Y(n-1) is the output value of the previous filter, and a is the filter coefficient.

[0080] After processing, it is converted into a time domain signal through inverse FFT. Figure 4 shown.

[0081] Sound noise reduction: First-order filtering is used for special scenarios in prisons. First-order filtering is a traditional noise reduction algorithm with high requirements for the environment. However, in the present invention, due to the high discipline requirements in prisons, they are usually relatively quiet. If an abnormality occurs, it is often accompanied by abnormal sounds, and the abnormal sounds will be significantly increased in the sound waves.

[0082] Therefore, first-order filtering is used for noise reduction, which is more suitable for prison scenes, while retaining the audio that needs to be classified and analyzed, making subsequent recognition more accurate.

[0083] S3. Extract MFCC features from the noise-reduced sound signal, process and analyze it through a CNN model, and obtain separated human voice signals and / or environmental sound signals and sound classification results.

[0084] MFCC+CNN is used to classify human voices and environmental sounds in prison. According to the prison scene, environmental sounds mainly include abnormal sounds such as fighting, collision, and friction, while human voices mainly include exhalation, snoring, and conversation.

[0085] S31, extracting the MFCC features of the noise-reduced sound to obtain prison sound features.

[0086] S311, performing data preprocessing on the noise-reduced sound to obtain a preprocessed signal;

[0087] S312, performing fast Fourier transform on the preprocessed signal to obtain a spectrum diagram;

[0088] S313, mapping the spectrogram to a Mel-scale spectrogram using overlapping triangular windows whose center frequencies are distributed on a Mel-scale, to obtain a mapped Mel-scale spectrogram;

[0089] S314, performing a logarithmic operation on the mapped Mel spectrum to obtain a Mel logarithmic power spectrum;

[0090] S315, performing discrete cosine transform on the Mel log power spectrum to obtain MFCC features.

[0091] The MFCC features of the denoised sound are extracted based on the following formula:

[0092]

[0093] Where M is the number of triangular filters, X m is the logarithmic energy of the mth log Mel-spectrogram, and c is the index of the cepstral coefficient.

[0094] S32. Determine the sound classification in the prison scene based on the prison sound features, construct and train a six-layer CNN model, and obtain human voice signals and / or environmental sound signals separated according to the sound classification in the prison scene.

[0095] For the current prison scene sound classification, we provide a six-layer CNN model as follows Figure 5 shown.

[0096] The six-layer CNN model consists of six sequentially connected convolutional layers, two fully connected layers, and one output layer.

[0097] The 6 convolutional layers are divided into 3 groups, each of which uses a 3×3 receptive field and a convolution operation with a stride of 1×1;

[0098] The six convolutional layers all use BN and ReLUs activation functions;

[0099] The first group consists of the first and second convolutional layers, each using 32 kernels, and a maximum pooling layer with a stride of 2×2 and a dropout layer after the second convolutional layer;

[0100] The second group consists of the third and fourth convolutional layers, each using 64 kernels, and a maximum pooling layer with a stride of 2×2 and a dropout layer after the fourth convolutional layer;

[0101] The third group consists of the fifth and sixth convolutional layers, each using 128 kernels, and a maximum pooling layer with a stride of 2×2 and a dropout layer after the sixth convolutional layer;

[0102] The two fully connected layers are each composed of 1024 neurons, forming a classifier.

[0103] The data output by the last convolutional layer passes through the maximum pooling layer and the dropout operation, and is then input into the classifier consisting of two fully connected layers;

[0104] The two fully connected layers are composed of 1024 neurons each;

[0105] The output layer is composed of multiple neurons, corresponding to multiple sound classifications in training.

[0106] The output layer consists of 67 neurons, representing the 67 environmental sound categories in training.

[0107] Prison environment sound separation, the present invention collects 67 sounds related to fighting, collision and friction in prison scenes to make the separation more accurate.

[0108] Due to the particularity of the prison scene, the abnormal problems that may occur are more hidden, but because the daily supervision is relatively strict, the classification scenes of the prison are more focused. Based on this, we proposed a 6-layer CNN model that can clarify the usage scenarios and effectively control the parameter range.

[0109] The 6 layers are 1. input layer 2. convolution layer 3. ReLUs layer (excitation layer) 4. maximum pooling layer 5. fully connected layer 6. output layer.

[0110] Quantized data is obtained through the input layer, and then features are extracted through the convolution layer. The excitation layer then performs nonlinear mapping on the output of the convolution layer. In this step, we use ReLUs to avoid the problem of gradient disappearance. The pooling part acts like a filter. After pooling, it is passed to the fully connected layer (classifier) ​​for data classification and final output.

[0111] The reason why the number of convolution kernel settings continues to increase is that the deeper the network, the more abstract the features extracted. Therefore, by increasing the number of convolution kernels, the features learned in the previous layers are combined, so the possibility of various features is covered more comprehensively.

[0112] S4. If there is a human voice in the sound classification result, perform voiceprint recognition on the separated human voice signal to obtain a suspicious person;

[0113] S41. If there is a human voice in the sound classification result, use the ECAPA-TDNN scheme to perform prison-wide screening and small room matching;

[0114] S42. Determine the highest-scoring personnel based on the matching results, and increase the weighted scores based on the personnel's actions in the environment. The scoring formula is:

[0115] S=0.68T+0.2R+0.12F

[0116] In the formula, S is the total score, T is the algorithm score, R indicates whether the person exists in the room where the microphone is located, and F indicates whether the person exists on the current floor.

[0117] Voiceprint recognition. The voice separated in step 3 is used for comparison with the voices of the people in the prison. The voiceprint recognition adopts the ECAPA-TDNN solution. In order to increase the accuracy of recognition, the whole prison screening and small room matching method is adopted. Through the 1:n matching of the whole prison, the top 5 scoring people are obtained. The weighted score is increased according to the people living in the room and the people who may pass through the room.

[0118] It should be noted that ECAPA-TDNN is a speech recognition model based on the combination of TDNN (Time Delay Neural Network) and ECAPA (Extended Context-Aware Parallel Attention), and has performed well in the field of speech recognition.

[0119] The scoring weight setting is determined by the business scenario. This is a scoring standard for the scenario. The prison is a closed space where the flow of personnel is strictly controlled, so the present invention has a spatial concept. Based on the 68% of the algorithm score, the personnel space location score is added, accounting for 0.32%, 0.2 for the room, and 0.12 for the floor.

[0120] For example, on the first floor, room A has three people, A, B, and C. Room B has D, and floor two has E. In daily management, E will not appear on the first floor, let alone in room A. Now A in room A imitates E's voice and shouts, triggering an alarm. Then the algorithm first gets a score of 0.91 for E and 0.85 for A based on the human voice. Because the imitation is more similar, the algorithm recognizes that the first person is E, but after this algorithm, the final score is 0.6188 for E and 0.898 for A, which is obviously a better match.

[0121] S5. Perform sound source localization according to the relative position of the microphone, the separated human voice signal and / or the ambient sound signal to determine the sound source position.

[0122] Sound source localization: Localize the sound source based on the ambient sound or human voice recognized by S4.

[0123] First, the environmental space is segmented in S1, and the specific location of the sound source is inferred through the MUSIC algorithm.

[0124] The specific formula is as follows:

[0125]

[0126] Where: That is the noise subspace, where a(θ) is the direction vector of the array element, N is the noise matrix, and the two dimensions are the number of array elements and the number of sampling points.

[0127] The position of the sound source relative to the microphone is calculated by MUCIS. Then the position of the sound source in the small room is obtained based on the closed space coordinates of the microphone. Figure 3 As shown, let the origin of room A be (0, 0, 0), the coordinates of the microphone be (x, y, z), and calculate the relative position of the sound source (x1, y1, z1) according to MUSIC. However, room A is a bounded space.

[0128] So we get the following function

[0129]

[0130]

[0131] Calculate the location of the sound source.

[0132] Furthermore, the present invention can locate events according to the location of the sound source, and also push the specific locations of abnormal events and persons involved in the events to supervisors, and mark the relevant locations through spatial modeling, so that prison guards can handle events more accurately and efficiently.

[0133] S6. Confirm the event using the suspicious person, the sound classification result, the sound source location and the sound source occurrence time, connect the timeline and the event line, generate the prison event of the day and output an automated inspection report.

[0134] Event space analysis, based on the timeline and event line, connects the prison events of the day. Provides an automated inspection report. When abnormal behavior occurs, space, events, and timelines are used to analyze the behavior in the last three days, seven days, and one month. For example, event analysis will show that a prisoner's room was dug for n hours on that day. Whether the prison guards intervened and the results of the intervention. Or whether there was a conspiracy to escape in the prison and who were the participants.

[0135] In addition, after a violation occurs, supervisors can use the report generated by the present invention as daily records to investigate the entire life cycle of the problem from the trigger to the outcome of the incident, so as to avoid similar incidents next time.

[0136] S7. Analyze the historical inspection data to determine the characteristics of the suspicious person;

[0137] S8. Predict the event based on the personnel characteristics, and if it is likely to develop into a serious event, send a warning notification to the prison guard.

[0138] Prediction analysis is based entirely on historical data. The detailed logic is as follows: If events A, B, and C occur, which are similar disciplinary violations, the prediction logic will find five steps forward based on the timeline of such events within one hour, and analyze the behavioral events. Then, the same events will be found. When the number of identical events is greater than or equal to 75%, the prediction logic will be generated. If beating is a disciplinary violation, the system will find that each beating event is accompanied by 1. talking loudly 2. insulting 3. pushing sounds based on historical beating events. Then, when 1.2.3 behaviors occur, a prediction warning will be issued.

[0139] For example, in the historical records, if A bullies others in prison, the order of bullying behavior is usually 1. talking loudly -> 2. insulting -> 3. pushing -> 4. beating. Then, in the third link, the prediction analysis will send a notification to the early warning system for intervention. This helps the prison solve the problem from the root and improve the efficiency of prison discipline.

[0140] This embodiment is a prison inspection method, including constructing a prison space coordinate system by modeling the prison structure, locating the relative position of the microphone in the prison space coordinate system, obtaining the sound signal collected by the microphone and performing first-order filtering and noise reduction processing to obtain the noise-reduced sound signal, extracting MFCC features from the noise-reduced sound signal, processing and analyzing through a CNN model, obtaining separated human voice signals and / or environmental sound signals and sound classification results, if there is a human voice in the sound classification result, then performing voiceprint recognition on the separated human voice signal to obtain a suspicious person, locating the sound source according to the relative position of the microphone, the separated human voice signal and / or environmental sound signal, determining the sound source position, confirming the event using the suspicious person, the sound classification result, the sound source position and the time when the sound source occurred, performing a series connection of the timeline and the event line, generating the prison event of the day and outputting an automated inspection report. The present invention is established by modeling and spatial coordinates. The location of the event can be accurately identified in real time, allowing prison guards to quickly check and handle it. The key disciplinary incidents are traced back. After a disciplinary problem occurs, the entire life cycle of the problem can be investigated through daily records. Accurate positioning, prediction and traceback improve the efficiency of prison discipline inspections.

[0141] Embodiment 2

[0142] An embodiment of the present invention provides a prison inspection device 200, such as Figure 3 As shown, including:

[0143] A microphone positioning module 201 is used to construct a prison space coordinate system by modeling the prison structure, and locate the relative position of the microphone in the prison space coordinate system;

[0144] A noise reduction module 202 is used to obtain the sound signal collected by the microphone and perform first-order filtering and noise reduction processing to obtain a noise-reduced sound signal;

[0145] The sound separation module 203 is used to extract MFCC features from the noise-reduced sound signal, process and analyze it through a CNN model, and obtain a separated human voice signal and / or environmental sound signal and a sound classification result;

[0146] The voiceprint recognition module 204 is used to perform voiceprint recognition on the separated human voice signal to obtain a suspicious person if there is a human voice in the sound classification result;

[0147] A sound source confirmation module 205 is used to locate the sound source according to the relative position of the microphone, the separated human voice signal and / or the ambient sound signal, and determine the position of the sound source;

[0148] The report generation module 206 is used to confirm the event by using the suspicious person, the sound classification result, the sound source location and the sound source occurrence time, connect the time line and the event line, generate the prison event of the day and output an automated inspection report.

[0149] This embodiment is a prison inspection device, which includes a microphone positioning module, which is used to construct a prison space coordinate system by modeling the prison structure, and locate the relative position of the microphone in the prison space coordinate system; a noise reduction module, which is used to obtain the sound signal collected by the microphone and perform first-order filtering and noise reduction processing to obtain the noise-reduced sound signal; a sound separation module, which is used to extract MFCC features from the noise-reduced sound signal, and process and analyze it through a CNN model to obtain a separated human voice signal and / or environmental sound signal and a sound classification result; a voiceprint recognition module, which is used to perform voiceprint recognition on the separated human voice signal if there is a human voice in the sound classification result to obtain a suspicious person; a sound source confirmation module, which is used to locate the sound source according to the relative position of the microphone, the separated human voice signal and / or environmental sound signal, and determine the sound source position; a report generation module, which is used to confirm the event using the suspicious person, the sound classification result, the sound source position and the sound source occurrence time, and perform a series connection of the timeline and the event line to generate the prison event of the day and output an automatic inspection report. Through prison modeling, and spatial coordinates are established. It can accurately identify the location of the incident in real time, allowing prison guards to quickly check and deal with it. Focus on disciplinary incidents. After a disciplinary problem occurs, the entire life cycle of the problem can be investigated through daily records. Accurate positioning, prediction and tracing improve the efficiency of prison discipline inspections.

[0150] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A prison inspection method, characterized in that: include: S1. constructing a prison space coordinate system by modeling the prison structure, and locating the relative position of the microphone in the prison space coordinate system; S2, obtaining the sound signal collected by the microphone and performing first-order filtering and noise reduction processing to obtain a noise-reduced sound signal; S3, extracting MFCC features from the noise-reduced sound signal, processing and analyzing it through a CNN model, and obtaining a separated human voice signal and / or environmental sound signal and a sound classification result; S4. If there is a human voice in the sound classification result, perform voiceprint recognition on the separated human voice signal to obtain a suspicious person; S5, performing sound source localization according to the relative position of the microphone, the separated human voice signal and / or the ambient sound signal to determine the sound source position; S6. Confirm the event by using the suspicious person, the sound classification result, the sound source location and the sound source occurrence time, connect the timeline and the event line, generate the prison event of the day and output an automated inspection report; The S1 includes: S11, performing spatial modeling based on the prison structure to obtain a prison spatial model; S12, establishing a spatial coordinate system according to the prison space model, and dividing the space according to the space; S13, based on the segmented space, establishing room coordinates for each room, and locating the relative position of the microphone in the space coordinate system; The S4 includes: S41. If there is a human voice in the sound classification result, use the ECAPA-TDNN scheme to perform prison-wide screening and small room matching; S42. Determine the highest-scoring personnel based on the matching results, and increase the weighted scores based on the personnel's actions in the environment. The scoring formula is: S=0.68T+0.2R+0.12F; In the formula, S is the total score, T is the algorithm score, R indicates whether the person exists in the room where the microphone is located, and F indicates whether the person exists on the current floor.

2. The prison inspection method according to claim 1, characterized in that: Also includes: S7. Analyze the historical inspection data to determine the characteristics of the suspicious person; S8. Predict the event based on the personnel characteristics, and if it is likely to develop into a serious event, send a warning notification to the prison guard.

3. The prison inspection method according to claim 1, characterized in that: The S2 includes: S21, obtaining the sound signal collected by the microphone, and converting it into a frequency domain signal through FFT; S22, based on the frequency domain signal, processing it through a low-pass first-order filter to obtain a processed signal; S23, performing inverse FFT on the processed signal to convert it into a time domain signal to obtain the noise-reduced sound.

4. The prison inspection method according to claim 1, characterized in that: The S3 includes: S31, extracting the MFCC features of the noise-reduced sound to obtain prison sound features; S32. Determine the sound classification in the prison scene based on the prison sound features, construct and train a six-layer CNN model, and obtain human voice signals and / or environmental sound signals separated according to the sound classification in the prison scene.

5. The prison inspection method according to claim 4, characterized in that: The S31 includes: S311, performing data preprocessing on the noise-reduced sound to obtain a preprocessed signal; S312, performing fast Fourier transform on the preprocessed signal to obtain a spectrum diagram; S313, mapping the spectrogram to a Mel-scale spectrogram using overlapping triangular windows whose center frequencies are distributed on a Mel-scale, to obtain a mapped Mel-scale spectrogram; S314, performing a logarithmic operation on the mapped Mel spectrum to obtain a Mel logarithmic power spectrum; S315, performing discrete cosine transform on the Mel log power spectrum to obtain MFCC features.

6. The prison inspection method according to claim 4, characterized in that: The six-layer CNN model includes six convolutional layers, two fully connected layers and one output layer connected in sequence. The six convolutional layers are divided into three groups, each of which uses a 3×3 receptive field and a convolution operation with a step size of 1×1; The first group consists of the first and second convolutional layers, each using 32 kernels, and a maximum pooling layer and a dropout layer with a stride of 2×2 are applied after the second convolutional layer; The second group consists of the third and fourth convolutional layers, each using 64 kernels, and a maximum pooling layer with a stride of 2×2 and a dropout layer after the fourth convolutional layer; The third group consists of the fifth and sixth convolutional layers, each using 128 kernels, and a maximum pooling layer and a dropout layer with a stride of 2×2 are applied after the sixth convolutional layer; The two fully connected layers are each composed of 1024 neurons, forming a classifier; The output layer is composed of multiple neurons, corresponding to multiple sound classifications in training.

7. The prison inspection method according to claim 4, characterized in that: In the S31, The MFCC features of the denoised sound are extracted based on the following formula: ； In the formula, M is the number of triangular filters, X m is the logarithmic energy of the mth log Mel-spectrogram, c is the index of the cepstral coefficient.

8. A prison inspection device, characterized in that: include: A microphone positioning module, used to construct a prison space coordinate system by modeling the prison structure, and locate the relative position of the microphone in the prison space coordinate system; A noise reduction module, used to obtain the sound signal collected by the microphone and perform first-order filtering and noise reduction processing to obtain a noise-reduced sound signal; A sound separation module is used to extract MFCC features from the noise-reduced sound signal, process and analyze it through a CNN model, and obtain a separated human voice signal and / or environmental sound signal and a sound classification result; A voiceprint recognition module, configured to perform voiceprint recognition on the separated human voice signal to identify a suspicious person if there is a human voice in the sound classification result; A sound source confirmation module, used to locate the sound source according to the relative position of the microphone, the separated human voice signal and / or the ambient sound signal, and determine the position of the sound source; A report generation module, used to confirm the event by using the suspicious person, the sound classification result, the sound source location and the sound source occurrence time, connect the timeline and the event line, generate the prison event of the day and output an automated inspection report; The pickup positioning module is used to perform the following steps: S11, performing spatial modeling based on the prison structure to obtain a prison spatial model; S12, establishing a spatial coordinate system according to the prison space model, and dividing the space according to the space; S13, based on the segmented space, establishing room coordinates for each room, and locating the relative position of the microphone in the space coordinate system; The voiceprint recognition module is used to perform the following steps: S41. If there is a human voice in the sound classification result, use the ECAPA-TDNN scheme to perform prison-wide screening and small room matching; S42. Determine the highest-scoring personnel based on the matching results, and increase the weighted scores based on the personnel's actions in the environment. The scoring formula is: S=0.68T+0.2R+0.12F; In the formula, S is the total score, T is the algorithm score, R indicates whether the person exists in the room where the microphone is located, and F indicates whether the person exists on the current floor.

Images