Public place intelligent public security management system, device and method

Abstract

The invention discloses an intelligent public security management system, device and method for public places, and relates to the technical field of urban public security. Comprising the following steps: processing thermal infrared images of a plurality of public places through a preprocessing technology, and manually marking a human target to generate a training sample data set; and establishing a neural network model based on the data set, and carrying out people flow prediction. And continuously collecting and processing thermal infrared images of a to-be-monitored area, inputting the thermal infrared images into the prediction model, outputting the number of human targets, and calculating a crowd movement rate variance through the images at different moments. And collecting sound characteristic parameters, and calculating a sound abnormity evaluation index. And generating a behavior evaluation index by combining the number of human targets and the movement rate variance, and finally comprehensively obtaining an abnormal alarm index. The index is compared with a set abnormity judgment threshold value, and abnormity judgment is sent out according to a result, so that intelligent monitoring and management of public places are realized, and the safety management level of the public places is improved.

Description

Technical Field

[0001] The present invention relates to the field of urban public security technology, and specifically to a smart public security management system, device and method for public places. Background Art

[0002] With the rapid development of urbanization, the flow of people in public places is increasing, and public security management is facing unprecedented challenges. Traditional monitoring systems mainly rely on video surveillance technology, which collects real-time images by installing cameras and relies on manual monitoring and analysis. However, this method has serious limitations. First, in crowded environments, manual monitoring is not only difficult to fully cover all areas, but also prone to information omissions and slow responses. Second, video surveillance systems perform poorly under complex lighting conditions (such as at night or backlight), which may make it difficult to identify targets and fail to detect abnormal conditions in a timely manner.

[0003] In addition, existing technologies generally lack intelligent means for analyzing human behavior. Although some systems have basic motion detection functions, they are still insufficient in identifying the dynamic behavior, density and potential dangers of the crowd. Especially in the event of an emergency, traditional monitoring systems are often unable to quickly judge the situation, resulting in delayed emergency response and increased safety risks. Therefore, how to improve the level of public security management in public places through more advanced technologies has become an important issue that needs to be solved urgently.

[0004] In this context, thermal infrared imaging technology has gradually gained attention because of its ability to effectively monitor targets under various lighting conditions. The advantage of thermal infrared images is that they are sensitive to temperature changes and can clearly identify and track human targets. However, current thermal infrared monitoring technology still has some shortcomings, such as the lack of accurate crowd counting and behavior analysis capabilities, and usually fails to combine sound features with video surveillance for comprehensive evaluation. Therefore, an innovative solution is urgently needed to combine thermal infrared imaging with sound feature analysis to improve the level of smart security management in public places.

[0005] In the prior art, the publication number CN119251999A discloses a warning method, system, product and camera for abnormal events in public places. The method includes: monitoring the audio in public places, and when an abnormal sound appears, determining the sound source position corresponding to the audio; adjusting the preset camera to turn to the sound source position, collecting the image information and sound information at the sound source and transmitting them to the processor; the processor extracts the depth feature information of the sound information and inputs it into the public place abnormal sound discrimination network to determine whether the sound information is an abnormal sound in public places. If so, judging the category of the abnormal sound in public places; uploading the category of the abnormal sound in public places and the audio-visual data to the cloud platform to issue a warning prompt and push the police situation. However, the abnormal warning method that only relies on audio monitoring in this solution may miss important visual information. The situation in public places is often complex, and single sound monitoring may not provide enough background information or context. For example, some abnormal sound sources may not be visually obvious or are difficult to distinguish due to environmental factors (such as other noises). Therefore, the method that only relies on audio monitoring may have limitations in complex environments. This warning method mainly focuses on abnormal situations of sounds and fails to comprehensively analyze the dynamic situation by combining the behavioral characteristics of the crowd. It is impossible to extract valuable information from the behavior patterns and only relies on sound monitoring, unable to capture these important behavioral changes, thus affecting the warning ability of abnormal events. Therefore, the accuracy and effectiveness of the supervision system are reduced.

[0006] The above information disclosed in the background art section is only used to enhance the understanding of the background of the present disclosure, and thus it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0007] The purpose of the present invention is to provide a smart public security management system, device and method for public places to solve the problems raised in the above background art.

[0008] To achieve the above purpose, the present invention provides the following technical solutions:

[0009] A smart public security management system for public places specifically includes:

[0010] A training sample acquisition module, which is used to obtain thermal infrared images of several different public places, preprocess the collected thermal infrared images, and mark all human targets in the preprocessed thermal infrared images by manual marking. The marked thermal infrared images are recorded as training sample images, and the number of human targets corresponding to the training sample images is recorded. The preprocessing includes thermal infrared image denoising and enhancement processing;

[0011] A prediction model training module, which is used to map training sample images and the corresponding number of human targets one by one to generate a training sample data set. Based on the data in the training sample data set, a convolutional neural network model is established. The training sample images in the training sample data set are used as the input of the model, and the neural network model is trained with the human target annotations and the corresponding number of human targets in the training sample images as labels to obtain a pedestrian flow prediction model;

[0012] A feature parameter acquisition module, which is used to continuously collect thermal infrared images at different times within the current time period in a public place to be monitored. After preprocessing the collected thermal infrared images of the public place to be monitored, they are input into the trained pedestrian flow prediction model. The model marks the human targets in the images and outputs the predicted values of the corresponding number of human targets. Based on the marked images at different times, the variance of the movement speed of the crowd is obtained. At the same time, the sound feature parameters corresponding to the public place to be monitored at the corresponding time are collected. The sound feature parameters include the sound pressure, frequency, and spectral entropy of the sound signal;

[0013] An abnormal situation analysis module, which is used to calculate and generate a behavior evaluation index according to the predicted value of the number of human targets and the variance of the movement speed, calculate and generate a sound abnormality evaluation index according to the sound feature parameters corresponding to the public place to be monitored at the corresponding time, and comprehensively generate an abnormal alarm index based on the sound abnormality evaluation index and the behavior evaluation index;

[0014] A public security management decision-making module, which is used to compare the obtained abnormal alarm index with the set abnormal judgment threshold, issue corresponding abnormal judgment results according to different comparison results, and take corresponding measures according to the abnormal judgment results to complete the intelligent public security management of public places.

[0015] Furthermore, preprocess each collected thermal infrared image of the public place. The preprocessing includes: image enhancement and denoising preprocessing. Among them, the wavelet transform denoising method is used to denoise each thermal infrared image of the public place, and bilateral filtering is used to perform image enhancement preprocessing on each thermal infrared image of the public place;

[0016] The generation method of the training sample data set is: map the training sample images and the corresponding number of human targets one by one to form corresponding grids, and record the formed grids as the training sample data set.

[0017] Furthermore, based on the convolutional neural network, a pedestrian flow prediction model is established. The convolutional neural network consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The activation function in the convolutional layer is function, The specific expression of the function is:

[0018]

[0019] Among them, represents the th corresponding convolutional layer, then represents the th eigenvalue of the th training sample image in the th corresponding convolutional layer, where is the index of the convolutional layer, is the index of the training sample image, is the index of the eigenvalue in the training sample image;

[0020] For the fully connected layer, set the number of neurons in the fully connected layer to 32, the initial learning rate of the neural network to 0.001, and the number of training epochs to 300;

[0021] The input of the trained pedestrian flow prediction model is the thermal infrared image of a public place, and the output is the human target annotation in the image and the corresponding number of human targets.

[0022] Furthermore, based on the labeled images at different times, obtain the variance of the movement speed of the crowd. The method for obtaining the variance of the movement speed of the crowd is as follows: taking the center of the image captured by the thermal infrared camera as the origin, the due north direction as the positive x-axis direction, and the due west direction perpendicular to the x-axis as the y-axis, establish a plane coordinate system of the public place to be monitored. Randomly select a number of human targets, obtain the displacements of the selected human targets within the acquisition time interval, and calculate the variance of the movement speed according to the displacements. The formula for calculating the variance of the movement speed of the crowd is as follows:

[0023]

[0024] In the formula, is the variance of the movement speed of the crowd in the current time period, j is the index of different times in the current time period, is the total number of times in the current time period, where , is the average movement speed of the crowd in the current time period, is the average movement speed of the crowd at the jth time in the current time period; where and The formulas for calculation are as follows:

[0025]

[0026]

[0027] In the formula, is the total number of selected human targets, is the movement speed of the i-th selected human target at the j-th moment, where i is the index of the selected human target, , where the movement speed of the i-th selected human target is calculated from the displacement data, and the specific calculation formula is:

[0028]

[0029] In the formula, is the displacement of the i-th selected human target at the j-th moment from the previous acquisition moment, is the time interval between the acquisition moment and the previous acquisition moment, where the displacement of the i-th selected human target at the j-th moment from the previous acquisition moment is calculated according to the formula:

[0030]

[0031] In the formula, and are the abscissa and ordinate of the location of the i-th selected human target at the j-th moment respectively, and are the abscissa and ordinate of the location of the i-th selected human target at the (j - 1)-th moment respectively.

[0032] Furthermore, based on the predicted value of the number of human targets obtained and combined with the variance of the movement speed, a behavior evaluation index is calculated, and the specific formula for calculating the behavior evaluation index is:

[0033]

[0034] In the formula, is the behavior evaluation index, is the predicted value of the number of human targets in the public place to be monitored, is the reference value of the number of human targets set for the public place to be monitored, where ;

[0035] According to the sound characteristic parameters collected at the corresponding moment in the public place to be monitored, a sound anomaly evaluation index is calculated, and the specific formula for calculating the sound anomaly evaluation index is:

[0036]

[0037] In the formula, is the sound anomaly evaluation index, is the spectral entropy of the sound signal, is the frequency of the sound signal, is the sound pressure level of the sound signal, where the sound pressure level of the sound signal Calculate based on the sound pressure of the sound signal, and the specific formula is as follows:

[0038]

[0039] In the formula, is the sound pressure of the sound signal, is the reference sound pressure.

[0040] Furthermore, based on the sound anomaly evaluation index and the behavior evaluation index, an anomaly alarm index is comprehensively generated. The formula for calculating the anomaly alarm index is as follows:

[0041]

[0042] In the formula, is the anomaly alarm index, and are the weight coefficients of the behavior evaluation index and the sound anomaly evaluation index respectively, where , and and are both greater than 0.

[0043] Furthermore, compare the anomaly alarm index with the set anomaly judgment threshold, and issue corresponding anomaly judgment results according to different comparison results. The specific judgment logic is as follows:

[0044] When , it is judged that an abnormal situation has occurred in the public place to be monitored, and measures such as alarm and notification to relevant departments should be taken;

[0045] When , it is judged that there is a risk of an abnormal situation occurring in the public place to be monitored, and surveillance of this area should be strengthened and early warning should be given;

[0046] When , it is judged that there is no abnormal situation in the public place to be monitored, and no relevant management is required;

[0047] Where is the anomaly judgment threshold, which is dynamically corrected through environmental parameters. The environmental parameters include the average environmental wind speed and relative humidity. Among them, the anomaly judgment threshold The formula for calculation is as follows:

[0048]

[0049] In the formula, is the initial value of the anomaly judgment threshold, is the average environmental wind speed, is the environmental relative humidity, is the reference humidity, is the reference wind speed.

[0050] The present invention also provides a smart public security management device for public places. The smart public security management device for public places includes one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the smart public security management system as described above.

[0051] The present invention also provides a smart public security management method for public places. The smart public security management method for public places is used to control the smart public security management system as described above. The specific steps include:

[0052] Obtain thermal infrared images of several different public places, preprocess the collected thermal infrared images, and by means of manual marking, label all human targets in the preprocessed thermal infrared images. Denote the labeled thermal infrared images as training sample images, and record the number of human targets corresponding to the training sample images. The preprocessing includes thermal infrared image denoising and enhancement processing;

[0053] Map the training sample images and the number of human targets corresponding to the images one by one to generate a training sample data set. Based on the data in the training sample data set, establish a convolutional neural network model. Use the training sample images in the training sample data set as the input of the model, and use the human target labels and the corresponding number of human targets in the training sample images as labels to train the neural network model to obtain a crowd flow prediction model;

[0054] In the public place to be monitored, continuously collect thermal infrared images at different times within the current time period. After preprocessing the collected thermal infrared images of the public place to be monitored, input them into the trained crowd flow prediction model. The model marks the human targets in the images and outputs the predicted value of the corresponding number of human targets. Obtain the variance of the movement speed of the crowd based on the marked images at different times. At the same time, collect the sound feature parameters corresponding to the public place to be monitored at the corresponding time. The sound feature parameters include the sound pressure, frequency, and spectral entropy of the sound signal;

[0055] Calculate and generate a behavior evaluation index according to the obtained predicted value of the number of human targets combined with the variance of the movement speed. Calculate and generate a sound anomaly evaluation index according to the sound feature parameters collected at the corresponding time of the public place to be monitored. Based on the sound anomaly evaluation index and the behavior evaluation index, comprehensively generate an anomaly alarm index;

[0056] According to the obtained anomaly alarm index, compare the anomaly alarm index with the set anomaly judgment threshold. According to different comparison results, issue the corresponding anomaly judgment result, and take corresponding measures according to the anomaly judgment result to complete the smart public security management of public places.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] First, by establishing a high-quality training sample data set and using a pedestrian flow prediction model established by deep learning technology, the recognition accuracy of human targets is greatly improved. This method not only improves the accuracy of pedestrian flow statistics, but also can quickly reflect the dynamic changes of the crowd, thus providing real-time and reliable data support for public security management. Secondly, through crowd behavior analysis and combined with the analysis of sound feature parameters, the evaluation of crowd behavior becomes more comprehensive and flexible. By monitoring sound pressure, frequency, and spectral entropy, abnormal sound changes in the environment can be identified, which provides an important basis for judging the state of the crowd and potential safety risks. It makes up for the shortcoming of the traditional monitoring system's insufficient response to emergencies. Finally, by comparing the comprehensively generated abnormal alarm index with the set abnormal judgment threshold, the present invention can achieve an intelligent abnormal judgment and response mechanism. According to different comparison results, the system can automatically take corresponding measures, such as issuing warnings, notifying relevant security personnel, or triggering emergency plans. This flexible response method improves the safety management level of public places. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a schematic diagram of the overall system structure of the present invention;

[0060] Figure 2 is a schematic diagram of the overall method flow of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to specific embodiments.

[0062] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meaning understood by those of ordinary skill in the field to which the present invention belongs. The "first", "second", and similar terms used in the present invention do not represent any order, quantity, or importance, but are only used to distinguish different components. The terms "including" or "comprising" and the like mean that the elements or objects appearing before the term cover the elements or objects listed after the term and their equivalents, without excluding other elements or objects. The terms "connected" or "coupled" and the like are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0063] Embodiment:

[0064] Please refer toFigure 1 , the present invention provides a technical solution:

[0065] An intelligent public security management system for public places, specifically including:

[0066] A training sample acquisition module, which is used to obtain thermal infrared images of several different public places, preprocess the collected thermal infrared images, and mark all human targets in the preprocessed thermal infrared images by manual marking. The marked thermal infrared images are recorded as training sample images, and the number of human targets corresponding to the training sample images is recorded. The preprocessing includes thermal infrared image denoising and enhancement processing.

[0067] Among them, the LabelImg tool is used to mark the bounding boxes of human targets; the coordinates of the minimum circumscribed rectangle of the marked human targets are obtained, including the coordinates of the upper left corner boundary point and the lower right corner boundary point of the minimum circumscribed rectangle; the thermal infrared image of the human target to be recognized containing the complete human target image is cropped according to the coordinates of the minimum circumscribed rectangle of the marked human targets.

[0068] Each thermal infrared image of a public place collected is preprocessed, where the preprocessing includes: image enhancement and denoising preprocessing. Among them, the wavelet transform denoising method is used to denoise each thermal infrared image of a public place, and bilateral filtering is used to preprocess the image enhancement of each thermal infrared image of a public place;

[0069] The wavelet transform denoising method is used to denoise the thermal infrared image of a public place. The specific steps of the wavelet transform denoising method include: decomposing the thermal infrared image of a public place through wavelet transform to obtain wavelet coefficients of the image at different scales and directions; performing threshold processing on the wavelet coefficients, setting the low-amplitude wavelet coefficients to zero, and retaining the high-amplitude wavelet coefficients; performing inverse transform on the wavelet coefficients after threshold processing to reconstruct the processed coefficients into an image, completing the image denoising process;

[0070] Bilateral filtering is used to enhance the details of the thermal infrared image of a public place. The formula based on the specific filtering transformation is:

[0071]

[0072] In the formula, is the coordinate vector in the image coordinate system, is the coordinate vector at the gray value, is the gray value after bilateral filtering transformation, are all Gaussian functions, where The formula based on is:

[0073]

[0074]

[0075] In the formula, is the coordinate vector in the image coordinate system, is the coordinate vector at the gray value, and are respectively standard deviation.

[0076] The generation method of the training sample data set is as follows: The training sample images are mapped one by one with the corresponding number of human targets to form corresponding grids, and the formed grids are recorded as the training sample data set.

[0077] The prediction model training module is used to map the training sample images one by one with the number of human targets corresponding to the images to generate a training sample data set. Based on the data in the training sample data set, a convolutional neural network model is established. The training sample images in the training sample data set are used as the input of the model, and the neural network model is trained with the human target annotations and the corresponding number of human targets in the training sample images as labels to obtain a pedestrian flow prediction model.

[0078] Based on the convolutional neural network, a pedestrian flow prediction model is established. The convolutional neural network consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The activation function in the convolutional layer is function, The specific expression of the function is:

[0079]

[0080] Among them, represents the corresponding convolutional layer, then represents the in the th training sample image of the th eigenvalue, where is the index of the convolutional layer, is the index of the training sample image, is the index of the eigenvalue in the training sample image;

[0081] For the fully connected layer, set the number of neurons in the fully connected layer to 32, the initial neural network learning rate to 0.001, and the number of training rounds to 300;

[0082] The input of the completed training pedestrian flow prediction model is the thermal infrared image of a public place, and the output is the annotation of human targets in the image and the corresponding number of human targets.

[0083] The feature parameter acquisition module is used to continuously collect thermal infrared images at different times within the current time period in the public place to be monitored. After preprocessing the collected thermal infrared images of the public place to be monitored, it is input into the completed training pedestrian flow prediction model. The model marks the human targets in the image and outputs the predicted value of the corresponding number of human targets. Based on the marked images at different times, the variance of the movement speed of the crowd is obtained. At the same time, the sound feature parameters corresponding to the public place to be monitored at the corresponding time are collected. The sound feature parameters include the sound pressure, frequency, and spectral entropy of the sound signal.

[0084] The preprocessing of the collected thermal infrared images of the public place to be monitored is the same as the above preprocessing method and will not be elaborated here.

[0085] Based on the marked images at different times, the variance of the movement speed of the crowd is obtained. The method for obtaining the variance of the movement speed of the crowd is as follows: Taking the center of the image captured by the thermal infrared camera as the origin, taking the due north direction as the positive x-axis direction, and taking the due west direction perpendicular to the x-axis direction as the y-axis, a plane coordinate system of the public place to be monitored is established. Randomly select several human targets, obtain the displacements of the selected human targets within the acquisition time interval, and calculate the variance of the movement speed according to the displacements. The formula based on which the variance of the movement speed of the crowd is calculated is:

[0086]

[0087] In the formula, The variance of the movement speed of the crowd within the current time period, j is the index of different times within the current time period, is the total number of times within the current time period, where , is the average movement speed of the crowd within the current time period, is the average movement speed of the crowd at the j-th time within the current time period; where and The formulas based on which they are calculated are:

[0088]

[0089]

[0090] In the formula, is the total number of selected human targets, is the movement speed of the i-th selected human target at the j-th time, where i is the index of the selected human target, , where the movement speed of the i-th selected human target Calculated from displacement data, and the specific calculation formula is as follows:

[0091]

[0092] In the formula, is the displacement of the i-th selected human target at the j-th moment from the previous acquisition moment, is the time interval between the acquisition moment and the previous acquisition moment, where the displacement of the i-th selected human target at the j-th moment from the previous acquisition moment The calculation formula is as follows:

[0093]

[0094] In the formula, and are the abscissa and ordinate of the location of the i-th selected human target at the j-th moment respectively, and are the abscissa and ordinate of the location of the i-th selected human target at the (j - 1)-th moment respectively.

[0095] Among them, represents a total of moments are collected, The moment is the initial moment, used to calculate displacement and speed. Therefore, the variance of the crowd movement speed at a total of moments can be calculated.

[0096] Among them, the specific method for collecting the sound feature parameters corresponding to the moments of the public places to be monitored is as follows: In the public places to be monitored, the sound signals are collected in real time through the configured microphones. According to the sound frequency range to be captured, a suitable sampling frequency (usually 44.1 kHz or higher) is set to ensure that the high-frequency components can be effectively captured to obtain the original sound signals. Next, feature extraction is required. It can be directly measured by a sound level meter, or during the audio signal processing, the root mean square (RMS) value of the signal is calculated to estimate the sound pressure. By performing a fast Fourier transform (FFT) on the audio signal, the amplitude spectrum of each frequency component in the signal can be obtained. This process converts the signal in the time domain into a frequency domain representation, and can clearly identify the main frequency components and their intensities. The spectral entropy is a measure of the complexity of the signal frequency distribution, and can be obtained by first calculating the spectrum of the signal and then calculating the entropy value using the probability distribution.

[0097] The abnormal situation analysis module is used to calculate and generate a behavior evaluation index based on the predicted value of the number of human targets obtained and the variance of the movement speed, calculate and generate a sound abnormality evaluation index based on the sound feature parameters corresponding to the moments of the public places to be monitored, and comprehensively generate an abnormal alarm index based on the sound abnormality evaluation index and the behavior evaluation index.

[0098] The behavior evaluation index is calculated based on the predicted value of the number of human targets and the variance of the movement rate. The specific formula for calculating the behavior evaluation index is:

[0099]

[0100] In the formula, is the behavioral assessment index, is the predicted number of human targets in the public places to be monitored, is the human target reference value for the public places to be monitored, where ;

[0101] It should be noted that the Behavioral Assessment Index It is used to characterize the common behaviors of people when abnormal situations occur, among which the behavior assessment index The larger the value, the more similar the crowd's behavior is to the abnormal situation, and the greater the probability of an abnormal situation occurring.

[0102] Among them, when an abnormal situation occurs, people usually gather together, so the predicted value of the number of human targets in the public places to be monitored is Behavioral Assessment Index Proportional, It reflects the difference between the predicted value of the number of human targets currently monitored and the set reference value. Significantly higher than When the flow of people exceeds the normal level, it may indicate a potential safety risk. The square root form is used to prevent The exponential growth caused by excessive values can also be relatively smooth, so that when the fluctuation is small, the exponential change is not particularly drastic. This means that when the deviation of the flow of people increases, the behavior assessment index will show a rapid growth. This design emphasizes the potential risk of exceeding the normal flow of people and is suitable for rapid response to abnormal situations. The human target reference value set in the public place to be monitored It is set based on the average flow of people in the target public area and expert experience.

[0103] Sudden events usually trigger people's fear and anxiety. Under the influence of such emotions, people may be at a loss and feel overwhelmed, which may cause them to slow down their movement speed. Being overwhelmed may cause people to choose to stop or move slowly when facing unknown situations, or speed up their movement speed when encountering danger. Therefore, the movement speed variance is used to characterize the changes in movement speed fluctuations. Therefore, the movement speed variance of the crowd is related to the behavior evaluation index. is proportional to the value and is in the form of the natural logarithm function This form emphasizes the impact of crowd mobility. As the speed of crowd movement fluctuates rapidly, the behavior assessment index will increase, but the increase is affected by the logarithmic function and is relatively stable, avoiding the index being overly sensitive to small changes in movement speed.

[0104] The sound anomaly assessment index is calculated based on the sound characteristic parameters collected at the corresponding time in the public place to be monitored. The specific formula for calculating the sound anomaly assessment index is as follows:

[0105]

[0106] In the formula, is the sound anomaly assessment index, is the spectral entropy of the sound signal, is the frequency of the sound signal, is the sound pressure level of the sound signal. The sound pressure level of the sound signal is calculated based on the sound pressure of the sound signal. The specific formula it is based on is:

[0107]

[0108] In the formula, is the sound pressure of the sound signal, is the reference sound pressure.

[0109] It should be noted that the formula for the sound anomaly assessment index constructs a comprehensive assessment model by combining factors such as spectral entropy, frequency, and sound pressure level, effectively reflecting the abnormal characteristics of the sound signal. Among them, the larger the sound anomaly assessment index is, the greater the probability of an abnormal situation.

[0110] Among them, the spectral entropy of the sound signal , the spectral entropy measures the complexity or uncertainty of the speech signal. A higher spectral entropy may indicate that the speaker is emotionally unstable or anxious, while a lower spectral entropy may indicate a stable and calm mood. The higher the value, the more complex the spectrum of the sound signal and the greater the amount of information. If the sound signal is very simple (such as a single-frequency noise), the spectral entropy will be low, which may indicate a lower possibility of sound anomaly; while complex sounds (such as the noise of a crowd or a sudden noise) may indicate a higher risk of anomaly. Therefore, the spectral entropy of the sound signal is proportional to the sound anomaly assessment index and emphasizes the importance of complexity for anomaly assessment in the form of square .

[0111] The higher the frequency, the more complex the characteristics of the sound may be, and it is usually related to the nature of the sound (such as sharpness). Quarreling generally generates higher frequency components, especially human shouts and screams, usually between several hundred hertz and several thousand hertz. Therefore, the frequency of the sound signal is proportional to the sound anomaly assessment index . By performing a logarithmic transformation on the sound frequency , it reflects the influence of frequency on sound anomaly assessment, helps to smooth the influence of frequency, and avoids the disproportionate influence of high-frequency sounds on the assessment.

[0112] The sound pressure level of the sound signal represents the intensity of the sound, which directly reflects the intensity and energy of the sound wave. The higher the sound pressure, the greater the energy of the sound and the greater the probability of an abnormal situation. Therefore, the sound pressure level of the sound signal is proportional to the sound anomaly assessment index . Through the exponential function it shows the significant influence of the sound pressure level of the sound signal on the judgment of abnormal situations.

[0113] Among them, the reference sound pressure is generally 20 µPa.

[0114] Based on the sound anomaly assessment index and the behavior assessment index, an abnormal alarm index is comprehensively generated. The formula for calculating the abnormal alarm index is as follows:

[0115]

[0116] In the formula, is the abnormal alarm index, and are the weight coefficients of the behavior assessment index and the sound anomaly assessment index respectively, where , and and are both greater than 0.

[0117] It should be noted that the larger the value of the abnormal alarm index , the greater the probability of an abnormal situation. Since the proportional relationship between the behavior assessment index and the sound anomaly assessment index and the abnormal alarm index has been described above, it will not be elaborated here.

[0118] The importance of the behavior assessment index for abnormal judgment is represented in the form of a square , and the logarithmic function indicates that as the sound anomaly assessment index increases, the influence on abnormal judgment gradually decreases.

[0119] Since the behavior assessment index can usually more directly reflect population dynamics or individual responses, giving clearer alarm signals. In contrast, changes in sound signals may require more analysis time and processing. Therefore, giving a relatively lower value in terms of weight can ensure that the alarm system is more sensitive to behavior changes. At the same time, sound signals are vulnerable to interference, so set , and and are all greater than 0.

[0120] The public security management decision-making module is used to compare the abnormal alarm index obtained with the set abnormal judgment threshold, and according to different comparison results, issue corresponding abnormal judgment results, and take corresponding measures according to the abnormal judgment results to complete the intelligent public security management in public places.

[0121] Compare the abnormal alarm index with the set abnormal judgment threshold, and according to different comparison results, issue corresponding abnormal judgment results. The specific judgment logic is as follows:

[0122] When , it is judged that an abnormal situation has occurred in the public place to be monitored, and measures such as alarm and notification of relevant departments should be taken;

[0123] When , it is judged that there is a risk of an abnormal situation occurring in the public place to be monitored, and the area should be strengthened for surveillance and early warning;

[0124] When , it is judged that there is no abnormal situation in the public place to be monitored, and no relevant management is required;

[0125] Among them is the abnormal judgment threshold, which is dynamically corrected through environmental parameters. The environmental parameters include the average environmental wind speed and relative humidity. Among them, the abnormal judgment threshold The formula for calculation is:

[0126]

[0127] In the formula, is the initial value of the abnormal judgment threshold, is the average environmental wind speed, is the environmental relative humidity, is the reference humidity, is the reference wind speed.

[0128] Among them, the greater the environmental relative humidity, it may indicate weather conditions such as rain or snow, the probability of crowd gathering may increase, and the probability of abnormal situations occurring will increase. Therefore, the environmental relative humidity is inversely proportional to the abnormal judgment threshold. Therefore, the abnormal judgment threshold should be reduced to prevent the system from missing judgments.

[0129] Average environmental wind speed The greater the average environmental wind speed, the greater the noise will be. At the same time, the greater the wind speed, the more severe the environment is, and the probability of abnormal situations will increase. Therefore, the average environmental wind speed is inversely proportional to the abnormal judgment threshold. Therefore, the abnormal judgment threshold should be reduced to prevent the system from missing judgments.

[0130] Among them, the initial value of the abnormal judgment threshold , reference humidity and reference wind speed can be set according to the actual environmental conditions of the public area to be monitored and combined with expert experience.

[0131] The present invention also provides a smart public security management device for public places. The smart public security management device for public places includes one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned smart public security management system for public places.

[0132] Please refer to Figure 2 , the present invention also provides a smart public security management method for public places. The smart public security management method for public places is used to control the above-mentioned smart public security management system for public places. The specific steps include:

[0133] Step 1: Obtain thermal infrared images of several different public places, preprocess the collected thermal infrared images, and mark all human targets in the preprocessed thermal infrared images by manual marking. The marked thermal infrared images are recorded as training sample images, and the number of human targets corresponding to the training sample images is recorded. The preprocessing includes thermal infrared image denoising and enhancement processing.

[0134] Step 2: Map the training sample images and the number of human targets corresponding to the images one by one to generate a training sample data set. Based on the data in the training sample data set, a convolutional neural network model is established. The training sample images in the training sample data set are used as the input of the model, and the neural network model is trained with the human target annotations in the training sample images and the corresponding number of human targets as labels to obtain a pedestrian flow prediction model.

[0135] Step 3: Continuously collect thermal infrared images at different times within the public places to be monitored. After preprocessing the collected thermal infrared images of the public places to be monitored, input them into the trained crowd flow prediction model. The model marks the human targets in the images and outputs the predicted value of the corresponding number of human targets. Based on the marked images at different times, obtain the variance of the movement speed of the crowd. At the same time, collect the sound feature parameters corresponding to the public places to be monitored at that moment. The sound feature parameters include the sound pressure, frequency, and spectral entropy of the sound signal.

[0136] Step 4: Calculate and generate a behavior evaluation index according to the obtained predicted value of the number of human targets combined with the variance of the movement speed. Calculate and generate a sound anomaly evaluation index according to the collected sound feature parameters corresponding to the public places to be monitored at that moment. Based on the sound anomaly evaluation index and the behavior evaluation index, comprehensively generate an anomaly alarm index.

[0137] Step 5: According to the obtained anomaly alarm index, compare the anomaly alarm index with the set anomaly judgment threshold. According to different comparison results, issue the corresponding anomaly judgment result, and take corresponding measures according to the anomaly judgment result to complete the intelligent public security management of public places.

[0138] All the above formulas are dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the real situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0139] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0140] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units. They can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered within the protection scope of this application.

Claims

1. A smart public security management system for public places, characterized in that, Specifically include: A training sample acquisition module, which is used to obtain thermal infrared images of several different public places, preprocess the collected thermal infrared images, and label all human targets in the preprocessed thermal infrared images by manual marking. The labeled thermal infrared images are recorded as training sample images, and the number of human targets corresponding to the training sample images is recorded. The preprocessing includes thermal infrared image denoising and enhancement processing; A prediction model training module, which is used to map the training sample images and the number of human targets corresponding to the images one by one to generate a training sample data set. Based on the data in the training sample data set, a convolutional neural network model is established. The training sample images in the training sample data set are used as the input of the model, and the neural network model is trained with the human target labels in the training sample images and the corresponding number of human targets as labels to obtain a pedestrian flow prediction model; A feature parameter acquisition module, which is used to continuously collect thermal infrared images at different times in the current time period in the public place to be monitored. After preprocessing the collected thermal infrared images in the public place to be monitored, they are input into the trained pedestrian flow prediction model. The model marks the human targets in the images and outputs the predicted value of the corresponding number of human targets. The variance of the movement speed of the crowd is obtained based on the marked images at different times. At the same time, the sound feature parameters corresponding to the public place to be monitored at the corresponding time are collected. The sound feature parameters include the sound pressure, frequency, and spectral entropy of the sound signal; An abnormal situation analysis module, which is used to calculate and generate a behavior evaluation index according to the predicted value of the number of human targets and the variance of the movement speed, calculate and generate a sound abnormality evaluation index according to the sound feature parameters corresponding to the public place to be monitored at the corresponding time, and comprehensively generate an abnormal alarm index based on the sound abnormality evaluation index and the behavior evaluation index; A public security management decision-making module, which is used to compare the obtained abnormal alarm index with the set abnormal judgment threshold, issue corresponding abnormal judgment results according to different comparison results, and take corresponding measures according to the abnormal judgment results to complete the intelligent public security management of public places.

2. The intelligent public security management system for public places according to claim 1, characterized in that: Preprocess each collected thermal infrared image of a public place. The preprocessing includes: image enhancement and denoising preprocessing. Among them, the wavelet transform denoising method is used to denoise each thermal infrared image of a public place, and bilateral filtering is used to preprocess the image enhancement of each thermal infrared image of a public place; Among them, the generation method of the training sample data set is: map the training sample images and the corresponding number of human targets one by one to form a corresponding grid, and record the formed grid as the training sample data set.

3. The intelligent public security management system for public places according to claim 2, characterized in that: Based on a convolutional neural network, a pedestrian flow prediction model is established. The convolutional neural network consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The activation function in the convolutional layer is function, The specific expression of the function is: Among them, represents the th corresponding convolutional layer, while represents the th eigenvalue of the th training sample image in the th convolutional layer, where is the index of the convolutional layer, is the index of the training sample image, and is the index of the eigenvalue in the training sample image; For the fully connected layer, set the number of neurons in the fully connected layer to 32, set the initial neural network learning rate to 0.001, and the number of training rounds to 300; The input of the trained pedestrian flow prediction model is the thermal infrared image of a public place, and the output is the human target label in the image and the corresponding number of human targets.

4. A smart public security management system for public places according to claim 1, wherein: Obtain the variance of the movement speed of the crowd based on the marked images at different times. The method for obtaining the variance of the movement speed of the crowd is as follows: Taking the center of the image captured by the thermal infrared camera as the origin, the due north direction as the positive x-axis direction, and the due west direction perpendicular to the x-axis as the y-axis, establish a plane coordinate system for the public place to be monitored. Randomly select a number of human targets, obtain the displacements of the selected human targets within the acquisition time interval, and calculate the variance of the movement speed based on the displacements. The formula for calculating the variance of the movement speed of the crowd is as follows: In the formula, The variance of the population movement speed in the current time period, where j is the index of different moments in the current time period, is the total number of moments in the current time period, where , is the average movement speed of the population in the current time period, is the average movement speed of the population at the j-th moment in the current time period; where and The calculation is based on the formula: In the formula, is the total number of selected human targets, is the movement speed of the i-th selected human target at the j-th moment, where i is the index of the selected human target, , where the movement speed of the i-th selected human target is calculated from displacement data, and the specific calculation formula is as follows: In the formula, is the displacement of the i-th selected human target at the j-th moment from the previous acquisition moment, is the time interval between the acquisition moment and the previous acquisition moment, where the displacement of the i-th selected human target at the j-th moment from the previous acquisition moment is calculated according to the formula: Wherein, and are respectively the abscissa and ordinate of the location of the i-th selected human target at the j-th moment, and are respectively the abscissa and ordinate of the location of the i-th selected human target at the (j - 1)-th moment.

5. The intelligent public security management system for public places according to claim 4, wherein: Based on the obtained predicted value of the number of human targets and the variance of the movement speed, calculate and generate a behavior evaluation index. The specific formula for calculating the behavior evaluation index is as follows: In the formula, is the behavior evaluation index, is the predicted value of the number of human targets in the public place to be monitored, is the reference value of human targets in the public place to be monitored set, where ; Calculate and generate a sound anomaly evaluation index based on the sound characteristic parameters collected at the corresponding time of the public place to be monitored. The specific formula for calculating the sound anomaly evaluation index is as follows: In the formula, is the abnormal sound evaluation index, is the spectral entropy of the sound signal, is the frequency of the sound signal, is the sound pressure level of the sound signal, where the sound pressure level of the sound signal is calculated according to the sound pressure of the sound signal, and the specific formula is: Wherein, is the sound pressure of the sound signal, is the reference sound pressure.

6. The intelligent public security management system for public places according to claim 5, wherein: Based on the sound anomaly evaluation index and the behavior evaluation index, comprehensively generate an anomaly alarm index. The formula for calculating the anomaly alarm index is as follows: In the formula, is the abnormal alarm index, and are the weight coefficients of the behavior evaluation index and the sound abnormality evaluation index respectively, where , and and are both greater than 0.

7. The intelligent public security management system for public places according to claim 6, characterized in that: Compare the anomaly alarm index with the set anomaly judgment threshold, and issue corresponding anomaly judgment results according to different comparison results. The specific judgment logic is as follows: When it is determined that an abnormal situation has occurred in the public place to be monitored, measures such as alarm and notification to relevant departments should be taken; When it is determined that there is a risk of abnormal conditions occurring in the public place to be monitored, strengthen the surveillance of this area and give an early warning; When it is determined that there is no abnormal situation in the public place to be monitored, and no relevant management is required; Among them is the abnormal judgment threshold, which is dynamically corrected through environmental parameters. The environmental parameters include the average environmental wind speed and relative humidity. Among them, the abnormal judgment threshold The formula based on the calculation is as follows: Wherein, is the initial value of the abnormality judgment threshold, is the average environmental wind speed, is the environmental relative humidity, is the reference humidity, is the reference wind speed.

8. An intelligent public security management device for public places, characterized in that: The described intelligent public security management device for a public place includes one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent public security management system for a public place as described in any one of claims 1-7.

9. A smart public security management method for public places, characterized in that: The described intelligent public security management method for a public place is used to control the intelligent public security management system for a public place as described in any one of claims 1-7. The specific steps include: Obtain thermal infrared images of several different public places, preprocess the collected thermal infrared images, and through manual marking, label all human targets in the preprocessed thermal infrared images. Denote the marked thermal infrared images as training sample images, and record the number of human targets corresponding to the training sample images. The preprocessing includes thermal infrared image denoising and enhancement processing; Map the training sample images and the corresponding number of human targets in the images one by one to generate a training sample data set. Based on the data in the training sample data set, establish a convolutional neural network model. Use the training sample images in the training sample data set as the input of the model, and use the human target annotations and the corresponding number of human targets in the training sample images as labels to train the neural network model to obtain a human flow prediction model; In the public place to be monitored, continuously collect thermal infrared images at different times within the current time period. After preprocessing the collected thermal infrared images of the public place to be monitored, input them into the trained human flow prediction model. The model marks the human targets in the images and outputs the corresponding predicted value of the number of human targets. Obtain the variance of the movement speed of the crowd based on the marked images at different times, and at the same time collect the sound characteristic parameters at the corresponding time of the public place to be monitored. The sound characteristic parameters include the sound pressure, frequency, and spectral entropy of the sound signal; Based on the predicted value of the number of human targets obtained and combined with the variance of the movement speed, calculate and generate a behavior evaluation index. Calculate and generate a sound anomaly evaluation index according to the sound feature parameters collected at the corresponding moment in the public place to be monitored. Based on the sound anomaly evaluation index and the behavior evaluation index, comprehensively generate an anomaly alarm index; According to the obtained anomaly alarm index, compare the anomaly alarm index with the set anomaly judgment threshold. According to different comparison results, issue corresponding anomaly judgment results, and take corresponding measures according to the anomaly judgment results to complete the intelligent public security management in public places.

Images