Campus safety protection system based on image and sound recognition

Through image and sound recognition technology, combined with convolutional neural network and YOLO model, high accuracy recognition and automatic prevention of campus hazard behaviors are achieved, and the problems of low recognition rate and slow response in the existing technology are solved, and campus safety is improved Efficiency and coverage of prevention.

CN119068871APending Publication Date: 2024-12-03成都元智盈创科技有限公司

Patent Information

Application Number
CN202411092917.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing campus security measures rely on manual patrol and surveillance cameras, with low recognition rate and slow response, making it difficult to cover all areas, and it is impossible to effectively prevent and promptly stop campus hazardous behaviors.

Method used

Adopt a campus security prevention system based on image and sound recognition, including video acquisition, sound acquisition, image recognition, sound recognition, data processing and prevention measures execution modules, and process video and sound data through convolutional neural network and YOLO model and extract it. Features and feature fusion, automatically judge and trigger prevention measures.

Benefits of technology

It improves the accuracy of identifying hazardous behaviors on campus, monitors public and privacy areas in real time, automatically triggers preventive measures, quickly stops hazardous behaviors, reduces victim injuries, reduces manual inspection and monitoring costs, and improves security work efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068871A_ABST
    Figure CN119068871A_ABST
Patent Text Reader

Abstract

The invention discloses a campus safety protection system based on image and sound recognition. The campus safety protection system comprises the following modules: a video acquisition module, a sound acquisition module, an image recognition module, a sound recognition module, an image processing module, an image processing module, an image processing module and a sound processing module, wherein the image recognition module is used for performing image recognition on video data of the video acquisition module; the sound recognition module is used for recognizing sound data acquired by the sound acquisition module; the data processing module is used for processing results of the image recognition module and the voice recognition module; and the precautionary measure execution module is used for executing corresponding precautionary measures according to the judgment result of the data processing unit. According to the invention, the image and sound recognition technology is adopted, the recognition accuracy of the campus harmful behaviors is improved, the public area and the privacy area of the campus are monitored in real time, and the campus harmful behaviors are comprehensively prevented. Through image and sound identification, precautionary measures are automatically triggered, campus harmful behaviors are rapidly stopped, harm to victims is reduced, meanwhile, manual patrol and monitoring cost can be reduced, and campus security work efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things, and in particular relates to a campus safety prevention system based on image and sound recognition. Background Art

[0002] With the development of society, campus incidents are common, causing physical and mental harm to the victims, and also bringing negative impacts to schools and society. Existing campus security measures are mainly based on manual inspections and surveillance cameras, but this method has problems such as low recognition rate and slow response, making it difficult to effectively prevent and promptly stop campus harmful behaviors.

[0003] In the existing technology, campus security measures rely on manual patrols and surveillance cameras, but this method is limited by the sensitivity and accuracy of human vision and hearing, and it is difficult to detect and accurately judge campus harmful behaviors in a timely manner. Since human intervention is required, once a campus harmful incident occurs, the response speed of relevant departments is often not fast enough, and it is difficult to stop the incident in time; manual patrols have blind spots, and some private or imperceptible areas may become high-incidence areas of campus harm. Summary of the invention

[0004] The purpose of the present invention is to provide a campus security system based on image and sound recognition to solve the problems of low recognition rate, slow response and difficulty in covering all areas in the prior art proposed in the background technology for patrolling campus hazards through manual patrols and surveillance cameras.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: The campus security system based on image and sound recognition includes the following modules: Video acquisition module: used for collecting video information in public areas; Sound collection module: used for collecting sound information in private areas; Image recognition module: used to perform image recognition on the video data of the video acquisition module and identify harmful behaviors on campus; Voice recognition module: used to recognize the voice data collected by the voice collection module, including recognition of calls for help, screams and special words for help; Data processing module: used to process the results of the image recognition module and the sound recognition module to determine whether campus hazards have occurred; Preventive measures execution module: executes corresponding preventive measures according to the judgment results of the data processing unit.

[0006] According to the above technical solution, the image recognition module processes the collected video data and sound data; The video data processing is as follows: Step S1, first, decomposing the collected video data into separate image frames; Step S2, preprocessing the decomposed image frame; the preprocessing specifically includes: Step S201, the camera captures the video stream; first, the camera is used to capture the video stream; Step S202, frame extraction, extracting a single frame from the video stream as an image for processing; Step S203: image preprocessing to improve the effect and efficiency of image recognition; Step S3, extracting features from the processed image frame through a convolutional neural network is specifically as follows: Features include color, shape, texture, and more complex feature maps; the extracted features are used for image frame classification, object detection, object recognition, and object tracking in videos; Step S4, using the YOLO recognition model to recognize the action in the image frame. YOLO achieves target detection by dividing the entire image into grids and predicting bounding boxes and categories in each grid. When using YOLO for target detection, it is applied to each frame in the video data and tracked between consecutive frames to achieve continuous tracking of the target. The sound data is processed specifically as follows: filtering, denoising and feature extraction of the sound data to reduce noise; extracting sound features from the sound data; including sound frequency, sound Fourier transform, Mel spectrum features and sound intensity.

[0007] According to the above technical solution, in step S203, the image preprocessing is specifically as follows: Step S2031, scaling the image target size: using the YOLOv5 model, setting a target size; using bilinear interpolation or other interpolation methods to scale the image to maintain image clarity; Step S2032, padding the image and calculating the padding amount; in order to make the scaled image conform to the model input size, padding is usually performed on the short side of the image; the padding amount depends on the difference between the scaled image size and the model input size; Step S2033, color space conversion from BGR to RGB; Step S2034, normalize the numerical range: convert the pixel value of the image from an integer range of 0-255 to a floating point range of 0-1; normalization formula: img_normalized=img / 255.0.

[0008] According to the above technical solution, in step S3, the feature extraction by convolutional neural network is specifically as follows: Step S301, the input layer takes the original image data as input; these image data are usually normalized to the range of [0, 1] or [-1, 1]; Step S302, convolution layer convolution operation: use multiple learnable convolution kernels (slide on the input image and perform dot product operation; perform nonlinear transformation on the result of the convolution operation through a nonlinear activation function; Step S303, after the convolutional layer, a pooling layer is added to reduce the spatial size of the data, thereby reducing the amount of calculation and the number of parameters while retaining important features; Step S304, stacking multiple convolutional layers and pooling layers Multiple convolutional layers and pooling layers are stacked to extract higher-level features. As the number of layers increases, the network can capture more complex patterns and structures; Step S305, adding one or more fully connected layers after the convolution and pooling layers to further process the features; Step S306, output layer, the output layer is used to generate the final prediction result; Step S307, training process; during the training process, the network will update weights and bias items through backpropagation algorithm and gradient descent optimization algorithm to minimize the loss function; Step S308: After the training is completed, feature extraction uses the trained CNN model to extract features.

[0009] According to the above technical solution, in step S4, the YOLO recognition model is used to recognize the action in the image frame as follows: Step S401, dividing the input image into S×S grids; predicting B bounding boxes for each grid and the confidence and category probability of each bounding box; removing overlapping bounding boxes by non-maximum suppression (NMS) algorithm to obtain the final detection result; Step S402, target detection: first, use the YOLO model to perform target detection on the input image to obtain the location and category of the target; Feature extraction: extracting meaningful features from the detected target, including the target’s shape, size, position, motion trajectory, etc. Action classification, the extracted features are input into an action classifier for classification training or prediction.

[0010] Timing analysis,For action recognition in videos, it is also necessary to consider the changes of the target in timing, and identify the action by analyzing the position and feature changes of the target in consecutive frames.

[0011] According to the above technical solution, after the processing of the video data and the sound data is completed, feature fusion is performed; the feature fusion is specifically: the features extracted from the image and sound data are fused to form a comprehensive feature vector or representation for capturing the correlation and complementarity between the image and the sound.

[0012] According to the above technical solution, the fused feature vector is used to train the machine learning or deep learning model, specifically: First, features need to be extracted from the image and sound data separately and fused into a comprehensive feature vector; this process is achieved through concatenation, weighted averaging, or more complex feature fusion.

[0013] Data preparation labeling: Make sure that the dataset has been correctly labeled, that is, each sample has a corresponding category label or continuous value; Data preprocessing: Perform necessary preprocessing on feature vectors, such as normalization, standardization, or feature scaling, to ensure that different features are comparable in value; Divide the dataset: Divide the dataset into training set, validation set and test set; the training set is used to train the model, the validation set is used to adjust hyperparameters and perform early stopping, and the test set is used to evaluate the performance of the model; Training model setting loss function: select loss function; Optimization algorithm: Use stochastic gradient descent to update the parameters of the model; Training process: Use training data to iteratively update the model's parameters until the model performs best on the validation set or meets other stopping conditions.

[0014] Evaluate the model using the test set to evaluate the performance of the model; deploy the model.

[0015] According to the above technical solution, the dangerous behavior is judged as follows: when the model recognizes that an event occurs in both the image and sound data, the data processing unit determines that the dangerous behavior exists and needs to be stopped.

[0016] Compared with the prior art, the present invention has the following beneficial effects: In the present invention, image and sound recognition technology is used to improve the accuracy of identifying harmful behaviors on campus, monitor the public and private areas of the campus in real time, and comprehensively prevent harmful behaviors on campus. Through image and sound recognition, preventive measures are automatically triggered to quickly stop harmful behaviors on campus and reduce the harm to victims. At the same time, it can also reduce the cost of manual inspections and monitoring, and improve the efficiency of campus security work. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a system block diagram of the present invention; Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1 like Figure 1 As shown in the figure, the campus security system based on image and sound recognition includes the following modules: Video acquisition module: used for collecting video information in public areas; Sound collection module: used for collecting sound information in private areas; Image recognition module: used to perform image recognition on the video data of the video acquisition module and identify harmful behaviors on campus; Voice recognition module: used to recognize the voice data collected by the voice collection module, including recognition of calls for help, screams and special words for help; Data processing module: used to process the results of the image recognition module and the sound recognition module to determine whether campus hazards have occurred; Preventive measures execution module: executes corresponding preventive measures according to the judgment results of the data processing unit.

[0020] In the present invention, image and sound recognition technology is used to improve the accuracy of identifying harmful behaviors on campus, monitor the public and private areas of the campus in real time, and comprehensively prevent harmful behaviors on campus. Through image and sound recognition, preventive measures are automatically triggered to quickly stop harmful behaviors on campus and reduce the harm to victims. At the same time, it can also reduce the cost of manual inspections and monitoring, and improve the efficiency of campus security work.

[0021] Embodiment 2 This embodiment is a further refinement of the first embodiment.

[0022] The image recognition module processes the collected video data and sound data; like Figure 2 As shown, the video data processing is specifically as follows: Step S1, first, decomposing the collected video data into separate image frames; Step S2, preprocessing the decomposed image frame; the preprocessing specifically includes: Step S201, the camera captures the video stream; first, the camera is used to capture the video stream; this can be implemented through various programming languages ​​and libraries, such as OpenCV, DirectShow, etc.

[0023] Step S202, frame extraction, extracts a single frame from the video stream as an image for processing; this is usually completed in a loop, with one frame processed each time.

[0024] Step S203: image preprocessing to improve the effect and efficiency of image recognition; Step S3, extracting features from the processed image frame through a convolutional neural network is specifically as follows: Features include color, shape, and texture; the extracted features are used to classify picture frames, detect or track objects in videos; Step S4, using the YOLO recognition model to recognize the action in the image frame. YOLO achieves target detection by dividing the entire image into grids and predicting bounding boxes and categories in each grid. When using YOLO for target detection, it is applied to each frame in the video data and tracked between consecutive frames to achieve continuous tracking of the target. The sound data is processed specifically as follows: filtering, denoising and feature extraction of the sound data to reduce noise; extracting sound features from the sound data; including sound frequency domain features, sound time domain features and sound intensity.

[0025] In step S203, the image preprocessing is specifically as follows: Step S2031, scaling the image target size: according to the requirements of YOLOv5 or other target detection models, set a target size (for example, 640x640 pixels). Scaling method: use bilinear interpolation or other interpolation methods to scale the image to maintain the clarity of the image.

[0026] Step S2032, padding the image and calculating the padding amount: In order to make the scaled image conform to the model input size, padding is usually performed on the short side of the image. The padding amount depends on the difference between the scaled image size and the model input size. Filling color: Usually a specific color (such as black, gray, or the background color used during model training) is used for filling.

[0027] Step S2033, color space conversion from BGR to RGB: If the image captured by the camera is in BGR format (OpenCV default format), it needs to be converted to RGB format, because many deep learning libraries (such as PyTorch, TensorFlow) use RGB format by default.

[0028] Step S2034, normalize the numerical range: convert the pixel value of the image from the integer range of 0-255 to the floating point range of 0-1. Normalization formula: img_normalized = img / 255.0; Step S2035, data enhancement: According to the needs, data enhancement steps can be added in the preprocessing stage, such as random flipping, cropping, rotation, etc., to increase the robustness of the model.

[0029] In step S3, the features extracted by the convolutional neural network are specifically: In step S301, the input layer takes the original image data as input; these image data are usually normalized to the range of [0, 1] or [-1, 1].

[0030] Step S302, convolution layer Convolution operation: Use multiple learnable convolution kernels (also called filters or feature detectors) to slide on the input image and perform dot product operations. Activation function: The result of the convolution operation is nonlinearly transformed through a nonlinear activation function (such as ReLU, Sigmoid, Tanh, etc.). Parameters: Each convolution kernel has its weight and bias terms, which will be updated during the training process.

[0031] Step S303, pooling layer After the convolution layer, a pooling layer is usually added to reduce the spatial size of the data (dimensionality reduction), thereby reducing the amount of calculation and the number of parameters while retaining important features. Common pooling operations include MaxPooling and AveragePooling.

[0032] Step S304, stacking of multiple convolutional layers and pooling layers Multiple convolutional layers and pooling layers are usually stacked to extract higher-level features. As the number of layers increases, the network is able to capture more complex patterns and structures.

[0033] Step S305, fully connected layer After the convolution and pooling layers, one or more fully connected layers (also called dense layers or fully connected layers) are usually added to further process the features. In a fully connected layer, each neuron is connected to all neurons in the previous layer.

[0034] Step S306, output layer The output layer is used to generate the final prediction result. For classification tasks, the output layer usually uses a softmax function to generate probability distribution; for regression tasks, the output layer may have only one neuron, which directly outputs the prediction value.

[0035] Step S307, training process; during the training process, the network will update weights and bias items through optimization algorithms such as backpropagation and gradient descent to minimize the loss function.

[0036] The loss function measures the gap between the network's prediction results and the actual results. Commonly used loss functions include Cross-Entropy Loss, Mean Squared Error, etc.

[0037] Step S308, feature extraction After training is completed, the trained CNN model can be used to extract features. This is usually done by passing the input image through the network's forward propagation. Features are usually extracted from a convolutional layer or pooling layer before the fully connected layer. The activation values ​​in these layers (i.e., the output of the neurons) can be regarded as feature representations of the input image. These features can be used for various tasks such as classification, regression, clustering, etc.

[0038] In step S4, the YOLO recognition model is used to recognize the action in the image frame. Specifically, YOLO is a real-time target detection algorithm that converts the target detection task into a single regression problem. Through a single forward propagation, YOLO can predict the location and category of multiple targets in the image.

[0039] Step S401, divide the input image into S×S grids (such as in YOLOv3, S is usually 13 or 19). Each grid predicts B bounding boxes and the confidence and category probability of each bounding box. The non-maximum suppression (NMS) algorithm is used to remove overlapping bounding boxes to obtain the final detection result.

[0040] Step S402, YOLO combined with action recognition principle: Although YOLO is mainly used for target detection, by identifying the targets in the image, the spatial relationship and temporal relationship between these targets can be further analyzed to perform action recognition. Method: Target detection: First, use the YOLO model to detect the target of the input image to obtain the position and category of the target. Feature extraction: Extract meaningful features from the detected target, which may include the shape, size, position, motion trajectory, etc. of the target. Action classification: Input the extracted features into an action classifier (such as support vector machine, random forest, neural network, etc.) for classification training or prediction. The action classifier can judge the ongoing action based on the features of the target and the spatial relationship between them. Timing analysis: For action recognition in the video, the changes of the target in time sequence also need to be considered. Actions can be identified by analyzing the position and feature changes of the target in consecutive frames.

[0041] After the video data and sound data are processed, feature fusion is performed; feature fusion specifically includes: fusing the features extracted from the image and sound data to form a comprehensive feature vector or representation; The goal of feature fusion is to effectively combine the features extracted from image and sound data to form a comprehensive feature vector or representation. There are two main fusion methods: Definition of early fusion: Fusion is performed at the feature level, that is, after the image and sound data are extracted separately, these features are directly merged.

[0042] Method: Concatenation, which simply concatenates the feature vectors of the image and sound to form a longer feature vector. For example, if the length of the image feature vector is N and the length of the sound feature vector is M, the length of the concatenated feature vector is N+M. Weighted Average: Assign different weights to the feature vectors of the image and sound, and then calculate their weighted average. This method allows the model to take into account the importance of different modal features during the fusion process.

[0043] Definition of late fusion: Fusion is performed at the decision level, that is, after feature extraction and classification of image and sound data respectively, their classification results are merged.

[0044] Method: Voting, voting based on the output results of the image and sound classifiers to determine the final classification result. Weighted average assigns different weights to the output results of the image and sound classifiers, and then calculates their weighted average to determine the final classification result; used to capture the correlation and complementarity between images and sounds.

[0045] Using the fused feature vector to train a machine learning or deep learning model is an important step that allows the model to learn the associations and patterns between features extracted from multiple data modalities, thereby more accurately predicting or classifying the target. The following are the general steps of this process: 1. Feature fusion: First, you need to extract features from the image and sound data separately, and then fuse these features into a comprehensive feature vector. This process can be achieved through concatenation, weighted averaging, or more complex feature fusion techniques (such as attention-based fusion methods).

[0046] 2. Data preparation labeling: Make sure that the dataset has been correctly labeled, that is, each sample has a corresponding category label (for classification tasks) or continuous value (for regression tasks). Data preprocessing: Perform necessary preprocessing on the feature vector, such as normalization, standardization, or feature scaling, to ensure that different features are numerically comparable. Divide the dataset: Divide the dataset into training set, validation set, and test set. Typically, the training set is used to train the model, the validation set is used to adjust hyperparameters and perform early stopping, and the test set is used to evaluate the performance of the model.

[0047] 3. Choose a model, and choose a suitable machine learning or deep learning model according to the specific task and data characteristics. For example, for classification tasks, you can choose traditional machine learning algorithms such as logistic regression, support vector machine (SVM), random forest, or deep learning models such as convolutional neural network (CNN), recurrent neural network (RNN) or Transformer.

[0048] 4. Training model Setting loss function: Select an appropriate loss function according to the task, such as cross entropy loss (for classification tasks) or mean square error (for regression tasks). Optimization algorithm: Select an optimization algorithm to update the parameters of the model, such as stochastic gradient descent (SGD), Adam, RMSprop, etc. Training process: Use training data to iteratively update the parameters of the model until the model's performance on the validation set reaches the best or other stopping conditions are met (such as reaching the maximum number of iterations).

[0049] 5. Evaluate the model Use the test set to evaluate the performance of the model. Common evaluation indicators include accuracy, recall, F1 score (for classification tasks) or mean square error, R square value (for regression tasks), etc.

[0050] 6. Model tuning and hyperparameter adjustment: Adjust the model's hyperparameters, such as learning rate, batch size, regularization strength, etc., based on the performance of the validation set. Use ensemble methods: Try to use ensemble learning techniques (such as bagging, boosting) to further improve the performance of the model. Feature selection: Use feature selection techniques to reduce the number of features, reduce the complexity of the model, and possibly improve the performance of the model.

[0051] 7. Deploy the model Once you are satisfied with the performance of the model, you can deploy it to a real-world application. This usually involves integrating the model into a larger system to take input from real-time or batch data and generate predictions or classification results. After training is complete, use the test data set to evaluate the performance of the model. Based on the model's prediction results, make decisions or provide corresponding outputs.

[0052] Dangerous behavior judgment: When the model recognizes that an event occurs in both image and sound data, the data processing unit determines that the harmful behavior exists. For example, when bullying occurs, image recognition will generate a special curve A according to the model, and when bullying language appears, such as insulting words, sound recognition will generate a characteristic curve B according to the model; when one or both AB structures appear, the data processing unit determines that a bullying incident will be reported and needs to be stopped.

[0053] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0054] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. Campus security system based on image and sound recognition, characterized by: Includes the following modules: Video acquisition module: used for collecting video information in public areas; Sound collection module: used for collecting sound information in private areas; Image recognition module: used to perform image recognition on the video data of the video acquisition module and identify harmful behaviors on campus; Voice recognition module: used to recognize the voice data collected by the voice collection module, including recognition of calls for help, screams and special words for help; Data processing module: used to process the results of the image recognition module and the sound recognition module to determine whether campus hazards have occurred; Preventive measures execution module: executes corresponding preventive measures according to the judgment results of the data processing unit.

2. The campus security system based on image and sound recognition according to claim 1 is characterized in that: The image recognition module processes the collected video data and sound data; The video data processing is as follows: Step S1, first, decomposing the collected video data into separate image frames; Step S2, preprocessing the decomposed image frame; Preprocessing specifically includes: Step S201, the camera captures the video stream; first, the camera is used to capture the video stream; Step S202, frame extraction, extracting a single frame from the video stream as an image for processing; Step S203: image preprocessing to improve the effect and efficiency of image recognition; Step S3, extracting features from the processed image frame through a convolutional neural network, specifically: Features include color, shape, and texture; the extracted features are used to classify picture frames, detect or track objects in videos; Step S4, using the YOLOv5 recognition model to recognize the action in the image frame, the YOLOv5 model realizes target detection by dividing the entire image into grids and predicting the bounding box and category in each grid; when using the YOLOv5 model for target detection, it is applied to each frame in the video data and tracked between consecutive frames to achieve continuous tracking of the target; The sound data is processed specifically as follows: filtering, denoising and feature extraction of the sound data to reduce noise; extracting sound features from the sound data; including sound frequency, sound time domain features and sound intensity.

3. The campus security system based on image and sound recognition according to claim 2 is characterized in that: In step S203, the image preprocessing is specifically as follows: Step S2031, scaling the image target size: using the YOLOv5 model, setting a target size; using bilinear interpolation or other interpolation methods to scale the image to maintain image clarity; Step S2032, padding the image and calculating the padding amount; in order to make the scaled image conform to the model input size, padding is usually performed on the short side of the image; the padding amount depends on the difference between the scaled image size and the model input size; Step S2033, color space conversion from BGR to RGB; Step S2034, normalizing the numerical range: converting the pixel values ​​of the image from an integer range of 0-255 to a floating point range of 0-1; Normalization formula: img_normalized=img / 255.

0.

4. The campus security system based on image and sound recognition according to claim 2 is characterized in that: In step S3, the features extracted by the convolutional neural network are specifically: Step S301, the input layer takes the original image data as input; these image data are usually normalized to the range of [0, 1] or [-1, 1]; Step S302, convolution layer convolution operation: use multiple learnable convolution kernels (slide on the input image and perform dot product operation; perform nonlinear transformation on the result of the convolution operation through a nonlinear activation function; Step S303, after the convolutional layer, a pooling layer is added to reduce the spatial size of the data, thereby reducing the amount of calculation and the number of parameters while retaining important features; Step S304, stacking multiple convolutional layers and pooling layers Multiple convolutional layers and pooling layers are stacked to extract higher-level features. As the number of layers increases, the network can capture more complex patterns and structures; Step S305, adding one or more fully connected layers after the convolution and pooling layers to further process the features; Step S306, output layer, the output layer is used to generate the final prediction result; Step S307, training process; during the training process, the network will update the weights and bias terms through the backpropagation algorithm and the gradient descent optimization algorithm to minimize the loss function; Step S308: After the training is completed, feature extraction uses the trained CNN model to extract features.

5. The campus security system based on image and sound recognition according to claim 2 is characterized in that: In step S4, the YOLO recognition model is used to recognize the action in the image frame as follows: Step S401, dividing the input image into S×S grids; predicting B bounding boxes for each grid and the confidence and category probability of each bounding box; removing overlapping bounding boxes by non-maximum suppression (NMS) algorithm to obtain the final detection result; Step S402, target detection: first, use the YOLO model to perform target detection on the input image to obtain the location and category of the target; Feature extraction: extracting meaningful features from the detected target, including the target’s shape, size, position, motion trajectory, etc. Action classification: input the extracted features into an action classifier for classification training or prediction; Timing analysis,For action recognition in videos, it is also necessary to consider the changes of the target in timing, and identify the action by analyzing the position and feature changes of the target in consecutive frames.

6. The campus security system based on image and sound recognition according to claim 2 is characterized in that: After the processing of the video data and the sound data is completed, feature fusion is performed; feature fusion is specifically: the features extracted from the image and sound data are fused to form a comprehensive feature vector or representation to capture the correlation and complementarity between the image and the sound.

7. The campus security system based on image and sound recognition according to claim 6 is characterized in that: Use the fused feature vectors to train machine learning or deep learning models, specifically: First, features need to be extracted from the image and sound data separately, and these features need to be fused into a comprehensive feature vector; this process is achieved through concatenation, weighted averaging, or more complex feature fusion; Data preparation labeling: Make sure that the dataset has been correctly labeled, that is, each sample has a corresponding category label or continuous value; Data preprocessing: Perform necessary preprocessing on feature vectors, such as normalization, standardization, or feature scaling, to ensure that different features are comparable in value; Divide the dataset: Divide the dataset into training set, validation set and test set; The training set is used to train the model, the validation set is used to adjust hyperparameters and perform early stopping, and the test set is used to evaluate the performance of the model; Training model setting loss function: select loss function; Optimization algorithm: Use stochastic gradient descent to update the parameters of the model; Training process: Use training data to iteratively update the model's parameters until the model's performance on the validation set reaches the best or other stopping conditions are met; Evaluate the model using the test set to evaluate the performance of the model; deploy the model.

8. The campus security system based on image and sound recognition according to claim 7 is characterized in that: Dangerous behavior judgment: When the model recognizes that an event occurs in both image and sound data, the data processing unit determines that dangerous behavior exists and needs to be stopped.

Citation Information

Patent Citations

  • Campus safety management method, system and device and storage medium

    CN111582042A

  • Anti-spoofing intelligent watch system

    CN117912190A

  • Anti-spoofing system and method in privacy protection area

    CN117975693A

  • Abnormal behavior detection system and method based on heterogeneous data fusion analysis of big data

    CN118378210A

Cited By

  • Risk feature identification analysis method for campus security monitoring system

    CN119721732A

  • A risk feature identification and analysis method for campus security monitoring systems

    CN119721732B

  • Campus intelligent monitoring method for preventing campus bullying

    CN121121656A