Security scene recognition method, device, equipment and readable storage medium

Abstract

This invention relates to the field of artificial intelligence-based security, and provides a method, apparatus, device, and readable storage medium for security scene recognition. The method includes: selecting security scene images from historical scene images; training a target detection model based on the security scene images; generating a first scene image using the trained target detection model, wherein the first scene image is used to train a network model incorporating skeletal point detection; and recognizing the acquired target scene images based on the trained network model to obtain a security scene recognition result. This invention reduces the deployment and maintenance costs of the network model by generating a network model incorporating skeletal point detection from the trained target detection model, and improves the accuracy of security scene recognition by using the trained network model incorporating skeletal point detection.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security, and in particular to a security scene recognition method, apparatus, device, and readable storage medium. Background Technology

[0002] With the continuous development of deep learning networks and the advancement of computer hardware technology, artificial intelligence-related applications have permeated all aspects of production and life. In particular, in daily security scenarios, visual security technology is gradually becoming the mainstream method to replace manual monitoring. Manual monitoring is limited in terms of location, area, and time, and is also costly. Computer vision can play a more comprehensive and wider role in monitoring.

[0003] In campus security scenarios, surveillance cameras are mainly deployed in areas such as playgrounds, school gates, and teaching buildings. Existing security scene recognition is mainly based on existing image recognition technology using cloud or local servers, which results in relatively high deployment and maintenance costs and low recognition accuracy. Summary of the Invention

[0004] This invention provides a security scene recognition method, apparatus, device, and readable storage medium to solve the technical problems of relatively high deployment and maintenance costs and low recognition accuracy in existing security scene recognition solutions.

[0005] This invention provides a security scene recognition method, comprising:

[0006] Security scene images are selected from historical scene images, and a target detection model is trained based on the security scene images;

[0007] A first scene image is generated by a trained object detection model, and the first scene image is used to train a network model that incorporates skeletal point detection.

[0008] The trained network model is used to identify the collected target scene images to obtain security scene recognition results.

[0009] According to a security scene recognition method provided by the present invention, the step of training a target detection model based on the security scene image includes:

[0010] The security scene images are labeled, and the labeled security scene images are input into the target detection model;

[0011] Based on the loss value output by the target detection model, the parameters of the target detection model are adjusted.

[0012] The target detection model is considered to have completed training when the number of parameter adjustments to the target detection model reaches a first preset threshold, or when the loss value is less than a second preset threshold.

[0013] According to a security scene recognition method provided by the present invention, the step of generating a first scene image by means of a trained target detection model includes:

[0014] The acquired video stream is input into the trained target detection model, which then performs segmentation and scene detection on the video stream to obtain a first scene image.

[0015] According to a security scene recognition method provided by the present invention, the training of a network model based on the first scene image with added skeletal point detection includes:

[0016] The first scene image with scene labels is input into a network model with skeletal point detection, and the pose features of the first scene image are determined by the skeletal point detection.

[0017] The scene probability result output by the network model is determined based on the posture features;

[0018] Based on the scene probability results and the scene labels, the parameters of the network model are adjusted;

[0019] Based on the adjustment results of the network model's parameters, determine whether the network model has completed training.

[0020] According to the security scene recognition method provided by the present invention, the step of recognizing the acquired target scene image based on a trained network model to obtain the security scene recognition result includes:

[0021] The acquired target scene image is input into the trained network model, and the pose information of the target scene image is extracted through the trained network model;

[0022] Based on the posture information, determine whether the target scene image belongs to the target security scene;

[0023] If the target scene image belongs to the target security scene, the target security scene is determined as the security scene recognition result of the target scene image.

[0024] According to a security scene recognition method provided by the present invention, the step of recognizing the acquired target scene image based on a trained network model to obtain the security scene recognition result includes:

[0025] A quantized network model is generated based on the application programming interface of the framework accompanying the edge computing terminal.

[0026] According to the security scene recognition method provided by the present invention, the step of generating a quantized network model based on the application programming interface of the framework supporting the edge computing terminal includes:

[0027] Adjust the parameters of the quantized network model described in the weight file to enable the quantized network model to run at the edge computing terminal.

[0028] The present invention also provides a security scene recognition device, comprising:

[0029] The target detection model training module is used to filter out security scene images from historical scene images and train a target detection model based on the security scene images;

[0030] The first scene image generation module is used to generate a first scene image through a trained object detection model. The first scene image is used to train a network model that incorporates skeletal point detection.

[0031] The security scene recognition module is used to recognize the collected target scene images based on the trained network model and obtain the security scene recognition results.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the security scene recognition method as described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the security scene recognition method as described above.

[0034] The security scene recognition method, apparatus, device, and readable storage medium provided by this invention first selects security scene images from historical scene images, trains a target detection model using the security scene images, then generates a first scene image using the trained target detection model. The first scene image is used to train a network model with skeletal point detection functionality. Finally, the collected target scene images are recognized based on the trained network model to obtain the security scene recognition result of the target scene images. By generating a network model with skeletal point detection using the trained target detection model, the deployment and maintenance costs of the network model are reduced. Furthermore, by using the trained network model with skeletal point detection for security scene recognition, the accuracy of security scene recognition is improved. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is one of the flowcharts of the security scene recognition method provided by the present invention;

[0037] Figure 2 This is the second flowchart of the security scene recognition method provided by the present invention;

[0038] Figure 3 This is a schematic diagram of the security scene recognition device provided by the present invention;

[0039] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0041] Please refer to Figure 1 This invention provides a security scene recognition method, including:

[0042] Step 100: Select security scene images from historical scene images, and train a target detection model based on the security scene images;

[0043] Specifically, images containing specific scenes are selected from historically collected scene images through manual or automatic computer identification. These are the security scene images in this embodiment. Taking campus security scene recognition as an example, images containing specific scenes could include scenes such as people falling, people chasing and playing, and people fighting. After selecting security scene images containing specific scenes, they are tagged based on the aforementioned scene categories (people falling, people chasing and playing, and people fighting, etc.) to obtain security scene images with specific scene tags.

[0044] Security scene images with specific scene labels are input into the target detection model to train the target detection model. During the training process, the relevant parameters of the target detection model are adjusted according to the loss value output by the target detection model. By continuously optimizing the loss value output by the target detection model through parameter adjustment, the loss value output by the target detection model is minimized, thus completing the training of the target detection model.

[0045] Step 200: Generate a first scene image using the trained object detection model. The first scene image is used to train a network model that incorporates skeletal point detection.

[0046] Specifically, the trained object detection model is used to generate images containing specific scenes, namely the first scene image in this embodiment. It is understood that the method of filtering security scene images from historical scene images is time-consuming and has a limited scope, only allowing filtering from historical scene images. Generating images containing specific scenes using the trained object detection model not only reduces the time cost of acquiring security scene images but also solves the problem of limited scope. Generating the first scene image using the trained object detection model facilitates the initial screening of the required scene image dataset.

[0047] Based on a deep learning object detection network, a modified skeletal point detection network was added to the backbone of the deep learning object detection network. The first scene image generated by the object detection model was used to train the network model with skeletal point detection. The network model with skeletal point detection was used to capture the parts of the (campus) security scene that require pose information.

[0048] Step 300: Based on the trained network model, the collected target scene images are identified to obtain the security scene recognition results.

[0049] Specifically, after the network model with skeletal point detection is trained, the target detection model for the required scene is trained simultaneously. In the real detection scene, the posture of pedestrians in the scene image is first detected by skeletal point detection. Preliminary judgment is made on postures such as fighting, falling, and chasing and playing. Scene images with certain tendencies are selected and further target detection is performed to determine whether the above behaviors exist in the collected target scene image, thus obtaining the security scene recognition result of the target scene image.

[0050] This embodiment first selects security scene images from historical scene images, trains a target detection model using these images, and then generates a first scene image using the trained target detection model. This first scene image is used to train a network model with skeletal point detection functionality. Finally, the collected target scene images are recognized based on the trained network model to obtain the security scene recognition result. By generating a network model with skeletal point detection using the trained target detection model, the deployment and maintenance costs of the network model are reduced. Furthermore, by using the trained network model with skeletal point detection for security scene recognition, the accuracy of security scene recognition is improved.

[0051] In one embodiment, the security scene recognition method provided in this application may further include:

[0052] Step 110: Label the security scene image and input the labeled security scene image into the target detection model;

[0053] Step 120: Adjust the parameters of the target detection model based on the loss value output by the target detection model;

[0054] Step 130: If the number of parameter adjustments of the target detection model reaches a first preset threshold, or the loss value is less than a second preset threshold, the target detection model is determined to have completed training.

[0055] Specifically, before training the object detection model, security scene images selected from historical scene images are labeled to obtain a set of security scene images with classification labels. The categories of the classification labels include the aforementioned scene categories, such as people falling, people chasing and playing, and people fighting. Then, the security scene images labeled with classification labels are input into the object detection model. The object detection model calculates the predicted probability of the security scene image belonging to each category. For example, the predicted probability corresponding to the fourth category means: given the weights, the security scene image belongs to the fourth category. Based on the predicted probabilities of the security scene image belonging to each category calculated by the object detection model, the loss value output by the object detection model is determined. Then, based on the loss value output by the object detection model, the parameters of the object detection model (such as the aforementioned known weights) are adjusted in the direction of increasing the predicted probability value of the accurate category corresponding to the security scene image.

[0056] The parameters of the object detection model also include the number of iterations, i.e. the number of parameter adjustments. The object detection model is considered to have completed training when the number of parameter adjustments reaches a first preset threshold or the loss value is less than a second preset threshold.

[0057] This embodiment trains a target detection model to facilitate the subsequent generation of images containing specific scenes, reducing the time cost of acquiring security scene images and solving the problem of limited acquisition range of specific scene images.

[0058] In one embodiment, the security scene recognition method provided in this application may further include:

[0059] Step 210: Input the acquired video stream into the trained target detection model, and use the trained target detection model to extract and detect scenes from the video stream to obtain a first scene image.

[0060] Specifically, firstly, historical scene images are collected using high-frame-rate and high-quality cameras to capture campus security scene data. Scenes such as people falling, fighting, and chasing / playing are selected from these historical images. The selected scene data is then labeled manually or automatically by computer, and a simple scene object detection model is initially trained. The captured video stream is then input into the trained object detection model for initial screening and retention of scene data. This step can be repeated to gradually increase the amount of scene data and train the object detection model. The object detection model is then used to generate the first scene image, further increasing the scene data. During the video stream input process, due to the limited amount of training data, a large number of false positive images are captured. These false positive images provide sufficient negative samples for subsequent formal training, improving the model's generalization ability.

[0061] This embodiment increases scene data by using a trained target detection model, reducing the time cost of acquiring security scene images and solving the problem of limited image acquisition range for specific scenes.

[0062] In one embodiment, the security scene recognition method provided in this application may further include:

[0063] Step 220: Input the first scene image with scene label into the network model with skeleton point detection, and determine the pose features of the first scene image through the skeleton point detection.

[0064] Step 230: Determine the scene probability result output by the network model based on the pose features;

[0065] Step 240: Adjust the parameters of the network model based on the scene probability results and the scene labels;

[0066] Step 250: Determine whether the network model has been trained successfully based on the adjustment results of the network model parameters.

[0067] Specifically, the training process of a network model with skeletal point detection is similar to that of an object detection model. A first scene image with scene labels is input into the network model. Pose features of the first scene image are extracted through skeletal point detection. Based on these extracted pose features, the network model outputs a scene probability result—the probability of belonging to a particular scene. Then, based on the scene probability result output by the network model and the true scene labels, the parameters of the network model are adjusted. Finally, based on the adjusted parameters, it is determined whether the network model has completed training.

[0068] This embodiment improves the accuracy of security scene recognition by training a network model with skeletal point detection.

[0069] Please refer to Figure 2 In one embodiment, the security scene recognition method provided in this application may further include:

[0070] Step 310: Input the acquired target scene image into the trained network model, and extract the pose information of the target scene image through the trained network model;

[0071] Step 320: Determine whether the target scene image belongs to the target security scene based on the posture information;

[0072] Step 330: If the target scene image belongs to the target security scene, determine the target security scene as the security scene recognition result of the target scene image.

[0073] Specifically, a modified deep learning object detection network is used, adding a skeletal point detection component to the backbone network. This is primarily used to capture pose information needed in specific security scenarios (such as campuses). Compared to traditional methods, such as directly analyzing frame-by-frame images or optical flow, the skeletal pose modality is more consistent with human movements and better reflects key features. After training the network model using skeletal points, a target detection model for the required scene is simultaneously trained. In real detection scenarios, the pose of pedestrians in the image is first detected using skeletal points. Preliminary judgments are made for poses such as fighting, falling, and chasing, filtering out images with certain tendencies for the next step of target detection to determine whether the above behaviors exist. That is, based on pose information, it is determined whether the target scene image belongs to the target security scene. If the target scene image belongs to the target security scene, the target security scene is identified as the security scene recognition result of the target scene image.

[0074] This embodiment uses a network model with skeletal point detection to identify security scenes, thereby improving the accuracy of security scene recognition.

[0075] In one embodiment, the security scene recognition method provided in this application may further include:

[0076] Step 400: Generate a quantized network model based on the application programming interface of the framework supporting the edge computing terminal;

[0077] Step 500: Adjust the parameters of the quantized network model in the weight file to enable the quantized network model to run at the edge computing terminal.

[0078] Specifically, because the recognition scenario needs to be placed on the edge computing side, a quantization model needs to be generated based on the API (Application Programming Interface) of the framework supporting the edge computing module. This requires writing the relevant computational structures of the modified operators involved in step one. Simultaneously, GPU (graphics processing unit) decoding is needed to accelerate the frame extraction and decoding speed of the video stream, thereby increasing the number of recognized frames and the maximum number of supported video channels. Based on the requirements of the edge computing box, either int8-bit or float16-bit model quantization compression is performed. Here, int is a data type used to define identifiers for integer variables, and float stores single-precision or double-precision floating-point numbers. The weight parameters with a float32-bit data width in the weight file are converted to int8-bit data width weight parameters. Generating an int8-bit model may result in insufficient precision and computational bias, requiring some correction. Fine-tuning is needed using a simple network input training set images. After generating the int8-bit data width weight model, relevant encoding / decoding code and recognition code are written according to the recognition interface of the edge computing module framework. Recognition is then performed, and performance is tested.

[0079] This embodiment ensures high-speed and high-precision scene recognition at the edge by optimizing and adapting the model of the edge module.

[0080] The security scene recognition device provided by the present invention is described below. The security scene recognition device described below and the security scene recognition method described above can be referred to in correspondence.

[0081] Please refer to Figure 3 The present invention also provides a security scene recognition device, comprising:

[0082] The target detection model training module 301 is used to filter out security scene images from historical scene images and train a target detection model based on the security scene images;

[0083] The first scene image generation module 302 is used to generate a first scene image through a trained object detection model. The first scene image is used to train a network model with added skeletal point detection.

[0084] The security scene recognition module 303 is used to recognize the collected target scene images based on the trained network model to obtain the security scene recognition result.

[0085] Optionally, the object detection model training module includes:

[0086] The image annotation unit is used to annotate the security scene image and input the security scene image with the annotated classification label into the target detection model;

[0087] The first parameter adjustment unit is used to adjust the parameters of the target detection model according to the loss value output by the target detection model;

[0088] The target detection model training unit is used to determine that the target detection model training is complete when the number of parameter adjustments of the target detection model reaches a first preset threshold or the loss value is less than a second preset threshold.

[0089] Optionally, the first scene image generation module includes:

[0090] The first scene image generation unit is used to input the acquired video stream into the trained target detection model, and to perform scene detection and segmentation on the video stream through the trained target detection model to obtain the first scene image.

[0091] Optionally, the first scene image generation module further includes:

[0092] The pose feature determination unit is used to input a first scene image with scene labels into a network model with skeletal point detection, and determine the pose features of the first scene image through the skeletal point detection.

[0093] The scene probability result determination unit is used to determine the scene probability result output by the network model based on the pose features;

[0094] The second parameter adjustment unit is used to adjust the parameters of the network model based on the scene probability results and the scene labels;

[0095] The network model training unit is used to determine whether the network model has been trained successfully based on the adjustment results of the network model's parameters.

[0096] Optionally, the security scene recognition module includes:

[0097] The pose information extraction unit is used to input the acquired target scene image into the trained network model and extract the pose information of the target scene image through the trained network model.

[0098] A target security scene determination unit is used to determine whether the target scene image belongs to a target security scene based on the attitude information;

[0099] A security scene recognition unit is used to determine the target security scene as the security scene recognition result of the target scene image when the target scene image belongs to the target security scene.

[0100] Optionally, the security scene recognition device further includes:

[0101] The quantized network model generation module is used to generate quantized network models based on the application programming interface of the framework supporting the edge computing terminal.

[0102] Optionally, the security scene recognition device further includes:

[0103] The weight file parameter adjustment module is used to adjust the parameters of the quantized network model in the weight file so that the quantized network model can run at the edge computing terminal.

[0104] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions stored in the memory 430 to execute a security scene recognition method.

[0105] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the security scene recognition methods provided by the above methods.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A security scene recognition method, characterized in that, include: Security scene images are selected from historical scene images, and a target detection model is trained based on the security scene images; The acquired video stream is input into a trained object detection model. The trained object detection model is used to extract and detect scenes from the video stream to obtain a first scene image. The first scene image is used to train a network model with added skeletal point detection. The network model is based on the addition of skeletal point detection to the backbone network of the deep learning object detection network. Based on the trained network model, the collected target scene images are identified to obtain security scene recognition results; The target detection model trained based on the security scene image includes: The security scene images are labeled, and the labeled security scene images are input into the target detection model; Based on the loss value output by the target detection model, the parameters of the target detection model are adjusted. If the number of parameter adjustments of the target detection model reaches a first preset threshold, or if the loss value is less than a second preset threshold, the target detection model is determined to have completed training. The training steps of the network model include: The first scene image with scene labels is input into a network model with skeletal point detection, and the pose features of the first scene image are determined by the skeletal point detection. The scene probability result output by the network model is determined based on the posture features; Based on the scene probability results and the scene labels, the parameters of the network model are adjusted; Based on the adjustment results of the network model's parameters, determine whether the network model has completed training.

2. The security scene recognition method according to claim 1, characterized in that, The security scene recognition results obtained by recognizing the acquired target scene images based on the trained network model include: The acquired target scene image is input into the trained network model, and the pose information of the target scene image is extracted through the trained network model; Based on the posture information, determine whether the target scene image belongs to the target security scene; If the target scene image belongs to the target security scene, the target security scene is determined as the security scene recognition result of the target scene image.

3. The security scene recognition method according to claim 1, characterized in that, The process of identifying the acquired target scene image based on the trained network model to obtain the security scene identification result includes: A quantized network model is generated based on the application programming interface of the framework accompanying the edge computing terminal.

4. The security scene recognition method according to claim 3, characterized in that, After generating the quantized network model based on the application programming interface of the framework supporting the edge computing terminal, the process includes: Adjust the parameters of the quantized network model described in the weight file to enable the quantized network model to run at the edge computing terminal.

5. A security scene recognition device, characterized in that, include: The target detection model training module is used to filter out security scene images from historical scene images and train a target detection model based on the security scene images; The first scene image generation module is used to input the acquired video stream into a trained target detection model, and to perform scene detection and cropping on the video stream through the trained target detection model to obtain a first scene image. The first scene image is used to train a network model with added skeletal point detection. The network model is based on the addition of skeletal point detection to the backbone network of the deep learning target detection network. The security scene recognition module is used to recognize the collected target scene images based on the trained network model and obtain the security scene recognition results. The target detection model trained based on the security scene image includes: The security scene images are labeled, and the labeled security scene images are input into the target detection model; Based on the loss value output by the target detection model, the parameters of the target detection model are adjusted. If the number of parameter adjustments of the target detection model reaches a first preset threshold, or if the loss value is less than a second preset threshold, the target detection model is determined to have completed training. The training steps of the network model include: The first scene image with scene labels is input into a network model with skeletal point detection, and the pose features of the first scene image are determined by the skeletal point detection. The scene probability result output by the network model is determined based on the posture features; Based on the scene probability results and the scene labels, the parameters of the network model are adjusted; Based on the adjustment results of the network model's parameters, determine whether the network model has completed training.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the security scene recognition method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the security scene recognition method as described in any one of claims 1 to 4.

Images