Abnormal behavior identification method, device, electronic device and storage medium
Through the processing of image datasets and the training of convolutional neural networks, combined with pedestrian object detector and abnormal behavior classifier, the problem of low recognition accuracy caused by occlusion when crowds gather is solved, and the recognition accuracy of abnormal behavior is improved. It is suitable for wisdom City applications.
Patent Information
- Application Number
- CN202011077906.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-10
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-10-10
AI Technical Summary
In the prior art, when identifying abnormal behavior, especially when crowds are gathered, the recognition accuracy is low due to occlusion problems, making it difficult to effectively identify abnormal behavior.
By acquiring the image dataset, pedestrian detection, clustering and merging coordinate boxes with pre-trained pedestrian object detectors, intercepting image samples, and training convolutional neural networks to obtain anomaly behavior classifier models, combining pedestrian object detectors and The classifier model is used to identify it to improve the recognition accuracy.
It improves the accuracy of identification of abnormal behaviors, reduces the key points characteristics of artificial design, enhances the accuracy of the model, and is suitable for smart security, smart transportation and other fields.
Smart Images

Figure CN112200081B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an abnormal behavior recognition method, device, electronic device and storage medium. Background Art
[0002] As the population density increases, many abnormal incidents such as fighting often occur in communities and parks, which pose a great threat to social security. At present, when identifying abnormal behaviors, the algorithm usually adopted is based on posture estimation, and the classifier is trained based on the extracted key points. This method strongly relies on the recognition accuracy of the key points of the human body, but when fighting occurs, there will be many occlusions, especially when people gather, resulting in inaccurate recognition.
[0003] Therefore, how to improve the recognition rate of abnormal behavior is a technical problem that needs to be solved urgently. Summary of the invention
[0004] In view of the above, it is necessary to provide an abnormal behavior recognition method, device, electronic device and storage medium, which can improve the recognition rate of abnormal behavior.
[0005] A first aspect of the present invention provides an abnormal behavior recognition method, the abnormal behavior recognition method comprising:
[0006] Get image dataset;
[0007] Based on a pre-trained pedestrian target detector, pedestrian detection is performed on each image in the image dataset to obtain a pedestrian coordinate frame;
[0008] Clustering the pedestrian coordinate frames according to a set inter-class distance, and merging the clustered pedestrian coordinate frames to obtain a new coordinate frame;
[0009] According to the new coordinate frame, the image is intercepted to obtain a plurality of screenshot samples;
[0010] Based on the multiple screenshot samples, training a convolutional neural network to obtain an abnormal behavior classifier model;
[0011] According to the pedestrian target detector and the abnormal behavior classifier model, the image to be detected is identified to obtain the abnormal behavior probability;
[0012] An abnormal behavior recognition result is determined according to the abnormal behavior probability.
[0013] In a possible implementation, before acquiring the image data set, the abnormal behavior identification method further includes:
[0014] Get training set and validation set;
[0015] Setting an initial learning rate and an adjusted learning rate, wherein the initial learning rate is greater than the adjusted learning rate;
[0016] Based on the initial learning rate and the Adam optimization algorithm, the YOLOv3 framework is trained using the training set to obtain an intermediate model;
[0017] Inputting the verification set into the intermediate model, and when the loss function of the verification set reaches convergence, training the intermediate model based on the adjusted learning rate and the stochastic gradient descent (SGD) algorithm;
[0018] When the loss function of the verification set reaches convergence, it is determined that the current converged model is a pedestrian target detector.
[0019] In a possible implementation, clustering the pedestrian coordinate frames according to the set inter-class distance includes:
[0020] Sorting the pedestrian coordinate frames according to the probability of each pedestrian coordinate frame from high to low to obtain a sorted frame;
[0021] The coordinate frame of the pedestrian with the highest probability is determined as the reference frame;
[0022] Sequentially traverse the remaining frames in the sorted frame that are sorted after the reference frame, and calculate the Euclidean distance between the reference frame and each of the remaining frames;
[0023] The pedestrian coordinate frames are clustered according to the set inter-class distance and the Euclidean distance.
[0024] In a possible implementation, merging the clustered pedestrian coordinate frames to obtain a new coordinate frame includes:
[0025] Merge the clustered pedestrian coordinate frames to obtain a first coordinate frame;
[0026] Determining whether the width of the first coordinate frame is less than a first threshold, and determining whether the height of the first coordinate frame is less than a second threshold;
[0027] If the width of the first coordinate frame is smaller than a first threshold value, and the height of the first coordinate frame is smaller than a second threshold value, obtaining the center point of the first coordinate frame, and obtaining a preset width and a preset height;
[0028] Determining whether a second coordinate frame based on the center point, the preset width, and the preset height exceeds the boundary coordinates of the image;
[0029] If the second coordinate frame based on the center point, the preset width, and the preset height does not exceed the boundary coordinates of the image, the second coordinate frame is determined to be a new coordinate frame.
[0030] In a possible implementation, the abnormal behavior identification method further includes:
[0031] If the second coordinate frame based on the center point, the preset width and the preset height exceeds the boundary coordinates of the image, a new coordinate frame is determined based on the center point and the boundary coordinates.
[0032] In a possible implementation, the step of training a convolutional neural network based on the multiple screenshot samples to obtain an abnormal behavior classifier model includes:
[0033] Receiving positive samples and negative samples manually annotated for the plurality of screenshot samples, wherein the positive samples are samples with abnormal behaviors, and the negative samples are samples without abnormal behaviors;
[0034] Set the learning rate, stochastic gradient descent SGD algorithm, and training times threshold;
[0035] Using the positive sample and the negative sample, training the ResNet18 framework based on the learning rate and the SGD algorithm;
[0036] When the number of training times reaches the training times threshold, it is determined that the training is finished, and the model at the end is determined as the abnormal behavior classifier model.
[0037] In a possible implementation, the identifying the image to be detected based on the pedestrian target detector and the abnormal behavior classifier model to obtain the abnormal behavior probability includes:
[0038] Using the pedestrian target detector to detect the image to be detected, and obtaining a target coordinate frame of each pedestrian in the image to be detected;
[0039] Clustering and merging the target coordinate frames to obtain a coordinate frame to be detected;
[0040] Intercepting a screenshot to be detected corresponding to the coordinate frame to be detected from the image to be detected;
[0041] Preprocessing the screenshot to be detected to obtain a preprocessed image;
[0042] The abnormal behavior classifier model is used to identify the preprocessed image to obtain the abnormal behavior probability.
[0043] A second aspect of the present invention provides an abnormal behavior recognition device, the abnormal behavior recognition device comprising:
[0044] An acquisition module, used to acquire an image data set;
[0045] A detection module, used to perform pedestrian detection on each image in the image data set based on a pre-trained pedestrian target detector to obtain a pedestrian coordinate frame;
[0046] A merging module, used for clustering the pedestrian coordinate frames according to a set inter-class distance, and merging the clustered pedestrian coordinate frames to obtain a new coordinate frame;
[0047] A capture module, used to capture the image according to the new coordinate frame to obtain a plurality of screenshot samples;
[0048] A training module, used to train a convolutional neural network based on the multiple screenshot samples to obtain an abnormal behavior classifier model;
[0049] A recognition module, used to recognize the image to be detected according to the pedestrian target detector and the abnormal behavior classifier model, and obtain the abnormal behavior probability;
[0050] A determination module is used to determine an abnormal behavior recognition result according to the abnormal behavior probability.
[0051] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the processor is configured to implement the abnormal behavior identification method when executing a computer program stored in the memory.
[0052] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the abnormal behavior identification method when executed by a processor.
[0053] It can be seen from the above technical solutions that the present invention can be applied to fields such as smart security, smart transportation, smart communities and smart life that require abnormal behavior recognition, thereby promoting the development of smart cities. In the present invention, a convolutional neural network is trained by using an image data set that is easier to obtain rather than a video data set, and by obtaining multiple screenshot samples through model processing, which can provide more and more complete data support for model training. At the same time, since the key point features designed artificially in the prior art are reduced, the trained model is more accurate and can improve the accuracy of abnormal behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a flow chart of a preferred embodiment of an abnormal behavior identification method disclosed by the present invention.
[0055] Figure 2 It is a functional module diagram of a preferred embodiment of an abnormal behavior identification device disclosed in the present invention.
[0056] Figure 3 It is a schematic diagram of the structure of an electronic device of a preferred embodiment of the abnormal behavior identification method of the present invention. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0058] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0059] It should be noted that the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
[0060] The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc. The electronic device may also include network devices and / or user devices. Among them, the network device includes but is not limited to a single network server, a server group consisting of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing. The user device includes but is not limited to any electronic product that can interact with the user through a keyboard, mouse, remote control, touchpad or voice control device, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant PDA, etc.
[0061] See also Figure 1 , Figure 1 The flowchart of a preferred embodiment of an abnormal behavior identification method disclosed in the present invention is shown in FIG.
[0062] S11. Obtain an image dataset.
[0063] Wherein, the image data set includes multiple images.
[0064] Specifically, the acquiring of the image data set includes:
[0065] Obtain a video stream from a video surveillance device, extract frames from the video stream, obtain multiple images after the extraction, and construct an image data set; or
[0066] Use crawler technology to obtain multiple images from the Internet and build an image dataset.
[0067] In this optional implementation, a video stream can be obtained from a video surveillance device, wherein the video stream is usually an rtsp (Real Time Streaming Protocol) in the form of a url (Uniform Resource Locator), and the server can parse the rtsp stream and convert it into frames of video images by loop reading to construct an image data set; or, optionally, multiple images can be obtained from the Internet through crawler technology and an image data set can be constructed. The image data set obtained in this embodiment is easier to obtain than a traditional video data set, and can provide more and more complete data support for subsequent model training, which is conducive to improving the accuracy of model training.
[0068] S12. Based on a pre-trained pedestrian target detector, pedestrian detection is performed on each image in the image dataset to obtain a pedestrian coordinate frame.
[0069] Specifically, each image in the image data set can be scaled to a fixed size (such as 416*416) and input into a trained pedestrian target detector to obtain tens of thousands of anchor-based original coordinate frames, each of which has a corresponding probability value. According to the probability values of these original coordinate frames, the original coordinate frames with probability values greater than the preset threshold are screened, the original coordinate frames with probability values less than the preset threshold are removed, and the original coordinate frames retained are subjected to non-maximum suppression, such as suppression by the NMS (Non-maximum suppression) algorithm. The retained original coordinate frames are sorted from high to low according to probability, and the original coordinate frame with the highest current probability is selected as the reference frame. The original coordinate frames sorted after the reference frame are traversed, and the intersection and union ratio of each original coordinate frame sorted after the reference frame is compared with the reference frame. The original coordinate frames with high overlap are removed, and the traversal is repeated to obtain the pedestrian coordinate frame corresponding to each person in the final image.
[0070] Optionally, the method further includes:
[0071] Get training set and validation set;
[0072] Setting an initial learning rate and an adjusted learning rate, wherein the initial learning rate is greater than the adjusted learning rate;
[0073] Based on the initial learning rate and the Adam optimization algorithm, the YOLOv3 framework is trained using the training set to obtain an intermediate model;
[0074] Inputting the verification set into the intermediate model, and when the loss function of the verification set reaches convergence, training the intermediate model based on the adjusted learning rate and the stochastic gradient descent (SGD) algorithm;
[0075] When the loss function of the verification set reaches convergence, it is determined that the current converged model is a pedestrian target detector.
[0076] In this optional implementation, a pedestrian data set is obtained in advance, and the pedestrian data set includes images and labels, and the images are in common formats such as jpg or png. Among them, the horizontal direction of the image is the x-axis, the vertical direction is the y-axis, and the upper left corner of the image is the origin. The label is used to identify the center point coordinates of the rectangular box containing each pedestrian in each image and the width / height of the rectangular box. Among them, the format of the label can be an xml or txt file. Further, the pedestrian data set is divided into a training set and a verification set, the training set is used to train the pedestrian target detector, and the verification set is used to determine when to end the training. Optionally, the optimization method in the training stage is designed to be Adam+SGD, that is, the Adam algorithm is used in the early stage of training to improve the convergence speed, and the stochastic gradient descent (SGD) algorithm is used in the later stage to ensure model convergence. Among them, the probability loss is set as the cross entropy loss function, the coordinate loss is set as the mean square error loss function, and L2 regularization is set to suppress overfitting, and the data enhancement method is set. Optionally, the data enhancement method may include but is not limited to random horizontal flipping, image color temperature and saturation adjustment and random rotation. Set the learning rate decreasing mode. Specifically, set the initial learning rate, such as 0.001, and set the adjusted learning rate to 0.0001. When the initial learning rate is used for training, when the loss function of the validation set reaches the first convergence, the loss value is the first minimum value. After that, when the adjusted learning rate is used for training, the loss function of the validation set will change from the converged state. As the training progresses, it will slowly converge again. When the loss function of the validation set reaches the second convergence, the loss value is the second minimum value, and the training ends at this time. Among them, the first minimum value is greater than the second minimum value.
[0077] S13. Clustering the pedestrian coordinate frames according to the set inter-class distance, and merging the clustered pedestrian coordinate frames to obtain new coordinate frames.
[0078] Specifically, clustering the pedestrian coordinate frames according to the set inter-class distance includes:
[0079] Sorting the pedestrian coordinate frames according to the probability of each pedestrian coordinate frame from high to low to obtain a sorted frame;
[0080] The coordinate frame of the pedestrian with the highest probability is determined as the reference frame;
[0081] Sequentially traverse the remaining frames in the sorted frame that are sorted after the reference frame, and calculate the Euclidean distance between the reference frame and each of the remaining frames;
[0082] The pedestrian coordinate frames are clustered according to the set inter-class distance and the Euclidean distance.
[0083] In this optional implementation, it is not necessary to set the number of clustering categories, only the inter-class distance needs to be set. Among them, one fifth of the square root of the image area can be set as the inter-class distance according to the image size and the size of the pedestrian in the image. When the Euclidean distance between the center point of a certain remaining frame and the center point of the reference frame is less than the set inter-class distance, it is determined that the remaining frame and the reference frame belong to the same category. Repeat the traversal of the remaining frames and cluster the pedestrian coordinate frames according to the set inter-class distance and the Euclidean distance until all pedestrian coordinate frames are classified into a certain category.
[0084] Specifically, the clustered pedestrian coordinate frames are merged to obtain a new coordinate frame, including:
[0085] Merge the clustered pedestrian coordinate frames to obtain a first coordinate frame;
[0086] Determining whether the width of the first coordinate frame is less than a first threshold, and determining whether the height of the first coordinate frame is less than a second threshold;
[0087] If the width of the first coordinate frame is smaller than a first threshold value, and the height of the first coordinate frame is smaller than a second threshold value, obtaining the center point of the first coordinate frame, and obtaining a preset width and a preset height;
[0088] Determining whether a second coordinate frame based on the center point, the preset width, and the preset height exceeds the boundary coordinates of the image;
[0089] If the second coordinate frame based on the center point, the preset width, and the preset height does not exceed the boundary coordinates of the image, the second coordinate frame is determined to be a new coordinate frame.
[0090] In this optional implementation, for the pedestrian coordinate frames classified into one category, it is necessary to merge them into a new coordinate frame, and the new coordinate frame is set to be the frame with the smallest area containing this category of frames. Specifically, for each category, the smallest horizontal coordinate / vertical coordinate is found from the upper left corner vertex of all the frames of this category, recorded as x1 and y1, and the largest horizontal coordinate / vertical coordinate is found from the lower right corner vertex of all the frames of this category, recorded as x2 and y2, (x1, y1), (x2, y2) are the upper left corner vertex and lower right corner vertex of the first coordinate frame. In addition, the first threshold and the second threshold can be preset, for example, both are set to 224 pixels. If the width of the first coordinate frame is less than the first threshold, and the height of the first coordinate frame is less than the second threshold, it is also necessary to determine whether the second coordinate frame based on the center point, the preset width and the preset height exceeds the boundary coordinates of the image. If it does not exceed the boundary coordinates of the image, the second coordinate frame can be determined as a new coordinate frame. Among them, the preset width and the preset height can be set according to the first threshold and the second preset, for example, the preset width and the preset height are both 224 pixels. In this way, the accuracy of subsequent models in identifying abnormal behaviors can be improved.
[0091] Optionally, the method further includes:
[0092] If the second coordinate frame based on the center point, the preset width and the preset height exceeds the boundary coordinates of the image, a new coordinate frame is determined based on the center point and the boundary coordinates.
[0093] In this optional implementation, if the boundary coordinates of the image are exceeded, it is necessary to truncate the image based on the boundary coordinates of the image, that is, determine a new coordinate frame based on the center point and the boundary coordinates. In this way, the accuracy of subsequent model recognition of abnormal behavior can be improved.
[0094] S14. According to the new coordinate frame, the image is intercepted to obtain a plurality of screenshot samples.
[0095] A sub-image corresponding to the new coordinate frame may be captured from the image as a screenshot sample.
[0096] S15. Based on the multiple screenshot samples, train a convolutional neural network to obtain an abnormal behavior classifier model.
[0097] Specifically, the step of training a convolutional neural network based on the multiple screenshot samples to obtain an abnormal behavior classifier model includes:
[0098] Receiving positive samples and negative samples manually annotated for the plurality of screenshot samples, wherein the positive samples are samples with abnormal behaviors, and the negative samples are samples without abnormal behaviors;
[0099] Set the learning rate, stochastic gradient descent SGD algorithm, and training times threshold;
[0100] Using the positive sample and the negative sample, training the ResNet18 framework based on the learning rate and the SGD algorithm;
[0101] When the number of training times reaches the training times threshold, it is determined that the training is finished, and the model at the end is determined as the abnormal behavior classifier model.
[0102] In this optional implementation, after obtaining multiple screenshot samples, the screenshot samples can be classified by manual annotation, and the screenshot samples with abnormal behavior are classified as positive samples, and the screenshot samples without abnormal behavior are classified as negative samples. A convolutional neural network such as ResNet18 can be selected and loaded on ImageNet to pre-train model parameters. The training parameters can be set, such as setting the input image scaling size to 300*300, setting the random cropping to 224*224, setting random flipping, setting the learning rate to 0.001, setting the optimization method to SGD, setting the training number threshold, and reducing the learning rate to 0.0001 when the training reaches the predetermined training number threshold. After removing the mean and variance of the positive sample and the negative sample, the model training is started, and the training ends when the training reaches the predetermined training number threshold.
[0103] S16. According to the pedestrian target detector and the abnormal behavior classifier model, the image to be detected is identified to obtain the abnormal behavior probability.
[0104] Specifically, the identifying the image to be detected according to the pedestrian target detector and the abnormal behavior classifier model to obtain the abnormal behavior probability includes:
[0105] Using the pedestrian target detector to detect the image to be detected, and obtaining a target coordinate frame of each pedestrian in the image to be detected;
[0106] Clustering and merging the target coordinate frames to obtain a coordinate frame to be detected;
[0107] Intercepting a screenshot to be detected corresponding to the coordinate frame to be detected from the image to be detected;
[0108] Preprocessing the screenshot to be detected to obtain a preprocessed image;
[0109] The abnormal behavior classifier model is used to identify the preprocessed image to obtain the abnormal behavior probability.
[0110] In this optional embodiment, the preprocessing may include scaling the screenshot to be detected to a size randomly cropped during training, such as scaling it to a fixed size of 224*224, and performing mean and variance removal processing, wherein the mean and variance removal processing remain consistent with the data during training.
[0111] S17: Determine an abnormal behavior recognition result according to the abnormal behavior probability.
[0112] The range of the abnormal behavior probability is [0, 1], and the higher the probability value is, the higher the probability that fighting exists in the image to be detected.
[0113] Optionally, the method further includes:
[0114] If the abnormal behavior probability exceeds a preset probability threshold, packaging the coordinate frame to be detected, the screenshot to be detected, and the abnormal behavior probability into structured data;
[0115] The structured data is sent to a safety warning device.
[0116] In this optional implementation, when the probability of abnormal behavior exceeds a preset probability threshold (such as 0.5), the coordinate frame to be detected, the screenshot to be detected and the probability of abnormal behavior can be packaged as structured data and sent to the security warning device. This is beneficial for users of the security warning device to detect abnormalities in a timely manner and take corresponding preventive or stop measures, such as alarming, thereby improving social security and ensuring personal safety.
[0117] Optionally, for the privacy and security of data, the abnormal behavior identification results can also be uploaded to the blockchain.
[0118] exist Figure 1 In the described method flow, the convolutional neural network is trained by using a relatively easy-to-obtain image data set, which can provide more and more complete data support for model training, and is conducive to improving the accuracy of model training. At the same time, the images to be detected are identified by using the trained pedestrian target detector and abnormal behavior classifier model, which greatly reduces the artificially set features, thereby making the abnormal behavior more accurately identified and improving the recognition rate of abnormal behavior.
[0119] It can be seen from the above embodiments that the present invention can be applied to fields such as smart security, smart transportation, smart communities and smart life that require abnormal behavior identification, thereby promoting the development of smart cities.
[0120] The above is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. For ordinary technicians in this field, improvements can be made without departing from the creative concept of the present invention, but these all belong to the protection scope of the present invention.
[0121] See also Figure 2 , Figure 2 It is a functional module diagram of a preferred embodiment of an abnormal behavior identification device disclosed in the present invention.
[0122] In some embodiments, the abnormal behavior recognition device is run in an electronic device. The abnormal behavior recognition device may include a plurality of functional modules composed of program code segments. The program code of each program segment in the abnormal behavior recognition device may be stored in a memory and executed by at least one processor to perform Figure 1 For details, please refer to some or all of the steps in the abnormal behavior identification method described in Figure 1 The relevant description will not be repeated here.
[0123] In this embodiment, the abnormal behavior identification device can be divided into multiple functional modules according to the functions it performs. The functional modules may include: an acquisition module 201, a detection module 202, a merging module 203, an interception module 204, a training module 205, an identification module 206 and a determination module 207. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory.
[0124] The acquisition module 201 is used to acquire an image data set.
[0125] The detection module 202 is used to perform pedestrian detection on each image in the image data set based on a pre-trained pedestrian target detector to obtain a pedestrian coordinate frame.
[0126] The merging module 203 is used to cluster the pedestrian coordinate frames according to the set inter-class distance, and merge the clustered pedestrian coordinate frames to obtain new coordinate frames.
[0127] The capture module 204 is used to capture the image according to the new coordinate frame to obtain a plurality of capture samples.
[0128] The training module 205 is used to train a convolutional neural network based on the multiple screenshot samples to obtain an abnormal behavior classifier model.
[0129] The recognition module 206 is used to recognize the image to be detected according to the pedestrian target detector and the abnormal behavior classifier model to obtain the abnormal behavior probability.
[0130] The determination module 207 is used to determine the abnormal behavior recognition result according to the abnormal behavior probability.
[0131] exist Figure 2In the abnormal behavior recognition device described, the convolutional neural network is trained by using a relatively easy-to-obtain image data set, which can provide more and more complete data support for model training, and is conducive to improving the accuracy of model training. At the same time, the images to be detected are recognized by using the trained pedestrian target detector and abnormal behavior classifier model, which greatly reduces the artificially set features, thereby making the abnormal behavior recognized more accurately and improving the recognition rate of abnormal behavior.
[0132] like Figure 3 As shown, Figure 3 The electronic device 3 includes a memory 31, at least one processor 32, a computer program 33 stored in the memory 31 and executable on the at least one processor 32, and at least one communication bus 34.
[0133] Those skilled in the art will understand that Figure 3 The schematic diagram shown is merely an example of the electronic device 3 and does not constitute a limitation on the electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 3 may also include input and output devices, network access devices, etc.
[0134] The at least one processor 32 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 32 may be a microprocessor or any conventional processor, etc. The processor 32 is the control center of the electronic device 3, and uses various interfaces and lines to connect various parts of the entire electronic device 3.
[0135] The memory 31 can be used to store the computer program 33 and / or modules / units. The processor 32 implements various functions of the electronic device 3 by running or executing the computer program and / or modules / units stored in the memory 31 and calling the data stored in the memory 31. The memory 31 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the electronic device 3, etc. In addition, the memory 31 can include non-volatile and volatile memories, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other storage devices.
[0136] Combination Figure 1 The memory 31 in the electronic device 3 stores a plurality of instructions to implement an abnormal behavior identification method, and the processor 32 can execute the plurality of instructions to implement:
[0137] Get image dataset;
[0138] Based on a pre-trained pedestrian target detector, pedestrian detection is performed on each image in the image dataset to obtain a pedestrian coordinate frame;
[0139] Clustering the pedestrian coordinate frames according to a set inter-class distance, and merging the clustered pedestrian coordinate frames to obtain a new coordinate frame;
[0140] According to the new coordinate frame, the image is intercepted to obtain a plurality of screenshot samples;
[0141] Based on the multiple screenshot samples, training a convolutional neural network to obtain an abnormal behavior classifier model;
[0142] According to the pedestrian target detector and the abnormal behavior classifier model, the image to be detected is identified to obtain the abnormal behavior probability;
[0143] An abnormal behavior recognition result is determined according to the abnormal behavior probability.
[0144] Specifically, the specific implementation method of the processor 32 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.
[0145] exist Figure 3In the described electronic device 3, the convolutional neural network is trained by using a relatively easy-to-obtain image data set, which can provide more and more complete data support for model training, and is conducive to improving the accuracy of model training. At the same time, the images to be detected are identified by using trained pedestrian target detectors and abnormal behavior classifier models, which greatly reduces the artificially set features, thereby making the accuracy of abnormal behavior identification higher and improving the recognition rate of abnormal behavior.
[0146] If the module / unit integrated in the electronic device 3 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), etc.
[0147] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0148] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0149] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0150] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved. The multiple units or devices stated in the system claims can also be implemented by software or hardware.
[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A method for identifying abnormal behavior, characterized in that: The abnormal behavior identification method comprises: Get image dataset; Based on a pre-trained pedestrian target detector, pedestrian detection is performed on each image in the image data set to obtain a pedestrian coordinate frame, wherein the training method of the pedestrian target detector includes: obtaining a training set and a validation set; setting an initial learning rate and an adjusted learning rate, wherein the initial learning rate is greater than the adjusted learning rate; based on the initial learning rate and the Adam optimization algorithm, the YOLOv3 framework is trained using the training set to obtain an intermediate model; the validation set is input into the intermediate model, and when the loss function of the validation set reaches convergence, the intermediate model is trained based on the adjusted learning rate and the stochastic gradient descent SGD algorithm; when the loss function of the validation set reaches convergence, it is determined that the model at the time of convergence is a pedestrian target detector; According to the set inter-class distance, the pedestrian coordinate frames are clustered, and the clustered pedestrian coordinate frames are merged to obtain a new coordinate frame, wherein the merging of the clustered pedestrian coordinate frames to obtain the new coordinate frame includes: merging the clustered pedestrian coordinate frames to obtain a first coordinate frame; judging whether the width of the first coordinate frame is less than a first threshold, and judging whether the height of the first coordinate frame is less than a second threshold; if the width of the first coordinate frame is less than the first threshold, and the height of the first coordinate frame is less than the second threshold, obtaining the center point of the first coordinate frame, and obtaining a preset width and a preset height; judging whether a second coordinate frame based on the center point, the preset width and the preset height exceeds the boundary coordinates of the image; if the second coordinate frame based on the center point, the preset width and the preset height does not exceed the boundary coordinates of the image, determining that the second coordinate frame is a new coordinate frame; According to the new coordinate frame, the image is intercepted to obtain a plurality of screenshot samples; Based on the multiple screenshot samples, training a convolutional neural network to obtain an abnormal behavior classifier model; According to the pedestrian target detector and the abnormal behavior classifier model, the image to be detected is identified to obtain the abnormal behavior probability; An abnormal behavior recognition result is determined according to the abnormal behavior probability.
2. The abnormal behavior identification method according to claim 1, characterized in that: The clustering of the pedestrian coordinate frames according to the set inter-class distance includes: Sorting the pedestrian coordinate frames according to the probability of each pedestrian coordinate frame from high to low to obtain a sorted frame; The coordinate frame of the pedestrian with the highest probability is determined as the reference frame; Sequentially traverse the remaining frames in the sorted frame that are sorted after the reference frame, and calculate the Euclidean distance between the reference frame and each of the remaining frames; The pedestrian coordinate frames are clustered according to the set inter-class distance and the Euclidean distance.
3. The abnormal behavior identification method according to claim 1, characterized in that: The abnormal behavior identification method further includes: If the second coordinate frame based on the center point, the preset width and the preset height exceeds the boundary coordinates of the image, a new coordinate frame is determined based on the center point and the boundary coordinates.
4. The abnormal behavior identification method according to claim 1, characterized in that: The step of training a convolutional neural network based on the plurality of screenshot samples to obtain an abnormal behavior classifier model includes: Receiving positive samples and negative samples manually annotated for the plurality of screenshot samples, wherein the positive samples are samples with abnormal behaviors, and the negative samples are samples without abnormal behaviors; Set the learning rate, stochastic gradient descent SGD algorithm, and training times threshold; Using the positive sample and the negative sample, training the ResNet18 framework based on the learning rate and the SGD algorithm; When the number of training times reaches the training times threshold, it is determined that the training is finished, and the model at the end is determined as the abnormal behavior classifier model.
5. The abnormal behavior identification method according to claim 1, characterized in that: The identifying the image to be detected according to the pedestrian target detector and the abnormal behavior classifier model to obtain the abnormal behavior probability includes: Using the pedestrian target detector to detect the image to be detected, and obtaining a target coordinate frame of each pedestrian in the image to be detected; Clustering and merging the target coordinate frames to obtain a coordinate frame to be detected; Intercepting a screenshot to be detected corresponding to the coordinate frame to be detected from the image to be detected; Preprocessing the screenshot to be detected to obtain a preprocessed image; The abnormal behavior classifier model is used to identify the preprocessed image to obtain the abnormal behavior probability.
6. An abnormal behavior recognition device, characterized in that: The abnormal behavior identification device comprises: An acquisition module, used to acquire an image data set; A detection module, used to perform pedestrian detection on each image in the image data set based on a pre-trained pedestrian target detector to obtain a pedestrian coordinate frame, wherein the training method of the pedestrian target detector includes: obtaining a training set and a validation set; setting an initial learning rate and an adjusted learning rate, wherein the initial learning rate is greater than the adjusted learning rate; based on the initial learning rate and the Adam optimization algorithm, using the training set to train the YOLOv3 framework to obtain an intermediate model; inputting the validation set into the intermediate model, and when the loss function of the validation set reaches convergence, training the intermediate model based on the adjusted learning rate and the stochastic gradient descent SGD algorithm; when the loss function of the validation set reaches convergence, determining that the model at the time of convergence is a pedestrian target detector; A merging module, used for clustering the pedestrian coordinate frames according to a set inter-class distance, and merging the clustered pedestrian coordinate frames to obtain a new coordinate frame, wherein the merging of the clustered pedestrian coordinate frames to obtain a new coordinate frame includes: merging the clustered pedestrian coordinate frames to obtain a first coordinate frame; judging whether the width of the first coordinate frame is less than a first threshold, and judging whether the height of the first coordinate frame is less than a second threshold; if the width of the first coordinate frame is less than the first threshold, and the height of the first coordinate frame is less than the second threshold, obtaining the center point of the first coordinate frame, and obtaining a preset width and preset height; judging whether a second coordinate frame based on the center point, the preset width and the preset height exceeds the boundary coordinates of the image; if the second coordinate frame based on the center point, the preset width and the preset height does not exceed the boundary coordinates of the image, determining that the second coordinate frame is a new coordinate frame; A capture module, used to capture the image according to the new coordinate frame to obtain a plurality of screenshot samples; A training module, used to train a convolutional neural network based on the multiple screenshot samples to obtain an abnormal behavior classifier model; A recognition module, used to recognize the image to be detected according to the pedestrian target detector and the abnormal behavior classifier model, and obtain the abnormal behavior probability; A determination module is used to determine an abnormal behavior recognition result according to the abnormal behavior probability.
7. An electronic device, characterized in that: The electronic device includes a processor and a memory, and the processor is used to execute a computer program stored in the memory to implement the abnormal behavior identification method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by the processor, the abnormal behavior identification method according to any one of claims 1 to 5 is implemented.
