Abnormal condition detection method and device, equipment and storage medium
By using the YOLO model in the business hall to detect abnormal situations in the video data, the problem of low human supervision efficiency in the existing technology is solved, real-time abnormal detection and service quality improvement are achieved.
Patent Information
- Application Number
- CN202411603385.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2025-03-04
AI Technical Summary
The existing technology relies on human supervision and adjustment, is inefficient and subjective, making it difficult to effectively improve service quality and detect abnormal situations in the business hall in real time.
By obtaining the video data to be detected and inputting it into the behavior detection model, using the YOLO model for training, detecting the position of people and the confidence of behavior in the video data, and determining the abnormal situation based on the preset threshold.
It realizes real-time detection of abnormal situations in the business hall, improves service quality, reduces the burden of manual monitoring, and improves operational efficiency.
Smart Images
Figure CN119559694A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for detecting abnormal situations. Background Art
[0002] With the digital and intelligent transformation of business, enterprises are not only facing the continuous growth of business volume, but also need to improve operational efficiency while ensuring customer experience. Especially in the service process of some outlets, problems such as long queues and congestion not only affect customer satisfaction, but may also lead to a decline in the operating efficiency of the outlets and even safety hazards;
[0003] In existing technologies, real-time supervision and adjustments are often carried out by security personnel or other service personnel at outlets to enhance customer satisfaction and improve operational efficiency.
[0004] However, existing technologies that rely on human supervision and adjustment may be inefficient and overly subjective, thus affecting service quality. Summary of the invention
[0005] The embodiments of the present application provide a method, device, equipment and storage medium for detecting abnormal situations, so as to achieve the effect of detecting abnormal situations in a business hall in real time and improving service quality.
[0006] In a first aspect, an embodiment of the present application provides a method for detecting an abnormal situation, comprising:
[0007] Obtain the video data to be detected;
[0008] Inputting the video data to be detected into a behavior detection model to obtain a target detection result corresponding to the video data to be detected, wherein the behavior detection model is obtained by training a YOLO model based on a labeling result corresponding to at least one video data, and the labeling result is used to indicate the position of a person in the video data and the confidence corresponding to the person's behavior;
[0009] According to the target detection result, the target abnormality in the to-be-detected video data is determined.
[0010] In a possible implementation, the target detection result includes: the location of the target person, and the target confidence corresponding to the behavior of the target person;
[0011] Correspondingly, determining the target abnormality in the to-be-detected video data according to the target detection result includes:
[0012] Determining at least one piece of behavior information of the target person and a confidence level corresponding to the at least one piece of behavior information according to the position of the target person and the target confidence level;
[0013] For each behavior information, if the confidence corresponding to the behavior information is greater than a preset first confidence threshold and the behavior information is greater than a preset behavior abnormality threshold, it is determined that the target abnormality in the video data to be detected is the behavior abnormality corresponding to the behavior information.
[0014] In a possible implementation, the behavior information includes any of the following: gathering behavior, queuing behavior, running behavior;
[0015] Correspondingly, the abnormal behavior threshold includes any one of the following: an abnormal personnel density threshold corresponding to gathering behavior, an abnormal queue length threshold corresponding to queuing behavior, and an abnormal speed threshold corresponding to running behavior.
[0016] In a possible implementation manner, inputting the to-be-detected video data into a behavior detection model to obtain a target detection result corresponding to the to-be-detected video data includes:
[0017] For each frame of the image to be detected in the video data to be detected, the image to be detected is divided into grids to obtain at least one prediction frame;
[0018] For each prediction frame, determining whether a first person exists in the prediction frame;
[0019] If a first person exists in the prediction frame, determining the confidence level corresponding to the position of the first person and the first and second person's behaviors;
[0020] Wherein, the target personnel include: each first person.
[0021] In a possible implementation, the method further includes:
[0022] The prediction box corresponding to the confidence of the first person's behavior that is less than a preset second confidence threshold is cleared.
[0023] In a possible implementation, the method further includes:
[0024] Obtaining a first labeling result based on labeling of the to-be-detected video data by a user;
[0025] The behavior detection model is updated according to the video data to be detected and the first annotation result.
[0026] In a possible implementation, the method further includes:
[0027] Determining a target processing strategy corresponding to the target abnormal situation in a preset mapping relationship, wherein the mapping relationship records a corresponding relationship between at least one abnormal situation and at least one processing strategy;
[0028] The target processing strategy is executed.
[0029] In a second aspect, an embodiment of the present application provides a device for detecting an abnormal situation, comprising:
[0030] An acquisition module, used for acquiring video data to be detected;
[0031] A processing module, used for inputting the video data to be detected into a behavior detection model to obtain a target detection result corresponding to the video data to be detected, wherein the behavior detection model is obtained by training a YOLO model based on a labeling result corresponding to at least one video data, and the labeling result is used to indicate the position of a person in the video data and the confidence corresponding to the behavior of the person;
[0032] A determination module is used to determine target abnormality in the video data to be detected according to the target detection result.
[0033] In a possible implementation, the target detection result includes: the location of the target person, and the target confidence corresponding to the behavior of the target person;
[0034] Accordingly, the determining module is specifically used for:
[0035] Determining at least one piece of behavior information of the target person and a confidence level corresponding to the at least one piece of behavior information according to the position of the target person and the target confidence level;
[0036] For each behavior information, if the confidence corresponding to the behavior information is greater than a preset first confidence threshold and the behavior information is greater than a preset behavior abnormality threshold, it is determined that the target abnormality in the video data to be detected is the behavior abnormality corresponding to the behavior information.
[0037] In a possible implementation, the behavior information includes any of the following: gathering behavior, queuing behavior, running behavior;
[0038] Correspondingly, the abnormal behavior threshold includes any one of the following: an abnormal personnel density threshold corresponding to gathering behavior, an abnormal queue length threshold corresponding to queuing behavior, and an abnormal speed threshold corresponding to running behavior.
[0039] In a possible implementation manner, the processing module is specifically configured to:
[0040] For each frame of the image to be detected in the video data to be detected, the image to be detected is divided into grids to obtain at least one prediction frame;
[0041] For each prediction frame, determining whether a first person exists in the prediction frame;
[0042] If a first person exists in the prediction frame, determining the confidence level corresponding to the position of the first person and the first and second person's behaviors;
[0043] Wherein, the target personnel include: each first person.
[0044] In a possible implementation manner, the processing module is further configured to:
[0045] The prediction box corresponding to the confidence of the first person's behavior that is less than a preset second confidence threshold is cleared.
[0046] In a possible implementation manner, the processing module is further configured to:
[0047] Obtaining a first labeling result based on labeling of the to-be-detected video data by a user;
[0048] The behavior detection model is updated according to the video data to be detected and the first annotation result.
[0049] In a possible implementation manner, the processing module is further configured to:
[0050] Determining a target processing strategy corresponding to the target abnormal situation in a preset mapping relationship, wherein the mapping relationship records a corresponding relationship between at least one abnormal situation and at least one processing strategy;
[0051] The target processing strategy is executed.
[0052] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;
[0053] The memory stores computer-executable instructions;
[0054] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.
[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.
[0056] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.
[0057] The abnormal situation detection method, device, equipment and storage medium provided in the embodiments of the present application obtain the video data to be detected and input the video data to be detected into the behavior detection model to obtain the target detection result corresponding to the video data to be detected. The behavior detection model is obtained by training the YOLO model based on the annotation result corresponding to at least one video data. The annotation result is used to indicate the position of the person in the video data and the confidence corresponding to the person's behavior. Then, according to the target detection result, the means of determining the target abnormal situation in the video data to be detected is used to achieve the effect of real-time detection of abnormal situations in the business hall and improving service quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0059] Figure 1 Schematic diagram of the process of detecting abnormal situations provided in this application Figure 1 ;
[0060] Figure 2 Schematic diagram of the process of detecting abnormal situations provided in this application Figure 2 ;
[0061] Figure 3 Schematic diagram of the process of detecting abnormal situations provided in this application Figure 3 ;
[0062] Figure 4 A schematic diagram of the structure of the abnormal situation detection device provided in this application;
[0063] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.
[0064] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0066] First, the terms involved in this application are explained:
[0067] YOLO (You Only Look Once) is a real-time target detection algorithm. Its main idea is to transform the target detection problem into a regression problem and predict the location and category of the target in a single forward pass. The principle of YOLO can be summarized as follows:
[0068] Grid division: YOLO divides the input image into a fixed number of grids, each of which is responsible for detecting whether there is a target object in it. The number of targets and categories predicted by each grid is fixed.
[0069] Predicted boxes and categories: For each grid, YOLO predicts one or more bounding boxes (English: boundingbox), each of which contains the location information of the target (center coordinates, width and height of the box) and the confidence of the target. In addition, each box also predicts a category probability distribution, indicating the probability that the object in the box belongs to different categories.
[0070] Box adjustment: The position of each predicted box is normalized relative to the grid coordinates. To get the actual position relative to the entire image, the algorithm applies the grid dimension information for scaling.
[0071] Confidence threshold and non-maximum suppression: According to the confidence of the predicted box, the target box with a higher prediction probability can be screened out. Then, in order to reduce repeated detection, the non-maximum suppression (NMS) algorithm is used to remove highly overlapping boxes and only retain the most likely target box.
[0072] Secondly, the technical background involved in this application is described:
[0073] With the digital and intelligent transformation of business, enterprises are not only facing the continuous growth of business volume, but also need to improve operational efficiency while ensuring customer experience. Especially in the service process of some outlets, problems such as long queues and congestion not only affect customer satisfaction, but may also lead to a decline in the operating efficiency of the outlets and even safety hazards;
[0074] In existing technologies, real-time supervision and adjustments are often carried out by security personnel or other service personnel at outlets to enhance customer satisfaction and improve operational efficiency.
[0075] However, relying on human supervision and adjustment may lead to low efficiency and excessive subjectivity, thus affecting service quality.
[0076] In addition, in some implementations, a particle filter model can also be used to detect abnormal behavior, but it has the following disadvantages:
[0077] (1) High computational complexity: When processing a large number of particles, the particle filter model needs to perform multiple state updates and weight calculations, resulting in high computational complexity, which may not be suitable for real-time applications.
[0078] (2) Particle degeneration problem: During the iteration process, the weights of some particles may tend to zero, causing particle degeneration and reducing the estimation accuracy.
[0079] (3) High-dimensional state space problem: When the state space of the target is high-dimensional, the efficiency and accuracy of the particle filter model may be affected because a large number of particles are required to cover the entire state space.
[0080] (4) Difficulty in parameter selection: There are some parameters in the particle filter model, such as the number of particles and the state transfer equation, which need to be determined by experience or trial and error. Improper selection may affect the performance of the model.
[0081] (5) Sensitive to initial distribution: The selection of initial particle distribution has a great impact on the performance of particle filtering. Inappropriate initial distribution may cause the algorithm to be unable to effectively capture the state of the target.
[0082] The abnormal situation detection method provided by the present application obtains the video data to be detected; inputs the video data to be detected into the behavior detection model to obtain the target detection result corresponding to the video data to be detected, the behavior detection model is obtained by training the YOLO model based on the annotation result corresponding to at least one video data, and the annotation result is used to indicate the position of the person in the video data and the confidence corresponding to the person's behavior; according to the target detection result, the technical means of determining the target abnormal situation in the video data to be detected. The method is used to achieve the effect of detecting abnormal situations in the business hall in real time and improving the service quality.
[0083] It should be noted that the abnormal situation detection method and abnormal situation detection device of the present application can be used in the field of artificial intelligence, and can also be used in any field except the field of artificial intelligence. The application field of the abnormal situation detection method and abnormal situation detection device of the present application is not limited.
[0084] Figure 1 Schematic diagram of the process of detecting abnormal situations provided in this application Figure 1 ,like Figure 1 As shown, the method includes:
[0085] Step 11: Obtain the video data to be detected;
[0086] In this step, the video to be processed may be video data acquired in real time by camera units in many public places, streets, shopping malls, and the like.
[0087] Furthermore, the electronic device can obtain the video data to be detected from the camera unit in real time by configuring the streaming media protocol, for example, the monitoring video obtained in real time by the business hall.
[0088] Optionally, after obtaining the video data to be detected, the video data to be detected may be preprocessed to increase the accuracy of subsequent processing, such as:
[0089] Decompose the video data to be detected into continuous image frames, such as extracting 30 frames per second; remove the noise in the video frames through filters (such as Gaussian filtering, median filtering, etc.); enhance the contrast and brightness of the video, or use edge detection and other technologies to improve the target recognition rate in subsequent behavior detection.
[0090] Step 12: Input the video data to be detected into the behavior detection model to obtain the target detection result corresponding to the video data to be detected;
[0091] The behavior detection model is obtained by training the YOLO model based on the annotation results corresponding to at least one video data, and the annotation results are used to indicate the position of the person in the video data and the confidence level corresponding to the person's behavior;
[0092] In this step, after the video data to be detected is obtained as mentioned above, the video data to be detected is input into a pre-trained behavior detection model, and the behavior detection model analyzes and processes the video data to be detected, and outputs the labeling result corresponding to the video data to be detected, which is recorded as the target detection result.
[0093] Correspondingly, the target detection result may be: the location of the target person, and the target confidence corresponding to the behavior of the target person.
[0094] Optionally, for the behavior detection model, in order to increase the accuracy of the behavior detection model, real-time updates can also be performed, that is, the following implementations can be achieved:
[0095] Step 1: Obtain a first annotation result based on the annotation of the video data to be detected by the user;
[0096] In this implementation, for the video data to be detected, the user can obtain it and then manually annotate it to obtain an annotated result, which is recorded as a first annotated result, and the electronic device correspondingly obtains the first annotated result.
[0097] Step 2: Update the behavior detection model based on the video data to be detected and the first annotation result.
[0098] In this implementation, after obtaining the video data to be detected and the first annotation result, the video data to be detected and the first annotation result can be used as a training set for training the behavior detection model, thereby updating the parameters of the behavior detection model to achieve the purpose of updating the behavior detection model.
[0099] In other implementations, new data may be collected regularly to optimize the model, and the optimization includes but is not limited to: adjusting the parameters of the model and the definition of the following abnormal behaviors to maintain the accuracy and practicality of the detection.
[0100] Optionally, before step 12, a process of training the behavior detection model may also be performed:
[0101] Step 1: Obtain the annotation result corresponding to at least one video data;
[0102] In this implementation, historical video data in the business hall (eg, images or videos captured by a camera unit) may be collected, the video data may be annotated, and position frames corresponding to the annotated persons and their behaviors may be obtained.
[0103] Step 2: Preprocess the labeled data;
[0104] In this implementation, preprocessing can be processing to increase the robustness of labeled data, such as adjusting the image size, normalizing pixel values, and data enhancement (random cropping, rotation, brightness adjustment, etc.).
[0105] Step 3: Use the above data to train the pre-selected YOLO model;
[0106] In this implementation, the preprocessed data is input into the YOLO model, and multiple rounds of iterative training are performed to adjust the parameters of the model so that it can accurately detect the behavior of people.
[0107] Step 4: Define abnormal behavior;
[0108] In this implementation, abnormal behaviors may be: personnel density exceeding a threshold, queue length exceeding a limit, running behavior, etc.; specifically, adjustments may be made based on the actual situation and security requirements of the business hall.
[0109] Optionally, in the embodiment of the present application, the behavior detection model obtained by training with the YOLO model may have the following advantages:
[0110] Speed and real-time performance: The YOLO model performs target detection in a single forward propagation, so it has very high speed and real-time performance. Compared with the above-mentioned particle filter model, the YOLO model does not require complex iterative calculations when processing a large number of targets, and is suitable for applications that require fast response;
[0111] (2) Global information capture: The YOLO model can obtain the global information of the image in a single forward propagation, while the particle filter model usually requires multiple iterations to fuse local information. This makes YOLO better at object detection in multi-scale and complex scenes;
[0112] (3) Multi-target detection: The YOLO model can detect multiple targets in an image at the same time, while the particle filter model is usually used for state estimation and tracking and is not suitable for the detection task of multiple human targets in a business hall.
[0113] (4) End-to-end training: The YOLO model can be trained end-to-end, from the original image to the final target detection result, which simplifies the model design and training process.
[0114] (5) Performance of deep learning: In related fields such as target detection, YOLO usually has excellent performance on large-scale datasets, while the particle filter model is more suitable for state estimation problems, especially in high-dimensional, nonlinear or non-Gaussian distribution situations.
[0115] Step 13: According to the target detection result, determine the target abnormality in the video data to be detected.
[0116] In this step, the target detection result shows the location of the target person and the target confidence corresponding to the target person's behavior, which can determine the abnormal situations that may appear in the video data to be detected, and take the abnormal situations with higher confidence as the target abnormal situations.
[0117] Furthermore, after step 13, the following steps may be performed:
[0118] Step 1: Determine the target processing strategy corresponding to the target abnormal situation in the pre-set mapping relationship;
[0119] The mapping relationship records a correspondence between at least one abnormal situation and at least one processing strategy;
[0120] In this implementation, in order to ensure that timely response can be taken to the detected target abnormal situation, a correspondence between at least one abnormal situation and at least one processing strategy is pre-set. At this time, based on the target abnormal situation, a consistent abnormal situation can be determined in the mapping relationship, and the corresponding processing strategy can be determined as the target processing strategy.
[0121] For example, an abnormal situation may be that there are too many people gathered, and the handling strategy may be: inform the staff to go and handle it; an abnormal situation may be that the queue is too long, and the handling strategy may be: add windows for processing business, that is, notify the staff to go to the idle windows to handle the business, and notify other staff to coordinate personnel and divert the queued people.
[0122] Step 2: Execute the target processing strategy.
[0123] In this implementation, after the target processing strategy is determined, the corresponding target processing strategy is executed. As mentioned above, when it is determined that the target processing strategy is to increase the window for processing business, the staff can be notified to go to the idle window to handle the business, and other staff can be notified to coordinate personnel and divert the queued personnel.
[0124] The embodiment of the present application provides a method for detecting abnormal situations, which can be used in the field of artificial intelligence. The method includes: obtaining video data to be detected; inputting the video data to be detected into a behavior detection model to obtain a target detection result corresponding to the video data to be detected, the behavior detection model is obtained by training a YOLO model based on a labeling result corresponding to at least one video data, and the labeling result is used to indicate the position of a person in the video data and the confidence level corresponding to the person's behavior; according to the target detection result, determining the target abnormal situation in the video data to be detected. The method is used to achieve real-time detection of abnormal situations in the business hall and improve service quality.
[0125] Specifically, the solution of the embodiment of the present application can achieve the following technical effects:
[0126] 1. Real-time response: Detection of abnormal situations can monitor activities in the hall in real time. Once an abnormal situation is detected, the system can immediately trigger an alarm or notify relevant personnel so that quick action can be taken to reduce losses and risks.
[0127] 2. Improve service quality: By monitoring the people and behaviors in the hall, we can better analyze customer flow, queuing conditions, etc., so as to optimize service windows, provide better services, and improve customer satisfaction.
[0128] Efficiency optimization: Abnormal situation detection can help managers better understand business conditions, such as window efficiency, employee workload, etc., which helps to reasonably allocate resources and improve work efficiency.
[0129] 3. Prediction and analysis: By collecting and analyzing abnormal situation data, different patterns and trends can be discovered to better predict potential problems and changes and help make more informed decisions.
[0130] 4. Reduce manual monitoring: Abnormal situation detection can automatically monitor and alarm, reducing the burden of manual monitoring, reducing monitoring costs, and improving monitoring efficiency
[0131] Based on the above embodiment, the target detection result includes: the location of the target person, and the target confidence corresponding to the behavior of the target person.
[0132] Figure 2 Schematic diagram of the process of detecting abnormal situations provided in this application Figure 2 ,like Figure 2 As shown, the above step 13 may include the following steps:
[0133] Step 21: determining at least one piece of behavior information of the target person and a confidence level corresponding to the at least one piece of behavior information according to the position of the target person and the target confidence level;
[0134] In this step, the number of target persons can be one or more. Based on the position of each target person and the target confidence corresponding to the behavior of the target person, the behavior of the target person can be classified into at least one behavior information and the confidence corresponding to each behavior information.
[0135] Optionally, the behavior information includes any of the following: gathering behavior, queuing behavior, running behavior;
[0136] For example, within the preset range of position A, the confidence level of the target person's queuing behavior is 80%, the confidence level of the gathering behavior is 10%, and the confidence level of other behaviors is 10%.
[0137] Step 22: for each piece of behavior information, if the confidence corresponding to the behavior information is greater than a preset first confidence threshold and the behavior information is greater than a preset behavior abnormality threshold, determine that the target abnormality in the video data to be detected is the behavior abnormality corresponding to the behavior information.
[0138] In this step, when it is determined that the confidence corresponding to the behavior information is greater than a preset first confidence threshold, such as the first confidence threshold is 70%, the behavior information is considered to be the corresponding behavior in the current video data to be detected.
[0139] Furthermore, based on the behavior information, for example, if the behavior information is gathering behavior, the density of people corresponding to the gathering behavior can be determined; if the behavior information is queuing behavior, the length of the queue corresponding to the queuing behavior can be determined; if the behavior information is running behavior, the speed of people corresponding to the running behavior can be determined (which can be determined based on the video frames in the video data to be detected).
[0140] Optionally, the behavior abnormality threshold includes any one of the following: a personnel density abnormality threshold corresponding to gathering behavior, a queue length abnormality threshold corresponding to queuing behavior, and a speed abnormality threshold corresponding to running behavior.
[0141] In a possible implementation, when it is determined that the queue length corresponding to the queuing behavior is 6 and the queue length abnormal threshold is 5, it is determined that the target abnormal situation in the video data to be detected is the behavior abnormality corresponding to the queuing behavior.
[0142] The embodiment of the present application provides a method for detecting abnormal situations, including: determining at least one behavior information of the target person and the confidence level corresponding to the at least one behavior information according to the position of the target person and the target confidence level, and for each behavior information, if the confidence level corresponding to the behavior information is greater than a preset first confidence threshold, and the behavior information is greater than a preset behavior abnormality threshold, determining that the target abnormal situation in the video data to be detected is a behavior abnormality corresponding to the behavior information. This technical solution realizes the judgment of behavior abnormality.
[0143] Based on the above embodiments, Figure 3 Schematic diagram of the process of detecting abnormal situations provided in this application Figure 3 ,like Figure 3 As shown, the above step 12 may include the following steps:
[0144] Step 31: for each frame of the image to be detected in the video data to be detected, the image to be detected is divided into grids to obtain at least one prediction box;
[0145] In this step, the video data to be detected is input into the behavior detection model. There is the following processing flow: first, each frame of the image to be detected is obtained, and for each frame of the image to be detected, grid division processing is performed to obtain at least one prediction box.
[0146] Step 32: for each prediction box, determine whether the first person exists in the prediction box;
[0147] In this step, it is determined whether there is a person, that is, the first person, in each prediction frame, and the prediction frames in which the first person does not exist are removed.
[0148] Step 33: If the first person exists in the prediction frame, determine the position of the first person and the confidence level corresponding to the behavior of the first person.
[0149] Among them, the target personnel include: each first person.
[0150] Optionally, the prediction box corresponding to the confidence of the first person's behavior that is less than a preset second confidence threshold is cleared.
[0151] In this implementation, prediction boxes with higher prediction probabilities can be screened out based on their confidence levels. Then, in order to reduce duplicate detections, the non-maximum suppression (NMS) algorithm is used to remove highly overlapping boxes, leaving only the most likely prediction boxes.
[0152] The embodiment of the present application provides a method for detecting abnormal situations, including: for each frame of the image to be detected in the video data to be detected, the image to be detected is divided into a grid to obtain at least one prediction frame, and for each prediction frame, it is determined whether there is a first person in the prediction frame, and if there is a first person in the prediction frame, the position of the first person and the confidence corresponding to the behavior of the first person are determined, wherein the target person includes: each first person. In this technical solution, the position, behavior and confidence of the person in the video data to be detected are obtained, which provides a basis for the subsequent judgment of abnormal behavior.
[0153] Based on the above embodiments, the following are device embodiments involved in the embodiments of the present application:
[0154] Figure 4 A schematic diagram of the structure of the abnormal situation detection device provided in this application, such as Figure 4 As shown, the device comprises:
[0155] An acquisition module 41 is used to acquire video data to be detected;
[0156] A processing module 42 is used to input the video data to be detected into a behavior detection model to obtain a target detection result corresponding to the video data to be detected, wherein the behavior detection model is obtained by training a YOLO model based on a labeling result corresponding to at least one video data, and the labeling result is used to indicate the position of a person in the video data and the confidence corresponding to the person's behavior;
[0157] The determination module 43 is used to determine the target abnormality in the video data to be detected according to the target detection result.
[0158] In a possible implementation, the target detection result includes: the location of the target person, and the target confidence corresponding to the behavior of the target person;
[0159] Accordingly, the determination module 43 is specifically used for:
[0160] Determining at least one piece of behavior information of the target person and a confidence level corresponding to the at least one piece of behavior information according to the position of the target person and the target confidence level;
[0161] For each piece of behavior information, if the confidence corresponding to the behavior information is greater than a preset first confidence threshold and the behavior information is greater than a preset behavior abnormality threshold, it is determined that the target abnormality in the video data to be detected is the behavior abnormality corresponding to the behavior information.
[0162] In a possible implementation, the behavior information includes any of the following: gathering behavior, queuing behavior, running behavior;
[0163] Correspondingly, the behavior abnormality threshold includes any of the following: a personnel density abnormality threshold corresponding to gathering behavior, a queue length abnormality threshold corresponding to queuing behavior, and a speed abnormality threshold corresponding to running behavior.
[0164] In a possible implementation, the processing module 42 is specifically configured to:
[0165] For each frame of the image to be detected in the video data to be detected, the image to be detected is divided into grids to obtain at least one prediction box;
[0166] For each prediction box, determining whether the first person exists in the prediction box;
[0167] If the first person exists in the prediction frame, determine the position of the first person, the first person, and the confidence level corresponding to the behavior of the first person;
[0168] Among them, the target personnel include: each first person.
[0169] In a possible implementation, the processing module 42 is further configured to:
[0170] The prediction box corresponding to the confidence of the first person's behavior that is less than a preset second confidence threshold is cleared.
[0171] In a possible implementation, the processing module 42 is further configured to:
[0172] Obtaining a first labeling result based on labeling of the video data to be detected by the user;
[0173] The behavior detection model is updated according to the video data to be detected and the first annotation result.
[0174] In a possible implementation, the processing module 42 is further configured to:
[0175] Determining a target processing strategy corresponding to a target abnormal situation in a preset mapping relationship, wherein the mapping relationship records a corresponding relationship between at least one abnormal situation and at least one processing strategy;
[0176] Execute the target processing strategy.
[0177] The abnormal situation detection device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.
[0178] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device provided in this embodiment includes: at least one processor 51 and a memory 52. Optionally, the device also includes a communication component 53. The processor 51, the memory 52 and the communication component 53 are connected via a bus 54.
[0179] In a specific implementation process, at least one processor 51 executes the computer-executable instructions stored in the memory 52, so that at least one processor 51 executes the above method.
[0180] The specific implementation process of the processor 51 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0181] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0182] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.
[0183] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0184] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0185] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0186] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0187] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0188] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0189] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0190] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0191] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.
[0192] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.
[0193] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application, are not limited to the precise structures described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for detecting abnormal situations, characterized in that: The method comprises: Obtain the video data to be detected; Inputting the video data to be detected into a behavior detection model to obtain a target detection result corresponding to the video data to be detected, wherein the behavior detection model is obtained by training a YOLO model based on a labeling result corresponding to at least one video data, and the labeling result is used to indicate the position of a person in the video data and the confidence corresponding to the person's behavior; According to the target detection result, the target abnormality in the to-be-detected video data is determined.
2. The method according to claim 1, characterized in that The target detection result includes: the location of the target person, and the target confidence corresponding to the behavior of the target person; Correspondingly, determining the target abnormality in the to-be-detected video data according to the target detection result includes: Determining at least one piece of behavior information of the target person and a confidence level corresponding to the at least one piece of behavior information according to the position of the target person and the target confidence level; For each behavior information, if the confidence corresponding to the behavior information is greater than a preset first confidence threshold and the behavior information is greater than a preset behavior abnormality threshold, it is determined that the target abnormality in the video data to be detected is the behavior abnormality corresponding to the behavior information.
3. The method according to claim 2, characterized in that The behavior information includes any of the following: gathering behavior, queuing behavior, running behavior; Correspondingly, the abnormal behavior threshold includes any one of the following: an abnormal personnel density threshold corresponding to gathering behavior, an abnormal queue length threshold corresponding to queuing behavior, and an abnormal speed threshold corresponding to running behavior.
4. The method according to any one of claims 1 to 3, characterized in that: The step of inputting the video data to be detected into a behavior detection model to obtain a target detection result corresponding to the video data to be detected includes: For each frame of the image to be detected in the video data to be detected, the image to be detected is divided into grids to obtain at least one prediction frame; For each prediction frame, determining whether a first person exists in the prediction frame; If there is a first person in the prediction frame, determine the confidence level corresponding to the position of the first person and the first and second person's behaviors; Wherein, the target personnel include: each first person.
5. The method according to claim 4, characterized in that The method further comprises: The prediction box corresponding to the confidence of the first person's behavior that is less than a preset second confidence threshold is cleared.
6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Obtaining a first labeling result based on labeling of the to-be-detected video data by a user; The behavior detection model is updated according to the video data to be detected and the first annotation result.
7. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determining a target processing strategy corresponding to the target abnormal situation in a preset mapping relationship, wherein the mapping relationship records a corresponding relationship between at least one abnormal situation and at least one processing strategy; The target processing strategy is executed.
8. A device for detecting abnormal conditions, characterized in that: The device comprises: An acquisition module, used to acquire video data to be detected; A processing module, used for inputting the video data to be detected into a behavior detection model to obtain a target detection result corresponding to the video data to be detected, wherein the behavior detection model is obtained by training a YOLO model based on a labeling result corresponding to at least one video data, and the labeling result is used to indicate the position of a person in the video data and the confidence corresponding to the behavior of the person; A determination module is used to determine target abnormality in the video data to be detected according to the target detection result.
9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.