Abnormal event processing method and device, storage medium and calculation program product

Abstract

The invention discloses an abnormal event processing method and device, a storage medium and a computing program product. The method comprises the following steps: acquiring a first video frame sequence corresponding to a first event occurring in a first time period in a target area; identifying each first video frame in the first video frame sequence by using an image identification model to obtain first action generation information corresponding to each first video frame, and analyzing the first action generation information corresponding to each first video frame by using an event classification model to obtain a first event type of the first event; determining a first event influence index and a first event influence development index of the first event based on the first event type, and further determining a risk level corresponding to the first event; and processing the first event according to a processing strategy corresponding to the risk level. The technical problem that abnormal event processing is not timely due to the fact that related personnel are commanded and dispatched to process the abnormal event only according to the event type in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of cloud camera technology, and more specifically, to an abnormal event processing method and device, a storage medium, and a computer program product. Background Art

[0002] With the rapid development of computers, networks, and image processing and transmission technologies, video surveillance technology is moving towards high definition, digitization, intelligence, high speed, and networking. At the same time, more and more video storage technologies have emerged. Among them, video cloud technology, as an information collection and recognition technology, can automatically select effective images for video output, and analyze them according to the monitored actions, realizing unmanned operation and supporting automatic production and transmission of on-site signals.

[0003] However, when using video cloud technology to handle abnormal events, security personnel are generally dispatched to handle the event based on the type of event analyzed. However, this command and dispatch method does not take into account the scope of the event, and the number of dispatch personnel is randomly assigned and not targeted, making it impossible to handle abnormal events in a timely manner. As a result, abnormal events may escalate and seriously affect the public environment.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for handling abnormal events, a storage medium, and a computer program product, so as to at least solve the technical problem that the related art only directs and dispatches relevant personnel to handle abnormal events according to the event type, resulting in untimely handling of abnormal events.

[0006] According to one aspect of an embodiment of the present application, a method for handling abnormal events is provided, including: obtaining a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes multiple first video frames; using a pre-trained image recognition model to identify each first video frame in the first video frame sequence to obtain first occurrence action information corresponding to each first video frame, and using a pre-trained event classification model to analyze the first occurrence action information corresponding to each first video frame to obtain a first event type of the first event; determining a first event impact index and a first event impact development index of the first event based on the first event type, and determining a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within the first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area in the future time period; processing the first event according to a processing strategy corresponding to the risk level.

[0007] Optionally, obtaining a first video frame sequence corresponding to a first event occurring in a target area within a first time period includes: obtaining a video stream corresponding to a first event occurring in a target area within a first time period captured by a capture device; preprocessing the video stream, wherein the preprocessing includes at least one of the following: decoding, resolution adjustment, and frame rate adjustment; and parsing the preprocessed video stream to obtain a first video frame sequence.

[0008] Optionally, the training process of the image recognition model includes: determining multiple groups of first sample data consisting of multiple second video frames and second action information corresponding to each second video frame, wherein the second video frame is a video frame sequence within a video frame sequence obtained by parsing a video stream corresponding to a second event occurring in any area within a second time period acquired by an acquisition device, and the second time period is a historical time period before the first time period; determining a first neural network model; iteratively training the first neural network model using multiple groups of first sample data to obtain an image recognition model, including: for each training batch in the iterative training process, inputting the second video frames in each first sample data of the training batch into the first neural network model to obtain second predicted action information corresponding to each second video frame output by the first neural network model; constructing a first target loss function using the second predicted action information and the second action information corresponding to each second video frame, and adjusting the model parameters of the first neural network model based on the first target loss function.

[0009] Optionally, the training process of the event classification model includes: determining multiple groups of second sample data consisting of action sequences corresponding to multiple second video frame sequences and second event types of second events corresponding to each second video frame sequence, wherein the second video frame sequence is obtained by parsing a video stream corresponding to a second event occurring in any area within a second time period acquired by an acquisition device, and the action sequence corresponding to the second video frame sequence includes second occurring action information corresponding to each second video frame in the second video frame, and the second time period is a historical time period before the first time period; determining a second neural network model; iteratively training the second neural network model using multiple groups of second sample data to obtain an event classification model, including: for each training batch in the iterative training process, inputting the action sequence corresponding to the second video frame sequence in each second sample data of the training batch into the second neural network model to obtain a predicted event type of the second event corresponding to each second video frame sequence output by the second neural network model; constructing a second target loss function using the predicted event type and the second event type of the second event corresponding to each second video frame sequence, and adjusting the model parameters of the second neural network model based on the second target loss function.

[0010] Optionally, determining a first event impact index and a first event impact development index of a first event based on a first event type includes: determining from a preset video database at least one third target video frame sequence whose similarity to the first video frame sequence is not less than a preset similarity threshold and an event impact index and an event impact development index of a third target event corresponding to each third target video frame sequence, wherein the video database includes third video frame sequences corresponding to multiple third events of the same type as the first event acquired by multiple acquisition devices in a third time period and an event impact index and an event impact development index of each third event, and the third time period is a historical time period before the first time period and closest to the first time period; and using a weighted average algorithm to process the event impact index and the event impact development index of each third target event, respectively, to obtain a first event impact index and a first event impact development index.

[0011] Optionally, determining the risk level corresponding to the first event based on the first event impact index and the first event impact development index includes: determining a first weight value of the first event impact index and a second weight value of the first event impact development index; calculating the sum of the product of the first event impact index and the corresponding first weight value and the product of the first event impact development index and the corresponding second weight value, and using the resulting sum as the risk index corresponding to the first event; when the risk index is within a preset first range, determining that the risk level corresponding to the first event is a high risk level; when the risk index is within a preset second range, determining that the risk level corresponding to the first event is a medium risk level; when the risk index is within a preset third range, determining that the risk level corresponding to the first event is a low risk level.

[0012] Optionally, the first event is processed according to a processing strategy corresponding to the risk level, including: when the risk level is a high risk level, a prompt message is issued, and staff in the target area and adjacent areas are dispatched to jointly process the first event, wherein the prompt message is used to prompt other objects to avoid the target area; when the risk level is a medium risk level, a prompt message is issued, and staff in the target area are dispatched to process the first event; when the risk level is a low risk level, a prompt message is issued, and some staff in the target area are dispatched to process the first event.

[0013] According to another aspect of an embodiment of the present application, there is also provided an abnormal event processing device, including: an acquisition module, used to acquire a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes multiple first video frames; an event type determination module, used to identify each first video frame in the first video frame sequence using a pre-trained image recognition model, obtain first occurrence action information corresponding to each first video frame, and use a pre-trained event classification model to analyze the first occurrence action information corresponding to each first video frame to obtain a first event type of the first event; a risk determination module, used to determine a first event impact index and a first event impact development index of the first event based on the first event type, and determine the risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within the first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area in the future time period; a processing module, used to process the first event according to a processing strategy corresponding to the risk level.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is further provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned abnormal event processing method by running the computer program.

[0015] According to another aspect of an embodiment of the present application, a computer program product is further provided. The computer program product includes a stored computer program, wherein the computer program implements the above-mentioned abnormal event processing method when executed by a processor.

[0016] In the embodiment of the present application, firstly, a first video frame sequence corresponding to a first event occurring in a target area within a first time period is obtained; then, each first video frame in the first video frame sequence is identified using an image recognition model to obtain first action information corresponding to each first video frame (i.e., the user's body movements in the first video frame), and the first action information corresponding to each first video frame is analyzed using an event classification model to obtain a first event type of the first event; then, a first event impact index (immediate impact) and a first event impact development index (long-term impact) of the first event are determined based on the first event type, and a risk level corresponding to the first event is comprehensively assessed based on the first event impact index (immediate impact) and the first event impact development index (long-term impact), so that the risk level can accurately reflect the urgency and potential threats of the event; finally, a processing strategy corresponding to the risk level is determined, and the processing strategy is executed to promptly process the first event, thereby effectively reducing losses or avoiding the first event from developing into a more serious situation in advance. Thus, the technical problem that the related art only directs and dispatches relevant personnel to handle abnormal events according to the event type, resulting in untimely processing of abnormal events is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an abnormal event processing method according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of an optional abnormal event processing method according to an embodiment of the present application;

[0020] Figure 3 is a schematic structural diagram of an optional abnormal event processing device according to an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] In addition, the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0025] Example 1

[0026] According to an embodiment of the present application, a method embodiment of a method for handling an abnormal event is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing the abnormal event processing method is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0028] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the abnormal event handling method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the abnormal event handling method of the above-mentioned application is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0032] In the above operating environment, Figure 2 is a flow chart of an optional abnormal event processing method according to an embodiment of the present application, such as Figure 2 As shown, the method at least includes steps S202-S208, wherein:

[0033] Step S202: acquiring a first video frame sequence corresponding to a first event occurring in a target area within a first time period.

[0034] The first video frame sequence includes a plurality of first video frames.

[0035] Step S204, using a pre-trained image recognition model to identify each first video frame in the first video frame sequence, to obtain first action information corresponding to each first video frame, and using a pre-trained event classification model to analyze the first action information corresponding to each first video frame, to obtain a first event type of the first event.

[0036] In the technical solution provided in step S204, each first video frame in the first video frame sequence is traversed, and the image recognition model is used to analyze the currently traversed first video frame to determine the first action information corresponding to the current first video frame, wherein the first action information refers to the body movements of each user in the picture corresponding to the first video frame, such as raising hands, bending over, raising head, taking a stool, leaning back against a pillar, etc. Thus, the first action information corresponding to each first video frame in the first video frame sequence obtained through the above steps is spliced ​​to obtain an action sequence obtained by identifying the first video frame sequence.

[0037] Furthermore, the action sequence obtained by identifying the first video frame sequence is analyzed using an event classification model to identify a pattern of continuous actions in the first video frame sequence and obtain a first event type of the first event.

[0038] Step S206: determining a first event impact index and a first event impact development index of the first event based on the first event type, and determining a risk level corresponding to the first event based on the first event impact index and the first event impact development index.

[0039] The first event impact index is used to reflect the severity of the impact of the first event on the target area (other targets within the area) in the first time period, while the first event impact development index is used to reflect the severity of the impact of the first event on the target area (other targets within the area) in the future time period. Both of the above parameters quantify the degree of impact.

[0040] In the technical solution provided in step S206, the risk level of the first event is comprehensively assessed by comprehensively considering the severity of the impact of the first event on the target area at present (immediate impact) and the severity of the impact that the first event may have on the target area in the future (long-term impact), and the risk level can accurately distinguish the urgency and potential threats of different events.

[0041] Step S208: Process the first event according to a processing strategy corresponding to the risk level.

[0042] In the technical solution provided in step S208, a corresponding processing strategy is determined based on the risk level obtained in the above step S206, and the first event is processed according to the processing strategy, thereby effectively reducing losses or preventing the first event from developing into a more serious situation in advance.

[0043] The above method of this embodiment is further introduced below.

[0044] As an optional implementation, in the technical solution provided in the above step S202, the method may include:

[0045] Step S2021, obtaining a video stream corresponding to a first event occurring in a target area within a first time period captured by a capture device.

[0046] The above-mentioned acquisition device can be a single AI cloud camera or a group of AI cloud cameras under the same network address. The target area is the acquisition area of ​​the acquisition device.

[0047] Step S2022, pre-processing the video stream.

[0048] Among them, the above preprocessing includes but is not limited to: decoding processing, resolution adjustment, frame rate adjustment, etc. This is because, considering that the video stream is usually compressed and encoded, for example, the video captured by the acquisition device is encoded using encoding standards such as H.264, H.265 or MPEG to obtain a video stream. Therefore, before converting the video stream into a video frame sequence, the decoder can be used to parse the bit stream in the video stream to restore the pixel information of each video frame. During the decoding process, in order to avoid data packet loss or network delay in the video stream, the decoded video data can be temporarily stored in a buffer to ensure the continuity and smoothness of the video frame. In addition, in order to make the decoded video frame adapt to the current processing device, the resolution and frame rate of the video frame can be adjusted, such as converting from high-definition video to standard definition, or if the original frame rate is higher than the display frame rate, a frame drop operation is performed, otherwise if the original frame rate is lower than the display frame rate, a frame interpolation operation is performed.

[0049] Step S2023, parsing the preprocessed video stream to obtain a first video frame sequence.

[0050] Specifically, the preprocessed video stream can be parsed using the OpenCV library in Python to convert the video stream into a first video frame sequence, and the first video frame sequence includes multiple first video frames, and each video frame can be saved in the form of a separate image file.

[0051] As an optional implementation, in the technical solution provided in the above step S204, the training process of the image recognition model includes:

[0052] Step S11 , determining a plurality of groups of first sample data consisting of a plurality of second video frames and second action information corresponding to each second video frame.

[0053] Among them, the second video frame is a video frame sequence in a video frame sequence obtained by parsing a video stream corresponding to a second event occurring in any area (including the target area or other areas except the target area) collected by the acquisition device in a second time period (i.e., a historical time period before the first time period), and the second action information corresponding to the second video frame is obtained by manual analysis or model analysis. That is, the first sample data can be a data set stored in a video cloud storage system, which includes each second video frame in a video frame sequence corresponding to a historical video stream and the second action information obtained by manual identification, or a test sample of a model.

[0054] In addition, considering the diversity of video surveillance scenarios, the above-mentioned first sample data can also adopt currently known data sets, such as the Penn Action data set (which contains approximately 23,000 images and corresponding human action labels), the Kinetics data set (which contains a large number of video clips and corresponding action category labels, which can be used to train video action recognition models), the NTURGB+D data set (which contains RGB and depth image data captured by multiple sensors, which can be used to train human action recognition models), etc.

[0055] Step S12, determining a first neural network model.

[0056] Step S13, iteratively training the first neural network model using multiple sets of first sample data to obtain an image recognition model, including:

[0057] For each training batch in the iterative training process, the second video frames in each first sample data of the training batch are input into the first neural network model to obtain the second predicted action information corresponding to each second video frame output by the first neural network model; the first target loss function is constructed using the second predicted action information and the second action information corresponding to each second video frame, and the model parameters of the first neural network model are adjusted according to the first target loss function. The model parameters include at least one of the following: learning rate and number of iterations.

[0058] It should be noted that before iteratively training the first neural network model with multiple groups of first sample data, a preprocessing operation may be performed on the second video frame in each group of first sample data, wherein the preprocessing operation includes at least one of the following: cropping, scaling, and rotation, so as to ensure the accuracy of model training.

[0059] As an optional implementation, in the technical solution provided in the above step S204, the training process of the above event classification model includes:

[0060] Step S21 : determining a plurality of groups of second sample data consisting of action sequences corresponding to a plurality of second video frame sequences and second event types of second events corresponding to respective second video frame sequences.

[0061] Among them, the above-mentioned second video frame sequence is obtained by parsing the video stream corresponding to the second event occurring in any area (including the target area or other areas except the target area) within the second time period (i.e., a historical time period before the first time period) captured by the acquisition device, and the action sequence corresponding to the second video frame sequence includes the second occurring action information corresponding to each second video frame in the second video frame, which can be obtained through model analysis or by manually parsing the second video frames.

[0062] Step S22, determining a second neural network model. The second neural network model may be a recurrent neural network (RNN), a long short-term memory network (LSTM), a gated recurrent unit (GRU), or a Transformer architecture, etc., and the present application does not impose any specific restrictions on the selection of the second neural network model.

[0063] Step S23, iteratively training the second neural network model using multiple sets of second sample data to obtain an event classification model, including:

[0064] For each training batch in the iterative training process, the action sequence corresponding to the second video frame sequence in each second sample data of the training batch is input into the second neural network model to obtain the predicted event type of the second event corresponding to each second video frame sequence output by the second neural network model; the predicted event type of the second event corresponding to each second video frame sequence and the second event type are used to construct a second target loss function, and the model parameters of the second neural network model are adjusted according to the second target loss function. The model parameters include at least one of the following: learning rate, number of iterations.

[0065] In addition, in step S204, the image recognition model and the event classification model may be loaded into the memory, for example, the raw data of the image recognition model and the event classification model may be loaded from the non-volatile memory into the volatile memory, so that the processor runs the image recognition model and the event classification model. The raw data of the image recognition model and the event classification model refers to unprocessed data, and generally includes parameters and structural data of the image recognition model and the event classification model. The structural data may be a calculation relationship based on the parameters, such as a forward propagation calculation relationship between intermediate layers and neurons. Specifically, the structural data may include codes related to the structure of the image recognition model and the event classification model, such as codes for performing related calculations between intermediate layers and neurons.

[0066] In one embodiment, an area for loading the image recognition model and the event classification model can be divided in the memory, which can include a structure data storage area and a parameter storage area. The structure data storage area is used to store structure-related codes, and the parameters referenced by it can point to the address of specific parameters in the parameter storage area through a pointer. During the training process of the image recognition model and the event classification model, it may be necessary to frequently update the parameters, and the parameter values ​​in the parameter storage area can be updated.

[0067] As an optional implementation, in the technical solution provided in the above step S206, the method may include:

[0068] Step S2061, determining from a preset video database at least one third target video frame sequence whose similarity with the first video frame sequence is not less than a preset similarity threshold and an event impact index and an event impact development index of a third target event corresponding to each third target video frame sequence.

[0069] Among them, the above-mentioned video database includes third video frame sequences corresponding to multiple third events of the same type as the first event collected by multiple acquisition devices in a third time period, as well as an event impact index and an event impact development index of each third event, and the third time period is a historical time period before the first time period and closest to the first time period.

[0070] Step S2062: Use a weighted average algorithm to process the event impact index and the event impact development index of each third target event respectively to obtain a first event impact index and a first event impact development index.

[0071] That is to say, when calculating the first event impact index and the first event impact development index of the first event, one or more third events that are closest to the occurrence of the first event and have the same event type and the third video frame sequences corresponding to each third event can be obtained first; then, the Spearman correlation coefficient is used to calculate the similarity between the first video frame sequence and each third video frame sequence, so as to screen out the third target video frame that is most similar to the first video frame sequence from the multiple third video frame sequences; finally, the weighted average algorithm is used to process the event impact index and the event impact development index of the third target event corresponding to the screened third target video frame, so as to obtain the first event impact index and the first event impact development index.

[0072] Therefore, the above first event impact index can be expressed by the following formula:

[0073]

[0074] Wherein, i represents the sequence number of the third target event, n represents the total number of the third target events, and η represents the severity value of the impact of the third target event on the collection area where it is located within the third time period.

[0075] The above first event impact development index can be expressed by the following formula:

[0076]

[0077] Where i represents the sequence number of the third target event, n represents the total number of the third target events, and η represents the severity value of the impact of the third target event on the collection area where it is located in the future time period after the third time period.

[0078] Furthermore, based on the first event impact index and the first event impact development index, the risk level corresponding to the first event is determined according to the following method:

[0079] First, determining a first weight value of the first event impact index and a second weight value of the first event impact development index;

[0080] Then, the sum of the product of the first event impact index and the corresponding first weight value and the product of the first event impact development index and the corresponding second weight value is calculated, and the obtained sum is used as the risk index corresponding to the first event;

[0081] When the risk index is within a preset first range, determining that the risk level corresponding to the first event is a high risk level;

[0082] When the risk index is within the preset second range, determining that the risk level corresponding to the first event is a medium risk level;

[0083] When the risk index is within the preset third range, it is determined that the risk level corresponding to the first event is a low risk level.

[0084] It should be noted that the above-mentioned classification rules are only used as examples in the embodiments of the present application. In actual application, multiple risk levels can be divided according to a more detailed range to improve the accuracy of determining the risk level of an event.

[0085] As an optional implementation, in the technical solution provided in the above step S208, the method may include:

[0086] When the risk level is high, it means that the occurrence of the first event will have a greater impact on the target area in the current or future events. At this time, in order to minimize the impact, a prompt message can be issued first to prompt other objects to avoid the target area; then the staff in the target area and adjacent areas are dispatched to jointly handle the first event.

[0087] When the risk level is medium, it means that the occurrence of the first event will have a certain impact on the target area in the current or future period of time. At this time, in order to minimize the impact, a prompt message can also be issued first to prompt other objects to avoid the target area; then, considering that the impact of the first event is at a medium level, only the staff in the target area are dispatched to deal with the first event, without wasting too much dispatch resources in other adjacent areas, so that the order in the target area and other adjacent areas can be coordinated in a timely manner.

[0088] When the risk level is low, only a prompt message is issued to prompt other objects to avoid the target area. If necessary, some staff in the target area can be dispatched to handle the first event.

[0089] Based on the scheme defined in the above steps S202 to S208, it can be known that, in an embodiment, a video stream corresponding to a first event occurring in a target area within a first time period is collected by an acquisition device (such as a cloud AI camera), and converted into a first video frame sequence comprising a plurality of first video frames; then, each first video frame in the first video frame sequence is identified by using an image recognition model to obtain first action information corresponding to each first video frame (i.e., the user's body movements in the first video frame), and the first action information corresponding to each first video frame is analyzed by using an event classification model to obtain a first event type of the first event; then, a first event impact index (immediate impact) and a first event impact development index (long-term impact) of the first event are determined based on the first event type, and a risk level corresponding to the first event is comprehensively assessed based on the first event impact index (immediate impact) and the first event impact development index (long-term impact), so that the risk level can accurately reflect the urgency and potential threats of the event; finally, a processing strategy corresponding to the risk level is determined, and the processing strategy is executed to promptly process the first event, thereby effectively reducing losses or preventing the first event from developing into a more serious situation in advance.

[0090] Example 2

[0091] Based on Embodiment 1 of the present application, an embodiment of an abnormal event processing device is also provided, and the abnormal event processing method of the above embodiment is executed when the device is running. Figure 3 is a schematic diagram of the structure of an optional abnormal event processing device according to an embodiment of the present application, such as Figure 3 As shown, the abnormal event processing device at least includes an acquisition module 32, an event type determination module 34, a risk determination module 36 and a processing module 38, wherein:

[0092] An acquisition module 32 is used to acquire a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes a plurality of first video frames;

[0093] An event type determination module 34 is used to identify each first video frame in the first video frame sequence by using a pre-trained image recognition model to obtain first action information corresponding to each first video frame, and to analyze the first action information corresponding to each first video frame by using a pre-trained event classification model to obtain a first event type of the first event;

[0094] A risk determination module 36, configured to determine a first event impact index and a first event impact development index of the first event based on the first event type, and to determine a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within the first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area within the future time period;

[0095] The processing module 38 is used to process the first event according to a processing strategy corresponding to the risk level.

[0096] It should be noted that the various modules in the above-mentioned abnormal event handling device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0097] Example 3

[0098] According to an embodiment of the present application, a non-volatile storage medium is further provided, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the abnormal event handling method in Example 1.

[0099] Optionally, the device where the non-volatile storage medium is located implements the following steps by running the program: obtaining a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes multiple first video frames; using a pre-trained image recognition model to identify each first video frame in the first video frame sequence to obtain first occurrence action information corresponding to each first video frame, and using a pre-trained event classification model to analyze the first occurrence action information corresponding to each first video frame to obtain a first event type of the first event; determining a first event impact index and a first event impact development index of the first event based on the first event type, and determining a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within the first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area in a future time period; processing the first event according to a processing strategy corresponding to the risk level.

[0100] According to an embodiment of the present application, a computer program product is also provided, which includes a stored computer program, wherein when the computer program is executed by a processor, the abnormal event handling method in Example 1 is implemented.

[0101] Optionally, the computer program executes the following steps: obtaining a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes a plurality of first video frames; using a pre-trained image recognition model to identify each first video frame in the first video frame sequence to obtain first occurrence action information corresponding to each first video frame, and using a pre-trained event classification model to analyze the first occurrence action information corresponding to each first video frame to obtain a first event type of the first event; determining a first event impact index and a first event impact development index of the first event based on the first event type, and determining a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within the first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area in a future time period; processing the first event according to a processing strategy corresponding to the risk level.

[0102] According to an embodiment of the present application, a processor is also provided, which is used to run a program, wherein the abnormal event handling method in Example 1 is executed when the program is running.

[0103] Optionally, when the program is running, the following steps are executed: obtaining a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes a plurality of first video frames; using a pre-trained image recognition model to identify each first video frame in the first video frame sequence to obtain first occurrence action information corresponding to each first video frame, and using a pre-trained event classification model to analyze the first occurrence action information corresponding to each first video frame to obtain a first event type of the first event; determining a first event impact index and a first event impact development index of the first event based on the first event type, and determining a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within the first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area in a future time period; processing the first event according to a processing strategy corresponding to the risk level.

[0104] According to an embodiment of the present application, an electronic device is also provided, wherein: Figure 4 is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application, such as Figure 4As shown, the electronic device includes one or more processors; a memory for storing one or more programs, which, when the one or more programs are executed by the one or more processors, enables the one or more processors to run the programs, wherein the programs are configured to execute the abnormal event handling method in the above-mentioned embodiment 1 when running.

[0105] Optionally, the processor is configured to execute the following steps through a computer program: obtaining a first video frame sequence corresponding to a first event occurring in the target area within a first time period, wherein the first video frame sequence includes multiple first video frames; using a pre-trained image recognition model to identify each first video frame in the first video frame sequence to obtain first occurrence action information corresponding to each first video frame, and using a pre-trained event classification model to analyze the first occurrence action information corresponding to each first video frame to obtain a first event type of the first event; determining a first event impact index and a first event impact development index of the first event based on the first event type, and determining a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within the first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area in the future time period; processing the first event according to a processing strategy corresponding to the risk level.

[0106] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0107] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0109] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0112] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for handling abnormal events, characterized in that: include: Acquire a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes a plurality of first video frames; Using a pre-trained image recognition model to identify each of the first video frames in the first video frame sequence, to obtain first action information corresponding to each of the first video frames, and using a pre-trained event classification model to analyze the first action information corresponding to each of the first video frames, to obtain a first event type of the first event; Determine a first event impact index and a first event impact development index of the first event based on the first event type, and determine a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is used to reflect the severity of the impact of the first event on the target area within a first time period, and the first event impact development index is used to reflect the severity of the impact of the first event on the target area in a future time period; The first event is processed according to a processing strategy corresponding to the risk level.

2. The method according to claim 1, characterized in that Acquiring a first video frame sequence corresponding to a first event occurring in a target area within a first time period, including: Acquire a video stream corresponding to a first event occurring in a target area within a first time period captured by a capture device; Preprocessing the video stream, wherein the preprocessing includes at least one of the following: decoding processing, resolution adjustment, and frame rate adjustment; The preprocessed video stream is parsed to obtain the first video frame sequence.

3. The method according to claim 1, characterized in that The training process of the image recognition model includes: Determine a plurality of groups of first sample data consisting of a plurality of second video frames and second action information corresponding to each of the second video frames, wherein the second video frame is a video frame sequence in a video frame sequence obtained by parsing a video stream corresponding to a second event occurring in any area within a second time period acquired by a capture device, and the second time period is a historical time period before the first time period; determining a first neural network model; Iteratively training the first neural network model using the multiple groups of first sample data to obtain the image recognition model includes: For each training batch in the iterative training process, the second video frames in each first sample data of the training batch are input into the first neural network model to obtain the second predicted action information corresponding to each second video frame output by the first neural network model; the first target loss function is constructed using the second predicted action information and the second action information corresponding to each second video frame, and the model parameters of the first neural network model are adjusted according to the first target loss function.

4. The method according to claim 1, characterized in that: The training process of the event classification model includes: Determine a plurality of groups of second sample data consisting of action sequences corresponding to a plurality of second video frame sequences and second event types corresponding to each of the second video frame sequences, wherein the second video frame sequence is obtained by parsing a video stream corresponding to a second event occurring in any area within a second time period acquired by an acquisition device, and the action sequence corresponding to the second video frame sequence includes second action information corresponding to each of the second video frames in the second video frame, and the second time period is a historical time period before the first time period; determining a second neural network model; Iteratively training the second neural network model using the multiple groups of second sample data to obtain the event classification model includes: For each training batch in the iterative training process, the action sequence corresponding to the second video frame sequence in each second sample data of the training batch is input into the second neural network model to obtain the predicted event type of the second event corresponding to each second video frame sequence output by the second neural network model; the second target loss function is constructed using the predicted event type of the second event corresponding to each second video frame sequence and the second event type, and the model parameters of the second neural network model are adjusted according to the second target loss function.

5. The method according to claim 1, characterized in that Determining a first event impact index and a first event impact development index of the first event based on the first event type includes: Determine from a preset video database at least one third target video frame sequence whose similarity to the first video frame sequence is not less than a preset similarity threshold, and an event impact index and an event impact development index of a third target event corresponding to each of the third target video frame sequences, wherein the video database includes third video frame sequences corresponding to a plurality of third events of the same type as the first event collected by a plurality of collection devices in a third time period, and an event impact index and an event impact development index of each of the third events, and the third time period is a historical time period before the first time period and closest to the first time period; The event impact index and the event impact development index of each of the third target events are processed respectively by using a weighted average algorithm to obtain the first event impact index and the first event impact development index.

6. The method according to claim 1, characterized in that Determining a risk level corresponding to the first event based on the first event impact index and the first event impact development index includes: Determine a first weight value of the first event impact index and a second weight value of the first event impact development index; Calculating the sum of the product of the first event impact index and the corresponding first weight value and the product of the first event impact development index and the corresponding second weight value, and using the obtained sum as the risk index corresponding to the first event; When the risk index is within a preset first range, determining that the risk level corresponding to the first event is a high risk level; When the risk index is within a preset second range, determining that the risk level corresponding to the first event is a medium risk level; When the risk index is within a preset third range, it is determined that the risk level corresponding to the first event is a low risk level.

7. The method according to claim 6, characterized in that Processing the first event according to a processing strategy corresponding to the risk level includes: When the risk level is the high risk level, a prompt message is issued, and staff in the target area and adjacent areas thereof are dispatched to collaboratively handle the first event, wherein the prompt message is used to prompt other objects to avoid the target area; When the risk level is the medium risk level, issuing the prompt information, and dispatching staff in the target area to handle the first event; When the risk level is the low risk level, the prompt information is issued, and some staff in the target area are dispatched to handle the first event.

8. An abnormal event processing device, characterized in that: include: An acquisition module, configured to acquire a first video frame sequence corresponding to a first event occurring in a target area within a first time period, wherein the first video frame sequence includes a plurality of first video frames; an event type determination module, configured to identify each of the first video frames in the first video frame sequence using a pre-trained image recognition model to obtain first action information corresponding to each of the first video frames, and to analyze the first action information corresponding to each of the first video frames using a pre-trained event classification model to obtain a first event type of the first event; a risk determination module, configured to determine a first event impact index and a first event impact development index of the first event based on the first event type, and to determine a risk level corresponding to the first event based on the first event impact index and the first event impact development index, wherein the first event impact index is configured to reflect the severity of the impact of the first event on the target area within a first time period, and the first event impact development index is configured to reflect the severity of the impact of the first event on the target area within a future time period; A processing module is used to process the first event according to a processing strategy corresponding to the risk level.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, wherein the device where the non-volatile storage medium is located executes the abnormal event processing method described in any one of claims 1 to 7 by running the computer program.

10. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the abnormal event processing method described in any one of claims 1 to 7 is implemented.

Images