Abnormal event detection method, electronic equipment and computer readable storage medium

Abstract

The invention discloses an abnormal event detection method, electronic equipment and a computer readable storage medium, and the method comprises the steps: receiving a detection triggering instruction, and transmitting the detection triggering instruction when a collection device detects that there is a picture change in a collected current video stream; acquiring a current video stream from target acquisition equipment according to the detection trigger instruction; and performing abnormal event detection processing on the current video stream to obtain an event detection result. Therefore, the analysis cost can be reduced, and the real-time performance of abnormal event detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an abnormal event detection method, an electronic device and a computer-readable storage medium. Background Art

[0002] With the popularity of video IoT technology, users can view cameras and other devices in real time through cloud services. However, due to limited personal time, it is impossible for users to watch video streams 24 hours a day to detect any abnormalities immediately. Therefore, an intelligent solution is needed to assist this process, where the system detects and sends notifications to users immediately when abnormal events are detected.

[0003] Currently, the detection of abnormal events is mainly divided into cloud-based intelligent analysis based on continuous video streaming or device cloud recording. Among them, continuous video streaming requires real-time streaming analysis, which requires high bandwidth costs and continuous cloud-based intelligent analysis costs; device cloud recording cannot guarantee real-time performance. Summary of the invention

[0004] The main technical problem solved by the present application is to provide an abnormal event detection method, an electronic device and a computer-readable storage medium, which can reduce the cost of abnormal event detection.

[0005] To solve the above technical problems, a technical solution adopted in the present application is: providing an abnormal event detection method, the abnormal event detection method comprising: receiving a detection trigger instruction, the detection trigger instruction is sent when an acquisition device detects that there is a picture change in the current video stream being acquired; obtaining the current video stream from the target acquisition device according to the detection trigger instruction; performing abnormal event detection processing on the current video stream to obtain an event detection result.

[0006] In some embodiments, the method is applied to an abnormal event detection system, which includes at least two event detection nodes; the step of obtaining the current video stream from a target acquisition device according to the detection trigger instruction includes: determining a target event detection node from the at least two event detection nodes according to device information in the detection trigger instruction; and obtaining the current video stream from the target acquisition device through the target event detection node with the detection trigger instruction.

[0007] In some embodiments, each event detection node includes a stream pulling module and an analysis module; the step of obtaining the current video stream from the target acquisition device through the target event detection node using the detection trigger instruction includes: pulling target video information from the target acquisition device through the stream pulling module, the target video information being obtained after compressing the current video stream based on the target acquisition device; analyzing and processing the target video information through the analysis module to obtain the current video stream.

[0008] In some embodiments, the step of pulling target video information from the target acquisition device through the stream pulling module includes: the stream pulling module determines the target acquisition device from the acquisition device according to the device information in the detection trigger instruction; and pulls the target video information from the target acquisition device.

[0009] In some embodiments, the step of analyzing and processing the target video information through the analysis module to obtain the current video stream includes: obtaining the video compression format and video resolution in the target video information through the analysis module; querying and decoding the target handle information of the target video information from the historical handle information according to the video compression format and the video resolution; decoding and processing the target video information according to the target handle information to obtain the current video stream.

[0010] In some embodiments, the event detection result characterizes whether there is an abnormal event in the current video stream, and the step of performing abnormal event detection processing on the current video stream to obtain the abnormal event detection result includes: extracting a video frame to be detected from the current video stream according to a preset frame extraction rule; performing feature extraction processing on the video frame to be detected to obtain target object features in the video frame to be detected; in response to the match between the target object feature and the preset object feature being greater than a match threshold, determining that there is an abnormal event in the video frame to be detected.

[0011] In some embodiments, after the step of determining that an abnormal event exists in the video frame to be detected in response to the degree of match between the target object feature and the preset object feature being greater than a matching threshold, the method further includes: selecting a preset number of target video frames from the video frames to be detected in which the same abnormal event exists; and synthesizing the target video frames according to the acquisition time points of the target video frames to obtain dynamic images of the target video frames.

[0012] In some embodiments, the method is applied to an abnormal event detection system, in which a target inspection time point for each acquisition device is set; the method also includes: in response to the current time point reaching the target inspection time point, detecting whether the corresponding acquisition device has a collection video stream at the current time point; if so, obtaining the collection video stream shot at the current time point from the corresponding acquisition device; performing quality detection on the collection video stream to obtain a quality detection result, and the quality detection result characterizes whether the corresponding acquisition device has a device abnormality result.

[0013] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, including a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above abnormal event detection method.

[0014] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium including program data stored therein, wherein the program data is used to implement the above abnormal event detection method when executed by a processor.

[0015] Compared with the above scheme, the traditional abnormal event detection based on continuous video stream has too high bandwidth cost and analysis cost requirements, which cannot be widely used, and the abnormal event detection based on cloud recording cannot guarantee real-time performance. The present application provides an abnormal event detection method, which includes receiving a detection trigger instruction, which is sent when the acquisition device detects that there is a picture change in the current video stream being acquired; obtaining the current video stream from the target acquisition device according to the detection trigger instruction; performing abnormal event detection processing on the current video stream to obtain an event detection result. Therefore, whether the picture changes is used as a pre-detection, and after receiving the detection trigger instruction sent when the picture changes, the corresponding current video stream is obtained for abnormal event detection processing, so as to achieve accurate streaming, which can not only reduce the analysis cost, but also ensure the real-time performance of abnormal event detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0017] Figure 1 It is a flowchart of an exemplary embodiment of an abnormal event detection method shown in the present application;

[0018] Figure 2 yes Figure 1A flow chart of an exemplary embodiment of step S120 in the abnormal event detection method is shown;

[0019] Figure 3 yes Figure 1 A flow chart of an exemplary embodiment of step S130 in the abnormal event detection method is shown;

[0020] Figure 4 is a flowchart of an exemplary embodiment of an abnormal event detection process shown in the present application;

[0021] Figure 5 yes Figure 1 A flowchart of another exemplary embodiment of an abnormal event detection method is shown;

[0022] Figure 6 It is a flowchart of an exemplary embodiment of the equipment inspection process shown in the present application;

[0023] Figure 7 It is a specific flow chart of an exemplary embodiment of the abnormal event detection method shown in the present application;

[0024] Figure 8 is a structural schematic diagram of an exemplary embodiment of an abnormal event detection device shown in the present application;

[0025] Fig. 9 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application;

[0026] Fig.10 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be appreciated that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the art without making creative work are within the scope of protection of the present application.

[0028] First of all, it should be noted that through the Internet of Things technology, users can view the video stream collected by the camera in real time on the cloud. At the same time, in order to ensure the real-time nature of abnormal events, it is necessary to detect the video images and analyze whether there are set danger sources, such as fire, crossing the boundary, falling, etc., and send timely notifications to users after abnormal events are detected.

[0029] Based on this, the present application proposes an abnormal event detection method, an electronic device and a computer-readable storage medium. Figure 1 , Figure 1 It is a flowchart of an exemplary embodiment of the abnormal event detection method shown in the present application.

[0030] The execution subject of the abnormal event detection method can be a terminal device or a server or other processing device, wherein the terminal device can be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The execution subject of the abnormal event detection method can also be an abnormal event detection system. In some possible implementations, the abnormal event detection method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0031] Specifically, the abnormal event detection method of this embodiment includes the following steps:

[0032] S110: Receive a detection trigger instruction, where the detection trigger instruction is sent when the acquisition device detects that there is a picture change in the current video stream being acquired.

[0033] First of all, it should be explained that the abnormal event detection system includes collection equipment, abnormal event detection devices, equipment inspection devices, and clients.

[0034] The detection trigger instruction is used to instruct the abnormal event detection device to perform the task of abnormal event detection of the current video stream. Exemplarily, when the acquisition device detects that there is a picture change in the current video stream, the detection trigger instruction is sent to the abnormal event detection device. In some embodiments, the detection of picture changes can be determined based on the motion information between each video frame and the reference video frame in the video stream acquired by the acquisition device. The motion information can be determined by motion detection auxiliary detection, such as real-time calculation of the motion vector between the image block of the current video frame in the video stream and the corresponding image block in the reference video frame. If the motion vector is greater than the vector threshold, it means that there is a picture change in the corresponding video frame, where the frame sequence of the reference video frame in the video stream is earlier than the current video frame. In other embodiments, it can also be determined by frame difference whether a picture change occurs. Specifically, two adjacent video frames are converted into grayscale images, the pixel values ​​of adjacent video frames are subtracted, and then the average value between the obtained differences is calculated. If the average value is greater than the threshold, it is considered that there is an object in motion in the picture; otherwise, it is considered that there is no object in motion in the picture. The purpose of the threshold is to avoid the reduction of the recognition effect due to slight changes in the picture.

[0035] The acquisition device can be used to acquire video streams. Exemplarily, the acquisition device can be an image acquisition device, a video acquisition device, etc.

[0036] The current video stream can be a video stream collected in real time by a collection device. First, the collection device performs a motion detection on the current video stream collected in real time to determine whether there is a picture change in the current video stream; then, after receiving the detection trigger instruction, the abnormal event detection device obtains the current video stream from the target collection device corresponding to the detection trigger instruction to perform abnormal event detection processing.

[0037] S120: Acquire the current video stream from the target acquisition device according to the detection trigger instruction.

[0038] The target acquisition device may be an acquisition device among the acquisition devices that send the detection trigger instruction. It should be noted that the abnormal event detection device of this embodiment may include multiple abnormal event detection modules, and can perform abnormal event detection processing on the current video streams acquired by multiple target acquisition devices simultaneously and in parallel.

[0039] After receiving the detection trigger instruction sent by the acquisition device that detects the presence of picture changes in the current video stream, the abnormal event detection device obtains the current video stream from the target acquisition device according to the detection trigger instruction.

[0040] S130: Perform abnormal event detection processing on the current video stream to obtain an event detection result.

[0041] The event detection result may include whether there is an abnormal event in the current video stream, as well as the abnormal event type, abnormal event level, and abnormal event occurrence time, etc. Exemplarily, abnormal event detection processing may be performed on the current video stream to obtain the event detection result.

[0042] It can be seen that the abnormal event detection method of the embodiment of the present application receives a detection trigger instruction, which is sent when the acquisition device detects that there is a picture change in the current video stream being acquired; the current video stream is obtained from the target acquisition device according to the detection trigger instruction; the current video stream is subjected to abnormal event detection processing to obtain an event detection result. In this way, a pre-detection is performed based on whether the picture has changed, and after receiving the detection trigger instruction sent when the picture changes, the corresponding current video stream is obtained for abnormal event detection processing, so as to achieve accurate streaming, which can not only reduce the analysis cost, but also improve the real-time performance of abnormal event detection.

[0043] Based on the above embodiments, the embodiments of the present application adopt Figure 2 The flowchart details how to obtain the current video stream from the target acquisition device. Figure 2 , Figure 2 yes Figure 1The flowchart of an exemplary embodiment of step S120 in the abnormal event detection method is shown. Specifically, the process of obtaining the current video stream from the target acquisition device according to the detection trigger instruction in step S120 specifically includes the following steps:

[0044] S210: Determine a target event detection node from at least two event detection nodes according to the device information in the detection trigger instruction.

[0045] First, it should be noted that the abnormal event detection method of the embodiment of the present application is applied to an abnormal event detection system, which includes a collection device, an abnormal event detection apparatus and a client. The abnormal event detection apparatus includes at least two event detection nodes.

[0046] Device information refers to the device information of the acquisition device. Exemplarily, the device information includes the device number, device IP address, resolution, etc. of the acquisition device. As an example, the abnormal event detection system allocates the event detection node nearby according to the device IP address in the received detection trigger instruction. As another example, the abnormal event detection system allocates the event detection node according to the resolution in the received detection trigger instruction.

[0047] It is understandable that event detection nodes can be distributed in different provinces and cities. In order to improve the analysis speed and ensure real-time performance, event detection nodes can be allocated nearby based on the device IP address. For example, after receiving the detection trigger instruction, it is found that the device IP address is Province A, then the event detection node corresponding to Province A is used as the corresponding target event detection node. Of course, the processing capabilities of each event detection node may also be different. In order to improve system utilization, event detection nodes can be allocated based on resolution. For example, after receiving the detection trigger instruction, high-resolution acquisition devices are allocated to event detection nodes with strong processing capabilities, and low-resolution acquisition devices are allocated to event detection nodes with weaker processing capabilities. Of course, there can be other allocation rules, such as allocation based on the idle rate of event detection nodes, etc., which are not limited here.

[0048] S220: Acquire the current video stream from the target acquisition device by detecting the trigger instruction through the target event detection node.

[0049] After the abnormal event detection system determines the target event detection node from each event detection node according to the device information, the target event detection node obtains the current video stream from the target acquisition device with the detection trigger instruction. In other embodiments, it is not necessary to distinguish the event detection nodes, and a master control system obtains the current video stream from the corresponding target acquisition device according to the device information in the detection trigger instruction.

[0050] In some embodiments, each event detection node includes a stream pulling module and an analysis module. The abnormal event detection system pulls the target video information from the target acquisition device through the stream pulling module, and the target video information is obtained by compressing the current video stream based on the target acquisition device; the target video information is analyzed and processed by the analysis module to obtain the current video stream. In this way, the two tasks of stream pulling and analysis are distributed to the stream pulling module and the analysis module respectively, so that the multiplexing of the stream pulling module can be achieved. In other embodiments, each event detection node includes a stream pulling analysis module, that is, the abnormal event detection system pulls the target video information from the target acquisition device through the stream pulling analysis module, and analyzes and processes the target video information through the stream pulling analysis module to obtain the current video stream.

[0051] The stream pulling module is used to pull the target video information in the acquisition device. After receiving the detection trigger instruction, the abnormal event detection device allocates the corresponding target event detection node to the target acquisition device according to the device information in the detection trigger instruction, and allocates the corresponding abnormal event detection task. After receiving the abnormal event detection task, the stream pulling module sends a video stream pulling request to the corresponding target acquisition device according to the abnormal event detection task. In order to effectively utilize bandwidth and storage space, after receiving the video stream pulling request, the target acquisition device needs to compress the current video stream, obtain the target video information, and send the target video information to the stream pulling module. Exemplarily, the compression formats of different acquisition devices are also different, such as H264 format, H265 format, etc.

[0052] Specifically, the process of pulling the target video information by the stream pulling module may include the following steps: the stream pulling module determines the target acquisition device from the acquisition devices according to the device information in the detection trigger instruction; and pulls the target video information from the target acquisition device. Exemplarily, the abnormal event detection device generates an abnormal event detection task for each event detection node according to the device information in the detection trigger instruction, wherein the abnormal event detection task includes the device information in the detection trigger instruction. The stream pulling module sends a video stream pulling request to the corresponding target acquisition device according to the device information. As an example, the stream pulling module can determine the target acquisition device according to the device number in the device information, and the device number is a unique number of the acquisition device. According to the device number, a unique acquisition device can and can only be determined. After receiving the video stream pulling request sent by the stream pulling module, the target acquisition device compresses the current video stream collected to obtain the target video information, and sends the target video information to the stream pulling module.

[0053] Of course, in other embodiments, the process of the stream pulling module pulling the target video information may also include: the stream pulling module determines whether the device information in the detection trigger instruction meets the stream pulling distance requirement. If it does, the target video information is pulled from the target acquisition device through the stream pulling module; if it does not meet the requirement, the stream pulling module sends a coordination request to other stream pulling modules according to the device information in the detection trigger instruction, and in response to receiving the consent coordination instruction from other stream pulling modules, the abnormal event detection task corresponding to the target acquisition device is transferred to the other stream pulling modules that sent the consent coordination instruction, and the other stream pulling modules pull the target video information from the target acquisition device. Among them, the stream pulling distance requirement can be that the distance between the stream pulling module and the device IP address of the detection trigger instruction exists in the same city or a pre-divided area. In this way, the stream pulling module is allocated nearby, which can accelerate the outflow of the target acquisition device and ensure the real-time detection of abnormal events.

[0054] The analysis module is used to analyze and process the target video information pulled by the stream pulling module to obtain the current video stream. After pulling the target video information, the stream pulling module will upload the target video information to the cloud storage module and push the abnormal event detection task to the analysis module. After receiving the abnormal event detection task pushed by the stream pulling module, the analysis module downloads the corresponding target video information from the cloud storage module according to the abnormal event detection task for analysis and processing to obtain the current video stream; at the same time, the analysis module will also analyze and process the obtained current video stream for abnormal event detection and obtain the event detection result. That is, the analysis module not only includes the analysis function, but also includes the abnormal event detection function, the abnormal event result push function, etc. Of course, after pulling the target video information, the stream pulling module can also send it directly to the analysis module without storing it in the cloud storage module.

[0055] It should be noted that in the abnormal event detection system, the relationship between the streaming module and the analysis module can be one-to-one, that is, one streaming module corresponds to one analysis module. In other embodiments, the relationship between the streaming module and the analysis module can also be many-to-many, that is, the streaming module is only responsible for streaming, and after pulling the target video information, it stores the target video information in the cloud storage module; any idle analysis module downloads the corresponding target video information from the cloud storage module for analysis and processing to obtain the current video stream.

[0056] Among them, the analysis module analyzes and processes the target video information, and the process of obtaining the current video stream may include the following steps: obtaining the video compression format and video resolution in the target video information through the analysis module; querying the target handle information for decoding the target video information from the historical handle information according to the video compression format and video resolution; decoding and processing the target video information according to the target handle information to obtain the current video stream. In this way, the same batch of decoding handles are used for the target video information with the same video compression format and video resolution, and the analysis performance is improved by reusing the decoding handles. Of course, in other embodiments, the decoding handles may not be reused, that is, after the analysis module obtains the target video information, it creates a corresponding decoding handle according to the target video information and decodes it to obtain the current video stream.

[0057] It should be noted that to achieve the reuse of the decoding handle, the video compression format and video resolution in the target video information must be consistent. Hard decoding can be used during decoding to improve the high performance and real-time performance of the detection.

[0058] Among them, the video compression format is a technology used to reduce the file size of the current video stream while maintaining the visual quality as much as possible. Exemplarily, the visual compression format can be H264, H265, VP9, ​​etc. The historical handle information can be the decoding handle information created during the historical decoding process, and after use, the decoding handle information is not released. When the subsequent target video information is the same as the video compression format and video resolution decoded by the historical handle information, the historical handle information is used as the target handle information of the target video information, and the decoding process is continued.

[0059] Exemplarily, the historical handle information includes video compression format and video resolution. After the analysis module obtains the target video information from the cloud storage module, the video compression format and video resolution in the target video information are matched with the video compression format and video resolution in the historical handle information to determine the target handle information of the target video information; then, the target video information is decoded according to the target handle information to obtain the current video stream. If the target handle information is not found in the historical handle information, new handle information is created according to the target video information for decoding. In order to greatly improve the analysis performance, when the target video information is decoded through the target handle information, each pulled target video information can be decoded into YUV data of uniform resolution, for example, all decoded into a video stream with a resolution of 640*360; then an algorithm analysis is performed based on the YUV data to obtain the abnormal event detection result.

[0060] Therefore, by reusing the decoding handles of the same quality stream, the output decoding resolution is unified, and the performance of decoding and intelligent analysis is improved.

[0061] Based on the above embodiments, the embodiments of the present application adopt Figure 3 The flowchart details how to detect abnormal events on the current video stream. Figure 3 , Figure 3 yes Figure 1 The flowchart of an exemplary embodiment of step S130 in the abnormal event detection method is shown. Specifically, step S130 performs abnormal event detection processing on the current video stream, and the process of obtaining the abnormal event detection result specifically includes the following steps:

[0062] S310: Extracting a video frame to be detected from the current video stream according to a preset frame extraction rule.

[0063] The preset frame extraction rule can be determined according to the processing capability of the event detection node. For example, the weaker the processing capability, the more interval frames. As an example, the video frames to be detected can be extracted from the current video stream at intervals of a preset number of frames. The preset number of frames can be 2 frames, 3 frames, 4 frames, etc., which are not specifically limited here, as long as they do not affect the abnormal event detection. Of course, in other embodiments, all video frames in the current video stream can also be used as video frames to be detected for abnormal event detection processing.

[0064] The video frames to be detected refer to the video frames extracted for abnormal event detection. Exemplarily, the abnormal event detection system can extract video frames at intervals from the current video stream as the video frames to be detected.

[0065] S320: Perform feature extraction processing on the video frame to be detected to obtain the target object features in the video frame to be detected.

[0066] The target object feature may be a feature of the target object detected from the video frame to be detected. Exemplarily, the target object feature extraction process may be performed on the video frame to be detected to obtain the target object feature in the video frame to be detected. The feature extraction process may include but is not limited to convolutional neural networks and the like.

[0067] S330: In response to the matching degree between the target object feature and the preset object feature being greater than the matching threshold, determining that an abnormal event exists in the video frame to be detected.

[0068] The preset object feature may be a feature of a pre-set abnormal event. For example, the preset object feature may include but is not limited to fire, falling, crossing the boundary, etc. After the abnormal event detection system extracts the target object feature, it matches the target object feature with the preset object feature. If the matching degree is greater than the matching threshold, it is determined that there is an abnormal event in the video frame to be detected. At the same time, the preset abnormal event corresponding to the matched preset object feature can also be pushed to the client to inform the user.

[0069] After determining that an abnormal event exists, the video frame containing the abnormal event needs to be processed to send the abnormal event to the client. In some embodiments, the abnormal event occurrence message can be pushed to the client by making a dynamic image. Specifically, a preset number of target video frames are selected from the video frames to be detected in which the same abnormal event exists; the target video frames are synthesized according to the acquisition time point of the target video frames to obtain a dynamic image of the target video frame. In this way, the user can promptly discover abnormal events through the received dynamic images. Compared with the traditional method of sending text messages to inform users of abnormal events, this proposal can facilitate user confirmation through dynamic image push. Users or on-duty personnel can quickly browse the abnormal event and make judgments. At the same time, users can also configure phone notifications for specific events, such as falls, and users do not need to worry about the situation within the detection range.

[0070] The current video stream may include one or more abnormal events. The analysis module groups the video frames with the same abnormal event in the current video stream into a group, and selects a preset number of video frames in each group of video frames as target video frames; a dynamic image corresponding to the abnormal event is constructed from early to late according to the acquisition time point of the target video frame. Exemplarily, the preset number may be 24 frames, 48 ​​frames, etc. The first 24 frames or the first 48 frames of each group of video frames may be merged and processed from early to late according to the acquisition time point to obtain a dynamic image corresponding to the abnormal event. Graphics Interchange Format (GIF) is an image format that includes multiple frames of images to create animation effects.

[0071] In other embodiments, the preset abnormal event corresponding to the matched preset object feature can also be pushed to the client in text format to inform the user. In other embodiments, the timestamp of the video frame corresponding to the abnormal event can also be pushed to the client to inform the user, and the user can view the video stream corresponding to the timestamp through the cloud after receiving it.

[0072] The overall process of abnormal event detection is described in detail in conjunction with the above embodiments. Figure 4 :

[0073] When the acquisition device finds that there is a picture change in the current video stream, it reports a detection trigger instruction to the abnormal event detection device; after receiving the detection trigger instruction, the abnormal event detection device allocates the corresponding event detection node according to the device information in the detection trigger instruction; the stream pulling module in the event detection node pulls the target video information from the corresponding target acquisition device; after pulling the target video information, the target video information is stored in the cloud storage module, and the analysis module pushes the abnormal event detection task; after receiving the abnormal event detection task, the analysis module obtains the target video information for analysis and detection, and downloads the corresponding target video information from the cloud storage module; and determines the target handle information from the historical handle information according to the target video information; the target video information is decoded by the target handle information to obtain the current video stream; the analysis module extracts the video frames to be detected from the current video stream according to the preset frame extraction rules for abnormal event detection processing to obtain the event detection result; and selects a preset number of target video frames from the video frames to be detected of the same abnormal event to produce dynamic images, and pushes the produced dynamic images to the client to inform the user.

[0074] Based on the above embodiments, the embodiments of the present application adopt Figure 5 The flowchart details how to detect abnormal events on the current video stream. Figure 5 , Figure 5 yes Figure 1 A flow chart of another exemplary embodiment of an abnormal event detection method is shown. Specifically, the following steps may be included:

[0075] S510: In response to the current time point reaching the target inspection time point, detecting whether the corresponding acquisition device has a captured video stream at the current time point.

[0076] The target inspection time point may be a pre-set time point. Exemplarily, the target inspection time point may be set by the user. When the user does not set it, the system default inspection time may be used as the target inspection time point. As an example, a fixed time point may be set as the target inspection time point, such as 11 a.m. As another example, the inspection interval may be set, such as performing an inspection every two hours, and starting from midnight, 2 a.m., 4 a.m., 6 a.m., etc. may be used as the target inspection time points every day.

[0077] In addition, in addition to setting the target inspection time point, the user can also set the target inspection items. For example, during nighttime inspections, black and white images are not detected (infrared mode is black and white images) to avoid false detections that interfere with user judgment.

[0078] When the current time point reaches the target inspection time point, the abnormal event detection system detects whether the corresponding acquisition device has a collection video stream at the current time point. This process mainly determines whether the acquisition device is in working condition.

[0079] S520: If it exists, obtain the captured video stream shot at the current time point from the corresponding capture device.

[0080] If the abnormal event detection system detects that the corresponding acquisition device has a collection video stream at the current time point, if so, it means that the acquisition device is in working state. Only then will the corresponding acquisition device be tested.

[0081] In some embodiments, the acquisition video stream of the acquisition device in this embodiment can also be obtained by pulling the current video information from the corresponding acquisition device through the stream pulling module; then the business is differentiated through the data protocol, and the current video information pulled by the stream pulling module is sent to the corresponding module, such as the analysis module and the equipment inspection module. Therefore, the stream pulling module belongs to module reuse, which reduces the burden on the system. After the stream pulling module pulls the current video information, it sends the current video information to the equipment inspection module, and the equipment inspection module performs quality inspection.

[0082] Specifically, the abnormal event detection system also includes an equipment inspection device. After receiving the equipment inspection instruction, the equipment inspection device allocates the target stream pulling module nearby according to the equipment information in the equipment inspection instruction; the target stream pulling module pulls the current video information of the corresponding acquisition device, and the current video information is compressed based on the acquisition video stream shot by the acquisition device at the current time point; the stream pulling module stores the current video information in the cloud storage module, and sends the equipment inspection task to the equipment inspection module; after receiving the equipment inspection task, the equipment inspection module downloads the corresponding current video information from the cloud storage space; according to the video compression format and video stream in the current video information, the target handle information for decoding the current video information is queried from the historical handle information; according to the target handle information, the current video information is decoded and processed to obtain the acquisition video stream at the current time point.

[0083] S530: Perform quality detection on the collected video stream to obtain a quality detection result, where the quality detection result indicates whether there is any abnormal device result in the corresponding collection device.

[0084] The quality inspection results include whether the acquisition device has any abnormal results, the type of device abnormality, and device information. After obtaining the acquired video stream of the acquisition device at the current time point, the abnormal event detection system performs quality inspection on the acquired video stream to determine whether there are missing frames in the acquired video stream, whether there are snowflakes, stripes, etc. in the screen, and obtains the quality inspection results; and pushes the quality inspection results to the client to inform the user. Among them, it can be determined whether there are missing frames by analyzing the coherence of each video frame in the acquired video stream, that is, whether the device is streaming normally; each video frame in the acquired video stream can be analyzed in the frequency domain to determine whether there are abnormalities such as snowflakes and stripes in the screen.

[0085] Combined with the above embodiments, the overall process of equipment inspection is described in detail. Figure 6 :

[0086] First, the equipment inspection device obtains the target inspection time point; and after the current time point reaches the target inspection time point, the corresponding equipment inspection instruction is issued; the pull stream module is allocated nearby according to the equipment inspection instruction; the pull stream module pulls the acquisition video stream shot by the corresponding acquisition device at the current time point, for example, the acquisition video stream of the preset time period can be pulled, and the preset time period can be set according to the demand, for example, it can be set to 20 seconds; the equipment inspection module obtains the acquisition video stream, performs quality inspection on the acquisition video stream, and obtains the quality inspection result; if the quality inspection result indicates that the corresponding acquisition device has an abnormal device result, the quality inspection result is pushed to the client. In addition, the video frame with the most abnormalities can also be pushed to the client as a cover. So that the user can understand. In this way, the acquisition equipment is inspected regularly to ensure the high availability of the acquisition equipment at a low cost.

[0087] In order to elaborate on the abnormal event detection method of the present application, Figure 7 The flowchart shown further illustrates it, and the details are as follows:

[0088] The abnormal event detection system includes a client, an equipment inspection device, an abnormal event detection device and a collection device. The equipment inspection device includes an equipment package management center, an equipment inspection node and a message center. The equipment inspection node includes a streaming module, a cloud storage module and an equipment inspection module; the abnormal event detection device includes a message center and an event detection node. The event detection node includes a streaming module, a cloud storage module and an analysis module.

[0089] First, the process of abnormal event detection is explained. First, when the acquisition device detects a change in the picture, it reports the detection trigger instruction to the message center; after the message center receives the detection trigger instruction, it allocates the nearest stream pulling module according to the device IP address in the detection trigger instruction; the stream pulling module sends a video stream pulling request to the corresponding target acquisition device; after the target acquisition device receives the video stream pulling request, it compresses the current video stream to obtain the target video information, and sends the target video information to the stream pulling module; after the stream pulling module receives the target video information, it uploads the target video information to the cloud storage module, and pushes the abnormal event detection task to the analysis module; after the analysis module receives the abnormal event detection task, it extracts the target video information from the cloud storage The module downloads the corresponding target video information; obtains the video compression format and video resolution of the target video information; allocates the target handle information according to the video compression format and video resolution, and reuses and allocates the handles with the same video compression format and video resolution; finally, the pulled target video information is output as YUV data of uniform resolution ("Y" represents brightness (Luminance or Luma), "U" and "V" represent chrominance (Chrominance or Chroma)); performs frame extraction abnormal event detection based on YUV data to obtain abnormal event detection results; at the same time, dynamic images of abnormal events can be produced, and the dynamic images can be pushed to the client to inform the user.

[0090] The process of equipment inspection is then explained. The client is a terminal APP that supports users to customize the target inspection time point and detection items (for example, the night inspection time does not detect black and white). If not set, the platform assigns a default target inspection time point; the equipment package management center is used to read the configuration of the acquisition device and issue equipment inspection tasks regularly according to the target inspection time point, and detect the video stream screen and equipment status; after receiving the equipment inspection task, the current video information at the current time point is pulled from the acquisition device through the stream pulling module; the stream pulling module stores the current video information in the cloud storage module, and sends the equipment inspection task and video stream address index to the equipment inspection module; after receiving the equipment inspection task, the equipment inspection module downloads the corresponding current video information from the cloud storage module, and hard-decodes the current video information through decoding handle reuse to obtain the acquired video stream; the acquired video stream is quality tested to obtain the quality test result, and the quality test result is pushed to the message center; the message center determines whether there is an abnormality, and if so, the quality test result is pushed to the client.

[0091] See also Figure 8 , Figure 8It is a structural diagram of an exemplary embodiment of an abnormal event detection device shown in the present application. The abnormal event detection device 800 includes a receiving module 810, an acquisition module 820 and a detection module 830. The receiving module 810 is used to receive a detection trigger instruction, which is sent when the acquisition device detects that there is a picture change in the current video stream being acquired; the acquisition module 820 is used to acquire the current video stream from the target acquisition device according to the detection trigger instruction; the detection module 830 is used to perform abnormal event detection processing on the current video stream to obtain an event detection result.

[0092] In the above scheme, the abnormal event detection device receives a detection trigger instruction, which is sent when the acquisition device detects that there is a picture change in the current video stream being acquired; the current video stream is obtained from the target acquisition device according to the detection trigger instruction; the current video stream is subjected to abnormal event detection processing to obtain an event detection result. In this way, whether the picture changes is used as a pre-detection, and after receiving the detection trigger instruction sent when the picture changes, the corresponding current video stream is obtained for abnormal event detection processing, so as to achieve accurate streaming, which can not only reduce the analysis cost, but also ensure the real-time nature of abnormal event detection.

[0093] Among them, the functions of each module can be found in the embodiment of the abnormal event detection method, and will not be repeated here.

[0094] In order to implement the abnormal event detection method of the above embodiment, the present application proposes another electronic device, which is specifically referred to as Fig. 9 , Fig. 9 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application.

[0095] The electronic device 900 includes a memory 910 and a processor 920 , wherein the memory 910 and the processor 920 are coupled.

[0096] The memory 910 is used to store program data, and the processor 920 is used to execute the program data to implement the abnormal event detection method of the above embodiment.

[0097] In this embodiment, the processor 920 may also be referred to as a CPU (Central Processing Unit). The processor 920 may be an integrated circuit chip having signal processing capabilities. The processor 920 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 920 may also be any conventional processor, etc.

[0098] The present application also provides a computer-readable storage medium, such as Fig.10 As shown, the computer-readable storage medium 1000 is used to store program data 1001. When the program data 1001 is executed by the processor, it is used to implement the abnormal event detection method in the method embodiment of the present application.

[0099] The method involved in the embodiment of the abnormal event detection method of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0100] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting abnormal events, characterized in that: The method comprises: Receiving a detection trigger instruction, wherein the detection trigger instruction is sent when the acquisition device detects that there is a picture change in the current video stream being acquired; Acquire the current video stream from the target acquisition device according to the detection trigger instruction; An abnormal event detection process is performed on the current video stream to obtain an event detection result.

2. The abnormal event detection method according to claim 1, characterized in that: The method is applied to an abnormal event detection system, which includes at least two event detection nodes; the step of acquiring the current video stream from a target acquisition device according to the detection trigger instruction includes: Determining a target event detection node from the at least two event detection nodes according to the device information in the detection trigger instruction; The current video stream is acquired from the target acquisition device through the target event detection node using the detection trigger instruction.

3. The abnormal event detection method according to claim 2, characterized in that: Each event detection node includes a streaming module and an analysis module; The step of acquiring the current video stream from the target acquisition device through the target event detection node using the detection trigger instruction includes: Pulling target video information from the target acquisition device through the stream pulling module, the target video information being obtained after the target acquisition device compresses the current video stream; The target video information is analyzed and processed by the analysis module to obtain the current video stream.

4. The abnormal event detection method according to claim 3, characterized in that: The step of pulling target video information from the target acquisition device through the stream pulling module includes: The stream pulling module determines the target acquisition device from the acquisition devices according to the device information in the detection trigger instruction; Pull the target video information from the target acquisition device.

5. The abnormal event detection method according to claim 3, characterized in that: The step of analyzing and processing the target video information by the analysis module to obtain the current video stream includes: Acquire the video compression format and video resolution in the target video information through the analysis module; querying target handle information for decoding the target video information from historical handle information according to the video compression format and the video resolution; The target video information is decoded according to the target handle information to obtain the current video stream.

6. The abnormal event detection method according to claim 1, characterized in that: The event detection result indicates whether there is an abnormal event in the current video stream, and the step of performing abnormal event detection processing on the current video stream to obtain the abnormal event detection result includes: Extracting a video frame to be detected from the current video stream according to a preset frame extraction rule; Performing feature extraction processing on the video frame to be detected to obtain target object features in the video frame to be detected; In response to the matching degree between the target object feature and the preset object feature being greater than a matching threshold, it is determined that an abnormal event exists in the video frame to be detected.

7. The abnormal event detection method according to claim 6, characterized in that: After the step of determining that an abnormal event exists in the video frame to be detected in response to the matching degree between the target object feature and the preset object feature being greater than a matching threshold, the method further includes: Selecting a preset number of target video frames from the to-be-detected video frames having the same abnormal event; The target video frame is synthesized according to the acquisition time point of the target video frame to obtain a dynamic image of the target video frame.

8. The abnormal event detection method according to claim 1, characterized in that: The method is applied to an abnormal event detection system, wherein the abnormal event detection system is provided with a target inspection time point for each acquisition device; the method further comprises: In response to the current time point reaching the target inspection time point, detecting whether the corresponding acquisition device has an acquisition video stream at the current time point; If it exists, obtaining the captured video stream shot at the current time point from the corresponding capture device; A quality test is performed on the collected video stream to obtain a quality test result, wherein the quality test result indicates whether there is any device abnormality result in the corresponding collection device.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: include: Program data is stored, and when the program data is executed by a processor, it is used to implement the method according to any one of claims 1 to 8.

Images