Monitoring video time epitome processing method and device, terminal and medium

Abstract

The invention discloses a monitoring video time epitome processing method and device, a terminal and a medium, and the scheme provided by the invention comprises the steps: carrying out the event recognition of a monitoring video through employing a trained video event recognition model, recording the alarm information corresponding to an alarm event when the occurrence of the corresponding alarm event is detected, and carrying out the recognition of the alarm information. The method comprises the steps of recording alarm information, generating alarm index information according to the recorded alarm information, and when it is detected that a time epitome processing instruction is triggered, performing frame extraction on an original monitoring video according to the alarm index information so as to generate a time epitome video containing important events, so that a lengthy monitoring video is converted into a short time epitome video, and the time epitome processing efficiency is improved. The key fragment with the important event can be quickly determined, time waste caused by checking all video files is avoided, and video checking efficiency and user experience are improved.

Description

Technical Field

[0001] The present application relates to the field of video technology, and in particular to a method for processing time-lapse images of surveillance videos. Background Art

[0002] With the development of video surveillance, the use of surveillance equipment has gradually become popular in many different scenarios. However, most surveillance equipment records videos all day and events. When users view the videos of the day, they need to manually click to review all the recorded videos, most of which are invalid videos, which is a waste of time. Summary of the invention

[0003] The present application provides a method, device, terminal and medium for processing time-lapse of surveillance videos, which are used to convert lengthy surveillance videos into short time-lapse videos, avoid wasting time by viewing all video files, and improve video review efficiency and user experience.

[0004] To achieve the above-mentioned invention object, the first aspect of the present application provides a method for processing a time-lapse image in a surveillance video, comprising:

[0005] Using a preset video event recognition model, the original surveillance video is used to identify alarm events, so as to record alarm information according to the recognition result, and to generate alarm index information according to the alarm information;

[0006] In response to the time-lapse processing instruction, the original surveillance video is frame-extracted according to the alarm index information to generate a time-lapse video.

[0007] Preferably, the alarm information includes: event sequence number, start time, end event and alarm type.

[0008] Preferably, in response to the time-lapse processing instruction, extracting frames from the original surveillance video according to the alarm index information to generate the time-lapse video includes:

[0009] Determine the duration ratio of each alarm event video according to the alarm index information;

[0010] Determine the video allocation duration of each alarm event video according to the duration ratio of each alarm event video and the configuration information of the miniature video duration;

[0011] According to the duration of each video, each alarm event video is framed to generate a time-lapse video.

[0012] Preferably, in response to the time-lapse processing instruction, extracting frames from the original surveillance video according to the alarm index information to generate the time-lapse video includes:

[0013] Dividing the original surveillance video into video segments of a plurality of unit time periods according to preset time period division configuration information;

[0014] According to the alarm index information, combined with the preset unit time period, determine the proportion of the number of alarms in each unit time period;

[0015] Determine the video allocation duration for each unit time period according to the alarm number ratio;

[0016] According to the duration of each video, the video segments of each unit time period are respectively extracted to generate a time-miniature video.

[0017] Preferably, according to the duration of each video allocation, extracting frames from each video segment of each unit time period to generate a time-lapse video includes:

[0018] The target alarm event videos are determined from the alarm event videos of each unit time period, and the target alarm event videos are framed to obtain time-miniature video segments corresponding to each target alarm event video, so as to splice the time-miniature video segments into a time-miniature video.

[0019] Preferably, it also includes:

[0020] According to the comparison result of the actual duration of the time-lapse video clip and the corresponding video allocation duration, if the time-lapse video clip is longer than the video allocation duration, the duration of the time-lapse video clip is adjusted according to the video allocation duration; if the time-lapse video clip is shorter than the video allocation duration, several target alarm event videos are determined from the corresponding unit time period for frame extraction processing.

[0021] Preferably, it also includes:

[0022] According to the alarm type, in combination with the preset alarm type filtering condition, the alarm information that meets the alarm type filtering condition is filtered.

[0023] At the same time, the second aspect of the present application provides a monitoring video time-lapse processing device, comprising:

[0024] An alarm event recognition unit is used to recognize alarm events on the original surveillance video through a preset video event recognition model, so as to record alarm information according to the recognition result, and generate alarm index information according to the alarm information;

[0025] The time-miniature processing unit is used to respond to the time-miniature processing instruction and extract frames from the original monitoring video according to the alarm index information to generate a time-miniature video.

[0026] A third aspect of the present application provides a monitoring video time-lapse processing terminal, comprising: a memory and a processor;

[0027] The memory is used to store program codes, and the program codes correspond to a method for processing time-lapse images in a surveillance video as provided in the first aspect of the present application;

[0028] The processor is used for reading and executing the program code.

[0029] The fourth aspect of the present application provides a computer-readable storage medium, which stores a program code, and the program code is used to be read and executed by a processor to implement a time-lapse processing method for monitoring video as provided in the first aspect of the present application.

[0030] It can be seen from the above technical solutions that this application has the following advantages:

[0031] The solution provided by the present application utilizes a trained video event recognition model to perform event recognition on surveillance videos; when a corresponding alarm event is detected, the alarm information corresponding to the alarm event is recorded, and alarm index information is generated based on the recorded alarm information; when a time-lapse processing instruction is detected to be triggered, frames are extracted from the original surveillance video based on the alarm index information to generate a time-lapse video containing important events, thereby realizing the conversion of lengthy surveillance videos into short time-lapse videos, being able to quickly determine the key clips of important events, avoiding the time wasted by checking all video files, and improving video review efficiency and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for processing time miniatures in surveillance videos provided in the present application.

[0034] Figure 2 A schematic diagram of the logical architecture of a system for implementing a method for processing time miniatures of surveillance videos provided in this application.

[0035] Figure 3 A flowchart of a complete implementation method of a time-lapse processing method for surveillance video provided in this application.

[0036] Figure 4A schematic diagram of a flow chart of an embodiment of a device for processing time-lapse images of surveillance videos provided in the present application.

[0037] Figure 5 A schematic diagram of a flow chart of an embodiment of a monitoring video time-lapse processing terminal provided in the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present application provide a method, device, terminal and medium for processing time-lapse images of surveillance videos, which are used to convert lengthy surveillance videos into short time-lapse images, avoid wasting time by viewing all video files, and improve video review efficiency and user experience.

[0039] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] First, a detailed description of an embodiment of a monitoring video time-lapse processing method provided by the present application is as follows:

[0041] See also Figure 1 and Figure 2 The present application provides an embodiment of a method for processing a time-lapse image of a surveillance video, comprising:

[0042] Step 101: Use a preset video event recognition model to identify alarm events on the original surveillance video, record alarm information according to the recognition result, and generate alarm index information according to the alarm information;

[0043] Step 102: In response to the time-lapse processing instruction, extract frames of the original surveillance video according to the alarm index information to generate a time-lapse video.

[0044] It should be noted that first, a pre-trained video event recognition model is used to identify alarm events in the original surveillance video. When an event alarm is triggered, the key information such as the start time, end time, sequence number, type, etc. of the alarm event is recorded. Then, the recorded alarm information is written to the alarm index information to obtain the corresponding alarm index information for subsequent time-lapse video generation.

[0045] Among them, the video event recognition model in this embodiment preferably adopts an object detection model trained based on the YOLO object detection algorithm, and constructs a YOLO object detection model for identifying video events by using pre-labeled alarm event images as training samples.

[0046] When it is detected that the preset timing condition is met or a time-miniature processing request is received from the user, the time-miniature processing instruction is triggered, and the original surveillance video is framed according to the alarm index information to convert the lengthy surveillance video into a short time-miniature video.

[0047] Furthermore, in response to the time-lapse processing instruction in step 102 of this embodiment, extracting frames of the original surveillance video according to the alarm index information to generate the time-lapse video includes:

[0048] According to the alarm index information, determine the duration ratio of each alarm event video;

[0049] Determine the video allocation duration of each alarm event video according to the duration ratio of each alarm event video and the configuration information of the miniature video duration;

[0050] According to the duration of each video, each alarm event video is framed to generate a time-lapse video.

[0051] or;

[0052] According to the preset time period division configuration information, the original surveillance video is divided into video clips of multiple unit time periods;

[0053] According to the alarm index information and the preset unit time period, determine the proportion of the number of alarms in each unit time period;

[0054] Determine the video allocation duration for each unit time period based on the proportion of alarm numbers;

[0055] According to the duration of each video, the video clips of each unit time period are extracted to generate a time-lapse video.

[0056] It should be noted that, with regard to the implementation method of frame extraction and generation of time-lapse video, the present application provides the above several methods. One method is to determine the allocated time of each alarm event in the time-lapse video according to the video length of each alarm event and the preset time threshold of the time-lapse video, and extract frames from the video of each alarm event according to the allocated time corresponding to each alarm event to obtain a time-lapse video clip corresponding to the allocated time, which are finally spliced ​​into a complete time-lapse video.

[0057] The other method is to first divide the configuration information according to the preset time period, and divide the original surveillance video into multiple video segments, the duration of each video segment is equivalent to the duration of the preset unit time period, and the duration of the unit time period is generally one hour.

[0058] According to the alarm index information and the preset unit time period, the proportion of the number of alarms in each unit time period is determined, and the video allocation duration of each unit time period is determined based on the proportion of the number of alarms. Then, according to the allocation duration of each unit time period, the video fragments of each unit time period are extracted to obtain time-miniature video fragments equivalent to the allocation duration, which are finally spliced ​​into a complete time-miniature video.

[0059] Furthermore, according to the duration of each video, the video segments of each unit time period are extracted to generate a time-lapse video, including:

[0060] The target alarm event videos are determined from the alarm event videos of each unit time period, and frames are extracted from the target alarm event videos to obtain time-miniature video segments corresponding to each target alarm event video, so as to splice each time-miniature video segment into a time-miniature video.

[0061] Furthermore, it also includes:

[0062] According to the comparison result of the actual duration of the time-lapse video clip and the corresponding video allocation duration, if the time-lapse video clip is longer than the video allocation duration, the duration of the time-lapse video clip is adjusted according to the video allocation duration; if the time-lapse video clip is shorter than the video allocation duration, several target alarm event videos are determined from the corresponding unit time period for frame extraction processing.

[0063] It should be noted that if the compressed duration is greater than the allocated recording duration for that hour, the allocated duration shall prevail; if the compressed duration is less than the allocated recording duration for that hour, the alarm event video shall continue to be appended to the time-lapse file.

[0064] Furthermore, it also includes:

[0065] According to the alarm type, combined with the preset alarm type filtering conditions, the alarm information that meets the alarm type filtering conditions is filtered. Among them, the alarm types that need to be filtered can be freely configured, including but not limited to: face control / electric vehicle recognition / area intrusion, etc.

[0066] like Figure 3 As shown, the following is an overall implementation example of the method for processing time-lapse images in surveillance videos provided by this application, which is as follows:

[0067] 1) Important event records:

[0068] Step 1: Initialize the video event recognition model.

[0069] Step 2: The video event recognition model triggers event alarm.

[0070] Step 3: Filter alarm types (configurable, face control / electric vehicle recognition / area intrusion, etc.)

[0071] Step 4: Record key information such as the start time, sequence number, and type of the alarm event.

[0072] Step 5: If the event alarms continuously, record and extend the end time of the alarm setting.

[0073] Step 6: Write the recorded alarm video information to the alarm index information. The writing granularity can be set to: 1 minute. If there is a write failure, retry logic.

[0074] 2) Generate a time snapshot file:

[0075] Step 1: When the specified time arrives or a request message is received from the user, the logic of generating the time snapshot is started and the alarm index information is read.

[0076] Step 2: Read the alarm index information and coordinate the data: obtain the total number of alarms per hour and obtain the alarm list (granularity: hour).

[0077] Step 3: Analyze the proportion of alarms per hour and allocate the duration of the time snapshot per hour in proportion (total duration: 3 minutes).

[0078] Step 4: Based on the obtained time length (in seconds) allocated to each hour, use a random value to find the sequence number of the alarm event corresponding to that hour.

[0079] Step 5: Find the corresponding video file through the start time and end time corresponding to the serial number.

[0080] Step 6: Load the video file and prepare for decoding.

[0081] Step 7: Convert the video file into frame data.

[0082] Step 8: Store the frame data into the video buffer.

[0083] Step 9: Determine the time interval or frequency of frame extraction, wherein the frame extraction method can extract frames at a frequency of 25fps, and I frames are retained.

[0084] Step 10: Extract frame data that meets the rules of step 9 from the frame sequence.

[0085] Step 11: Read frame data from the buffer.

[0086] Step 12: Encode the frame data.

[0087] Step 13: Write the encoded video data to the time-lapse file.

[0088] Step 14: If the compressed duration is greater than the allocated recording duration for that hour, the allocated duration shall prevail; if the compressed duration is less than the allocated recording duration for that hour, continue to append the alarm event video to the time-lapse file (repeat steps 5 to 13).

[0089] Step 15: Generate index file of time snapshot file.

[0090] The above is a detailed description of an embodiment of a surveillance video time miniature processing method provided by the present application. The scheme of the present application records an index file of event information, extracts frames from the video file, and stores them in the video cache area to record a time miniature file, thereby realizing the time miniature function of the device measurement, and returning the time miniature implementation subject to the device measurement, thereby achieving consistent time miniature effect and ensuring that users can quickly check the unity of important events of the day.

[0091] The following is a detailed description of an embodiment of a monitoring video time-lapse processing device, a terminal embodiment, and a computer-readable storage medium embodiment provided by the present application, specifically as follows:

[0092] See also Figure 4 The present application provides an embodiment of a monitoring video time-lapse processing device, including:

[0093] The alarm event recognition unit 201 is used to recognize alarm events on the original surveillance video through a preset video event recognition model, record alarm information according to the recognition result, and generate alarm index information according to the alarm information;

[0094] The time-lapse processing unit 202 is used to respond to the time-lapse processing instruction and extract frames from the original monitoring video according to the alarm index information to generate a time-lapse video.

[0095] The present application provides an embodiment of a monitoring video time-lapse processing terminal, the terminal types include but are not limited to: personal computers, servers, and embedded intelligent devices, etc., the main components of the terminal include: a memory 33 and a processor 31, the memory 33 and the processor 31 can be connected via a communication bus 34;

[0096] The memory is used to store program codes, and the program codes correspond to a method for processing time miniatures in a monitoring video according to the above-mentioned embodiment;

[0097] The processor is used to read and execute program code.

[0098] The present application provides an embodiment of a computer-readable storage medium, wherein the computer-readable storage medium stores program code, and the program code is used to be read and executed by a processor to implement a time-lapse processing method for monitoring video as provided in the above embodiment.

[0099] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0100] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0101] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0102] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0106] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing time miniatures in surveillance video, characterized in that: include: Using a preset video event recognition model, the original surveillance video is used to identify alarm events, so as to record alarm information according to the recognition result, and to generate alarm index information according to the alarm information; In response to the time-lapse processing instruction, the original surveillance video is frame-extracted according to the alarm index information to generate a time-lapse video.

2. A method for processing time-lapse images of surveillance videos according to claim 1, characterized in that: The alarm information includes: event sequence number, start time, end event and alarm type.

3. The method for processing time-lapse images of surveillance videos according to claim 1, characterized in that: In response to the time-lapse processing instruction, extracting frames from the original surveillance video according to the alarm index information to generate the time-lapse video includes: Determine the duration ratio of each alarm event video according to the alarm index information; Determine the video allocation duration of each alarm event video according to the duration ratio of each alarm event video and the configuration information of the miniature video duration; According to the duration of each video, each alarm event video is framed to generate a time-lapse video.

4. The method for processing time-lapse images of surveillance videos according to claim 1, characterized in that: In response to the time-lapse processing instruction, extracting frames from the original surveillance video according to the alarm index information to generate the time-lapse video includes: Dividing the original surveillance video into video segments of a plurality of unit time periods according to preset time period division configuration information; According to the alarm index information, combined with the preset unit time period, determine the proportion of the number of alarms in each unit time period; Determine the video allocation duration for each unit time period according to the alarm number ratio; According to the duration of each video, the video segments of each unit time period are respectively extracted to generate a time-miniature video.

5. A method for processing time-lapse images of surveillance videos according to claim 4, characterized in that: According to the duration of each video, extracting frames from each unit time segment to generate a time miniature video includes: The target alarm event videos are determined from the alarm event videos of each unit time period, and the target alarm event videos are framed to obtain time-miniature video segments corresponding to each target alarm event video, so as to splice the time-miniature video segments into a time-miniature video.

6. A method for processing time-lapse images of surveillance videos according to claim 5, characterized in that: Also includes: According to the comparison result of the actual duration of the time-lapse video clip and the corresponding video allocation duration, if the time-lapse video clip is longer than the video allocation duration, the duration of the time-lapse video clip is adjusted according to the video allocation duration; if the time-lapse video clip is shorter than the video allocation duration, several target alarm event videos are determined from the corresponding unit time period for frame extraction processing.

7. The method for processing time-lapse images of surveillance videos according to claim 2, characterized in that: Also includes: According to the alarm type, in combination with the preset alarm type filtering condition, the alarm information that meets the alarm type filtering condition is filtered.

8. A monitoring video time-lapse processing device, characterized in that: include: An alarm event recognition unit is used to recognize alarm events on the original surveillance video through a preset video event recognition model, so as to record alarm information according to the recognition result, and generate alarm index information according to the alarm information; The time-miniature processing unit is used to respond to the time-miniature processing instruction and extract frames from the original monitoring video according to the alarm index information to generate a time-miniature video.

9. A monitoring video time-lapse processing terminal, characterized in that: include: Memory and processor; The memory is used to store program codes, and the program codes correspond to a method for processing time-lapse images in surveillance videos according to any one of claims 1 to 7; The processor is used for reading and executing the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes are used to be read and executed by a processor to implement a method for processing time-lapse images in surveillance videos as described in any one of claims 1 to 7.

Images