Video event data processing method, system, equipment and medium

Abstract

The invention discloses a video event data processing method, system and device and a medium, and the method comprises the steps: carrying out the frame picture change judgment of a first key frame according to a second key frame, and obtaining a change judgment result; if the change judgment result is that the frame picture changes, performing data caching on the original video stream according to the first key frame to obtain a first intermediate video stream; performing video event identification on the first intermediate video stream to obtain a third key frame; performing video event termination identification on the first intermediate video stream according to the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream; and performing event time correction on the second intermediate video stream according to the second key frame, the third key frame and the fourth key frame to obtain video event data. According to the method, the judgment precision of the event occurrence time can be improved, and the event content integrity of the video event data is improved. The invention relates to the technical field of data processing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, system, device and medium for processing video event data. Background Art

[0002] A video surveillance system is a system that uses cameras and other devices to record real-time or scheduled videos of specific areas. It has attracted attention because it can analyze and identify video surveillance data through intelligent AI.

[0003] At present, the existing technology usually analyzes and identifies video surveillance data through intelligent AI, and when the intelligent AI identifies an event, it retains a few frames of video surveillance images as video event data for storage. This method has poor accuracy in determining the occurrence time of the event, and the event content completeness of the retained video event data is not satisfactory.

[0004] Therefore, the problems existing in the prior art still need to be solved and optimized. Summary of the invention

[0005] In order to solve at least one of the above-mentioned technical problems, the present application provides a method, system, device and medium for processing video event data, wherein the processing method can improve the accuracy of determining the occurrence time of an event and improve the completeness of the event content of the video event data.

[0006] According to a first aspect of the present application, a method for processing video event data is provided, comprising:

[0007] Acquire a first key frame and a second key frame in an original video stream, wherein the second key frame is a key frame subsequent to the first key frame;

[0008] According to the second key frame, performing frame picture change judgment on the first key frame to obtain a change judgment result;

[0009] If the change judgment result is that the frame image has changed, data of the original video stream is cached according to the first key frame to obtain a first intermediate video stream, and the first key frame of the first intermediate video stream is the first key frame;

[0010] Performing video event recognition on the first intermediate video stream to obtain a third key frame, wherein the third key frame is a key frame that successfully recognizes the event type causing frame image change in the first intermediate video stream;

[0011] According to the third key frame and the first key frame, video event termination recognition is performed on the first intermediate video stream to obtain a fourth key frame and a second intermediate video stream, wherein the fourth key frame is a key frame that is after the third key frame in the first intermediate video stream and has the same frame picture as the first key frame;

[0012] According to the second key frame, the third key frame and the fourth key frame, event time correction is performed on the second intermediate video stream to obtain video event data.

[0013] In some embodiments, performing frame picture change judgment on the first key frame according to the second key frame to obtain a change judgment result includes:

[0014] Obtaining a preset first threshold value;

[0015] According to the second key frame, performing frame picture difference calculation on the first key frame to obtain a frame picture difference value;

[0016] The first threshold value and the frame picture difference value are compared to obtain the change judgment result.

[0017] In some embodiments, performing video event recognition on the first intermediate video stream to obtain a third key frame includes:

[0018] Obtaining a current video frame in the first intermediate video stream;

[0019] Performing event type recognition on the current video frame to obtain an event type recognition result;

[0020] If the event type recognition result is that the event type recognition is unsuccessful, the current video frame is updated, and then the step of obtaining the current video frame in the first intermediate video stream is returned; or, if the event type recognition result is that the event type recognition is successful, the current video frame is determined as the third key frame.

[0021] In some embodiments, the method further comprises:

[0022] According to the current video frame, performing a first frame picture comparison on the first key frame to obtain a first frame picture comparison result;

[0023] If the first frame comparison result is that the frames are consistent, the current video frame is determined as the fifth key frame, and the first intermediate video stream is cleared according to the fifth key frame; or, if the first frame comparison result is that the frames are inconsistent, the process returns to the step of performing event type recognition on the current video frame to obtain an event type recognition result;

[0024] The fifth key frame is a key frame having the same frame picture as the first key frame in the first intermediate video stream before the key frame that successfully identifies the event type causing the frame picture change in the first intermediate video stream.

[0025] In some embodiments, performing video event termination identification on the first intermediate video stream according to the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream includes:

[0026] According to the third key frame, an intermediate video frame in the first intermediate video stream is obtained, where the intermediate video frame is a key frame after the third key frame;

[0027] According to the intermediate video frame, performing a second frame picture comparison on the first key frame to obtain a second frame picture comparison result;

[0028] If the second frame picture comparison result is that the frames are consistent, the intermediate video frame is determined as the fourth key frame, and according to the fourth key frame, data of the first intermediate video stream is intercepted to obtain the second intermediate video stream; or, if the second frame picture comparison result is that the frames are inconsistent, the intermediate video frame is updated, and then the step of obtaining the intermediate video frame in the first intermediate video stream according to the third key frame is returned.

[0029] In some embodiments, performing a second frame picture comparison on the first key frame according to the intermediate video frame to obtain a second frame picture comparison result includes:

[0030] Generate a second threshold value according to the first threshold value, wherein the second threshold value is smaller than the first threshold value;

[0031] According to the intermediate video frame, performing frame picture similarity calculation on the second key frame to obtain frame picture similarity;

[0032] The second threshold value is compared with the frame picture similarity to obtain the second frame picture comparison result.

[0033] In some embodiments, performing event time correction on the second intermediate video stream according to the second key frame, the third key frame, and the fourth key frame to obtain video event data includes:

[0034] Obtain a first threshold value and a second threshold value;

[0035] According to the first threshold value and the second threshold value, performing event time calibration processing on the timestamp information of the second key frame, the timestamp information of the third key frame, and the timestamp information of the fourth key frame to obtain an event calibration time;

[0036] According to the event calibration time, the second intermediate video stream is calibrated and corrected to obtain the video event data.

[0037] According to a second aspect of the present application, a system for processing video event data is provided, comprising:

[0038] A first processing unit is used to obtain a first key frame and a second key frame in an original video stream, where the second key frame is a key frame subsequent to the first key frame;

[0039] A second processing unit is used to perform frame picture change judgment on the first key frame according to the second key frame to obtain a change judgment result;

[0040] A third processing unit is configured to cache data of the original video stream according to the first key frame to obtain a first intermediate video stream if the change judgment result is that the frame picture has changed, wherein the first key frame of the first intermediate video stream is the first key frame;

[0041] a fourth processing unit, configured to perform video event recognition on the first intermediate video stream to obtain a third key frame, wherein the third key frame is a key frame that successfully recognizes an event type that causes a frame change in the first intermediate video stream;

[0042] a fifth processing unit, configured to perform video event termination recognition on the first intermediate video stream according to the third key frame and the first key frame, to obtain a fourth key frame and a second intermediate video stream, wherein the fourth key frame is a key frame that is after the third key frame in the first intermediate video stream and has the same frame picture as the first key frame;

[0043] The sixth processing unit is used to perform event time correction on the second intermediate video stream according to the second key frame, the third key frame and the fourth key frame to obtain video event data.

[0044] According to a third aspect of the present application, a computer device is provided, comprising:

[0045] at least one processor;

[0046] at least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of the above aspects.

[0048] According to a fourth aspect of the present application, a computer-readable storage medium is provided, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the method described in any of the above aspects.

[0049] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0050] The present application provides a method, system, device and medium for processing video event data, wherein the method obtains a first key frame and a second key frame in an original video stream, wherein the second key frame is a key frame after the first key frame; based on the second key frame, a frame picture change judgment is performed on the first key frame to obtain a change judgment result; if the change judgment result is that the frame picture has changed, data is cached on the original video stream based on the first key frame to obtain a first intermediate video stream, wherein the first key frame of the first intermediate video stream is the first key frame; video event recognition is performed on the first intermediate video stream to obtain a third key frame, wherein the third key frame is a key frame that successfully identifies the event type that causes the frame picture change in the first intermediate video stream; based on the third key frame and the first key frame, video event termination recognition is performed on the first intermediate video stream to obtain a fourth key frame and a second intermediate video stream, wherein the fourth key frame is a key frame that is after the third key frame in the first intermediate video stream and has the same frame picture as the first key frame; event time correction is performed on the second intermediate video stream based on the second key frame, the third key frame and the fourth key frame to obtain video event data. The method identifies the termination of a video event on a first intermediate video stream based on the third key frame and the first key frame, and can obtain a second intermediate video stream with higher event content completeness. In addition, the method also corrects the event occurrence time of the second intermediate video stream based on the second key frame, the third key frame and the fourth key frame, and can effectively improve the accuracy of determining the occurrence time of an event while ensuring the integrity of the event content. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of a method for processing video event data is provided for an embodiment of the present application;

[0052] Figure 2 A detailed flow chart of step S120 is provided for an embodiment of the present application;

[0053] Figure 3 A detailed flow chart of step S140 provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of an optional flow chart of a method for processing video event data provided in an embodiment of the present application;

[0055] Figure 5 A logic diagram of step S140 provided in an embodiment of the present application;

[0056] Figure 6 A detailed flow chart of step S150 provided in an embodiment of the present application;

[0057] Figure 7 A detailed flow chart of step C2 provided in an embodiment of the present application;

[0058] Figure 8 A detailed flow chart of step S160 provided in an embodiment of the present application;

[0059] Fig. 9 A schematic diagram of a framework of a video event data processing system provided in an embodiment of the present application;

[0060] Fig.10 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0062] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0064] At present, the existing technology usually analyzes and identifies video surveillance data through intelligent AI, and when the intelligent AI identifies an event, it retains several frames of video surveillance images as video event data for storage. This method usually determines the time when the intelligent AI identifies the event type as the time of event occurrence, and retains several adjacent frames of images at the time of occurrence as video event data. However, since it takes a certain amount of processing time for the intelligent AI to identify the specific event type of the event, and there are situations where the picture changes of adjacent images of the video surveillance data are relatively small when some events occur, this method cannot accurately determine the time of event occurrence. The accuracy of its determination of the time of event occurrence is poor, and the completeness of the event content of the retained video event data is unsatisfactory.

[0065] In view of this, an embodiment of the present application provides a method, system, device and medium for processing video event data, wherein the processing method identifies the termination of a video event on a first intermediate video stream based on a third key frame and a first key frame, and can obtain a second intermediate video stream with higher event content completeness; in addition, the method also corrects the event occurrence time of the second intermediate video stream based on the second key frame, the third key frame and the fourth key frame, and can effectively improve the accuracy of determining the occurrence time of the event while ensuring the integrity of the event content.

[0066] A method, system, device and medium for processing video event data provided in an embodiment of the present application can be specifically illustrated by the following embodiments. First, a method for processing video event data in an embodiment of the present application is described.

[0067] The method for processing video event data provided by the embodiment of the present application can be applied to video surveillance application scenarios. In video surveillance application scenarios, a video surveillance service provider can analyze and identify events in video surveillance data through the processing method provided by the embodiment of the present application, which can make the content integrity of the retained video event data higher and effectively improve the accuracy of determining the time when the event occurred.

[0068] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0069] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0070] Reference Figure 1 , Figure 1 It is an optional flowchart of a method for processing video event data provided in an embodiment of the present application, which may include but is not limited to steps S110 to S160.

[0071] Step S110, obtaining a first key frame and a second key frame in the original video stream, wherein the second key frame is a key frame subsequent to the first key frame;

[0072] In an embodiment of the present application, the original video stream may be video surveillance data acquired in real time by a camera or other imaging device, and the first key frame and the second key frame may be two current adjacent frame images in the original video stream, wherein the first key frame is the previous frame image of the two adjacent frame images, and the second key frame is the next frame image of the two adjacent frame images.

[0073] Step S120: performing frame image change judgment on the first key frame according to the second key frame to obtain a change judgment result;

[0074] In the embodiment of the present application, the frame picture change judgment in step S120 is used to judge whether the frame picture of the second key frame has changed compared with the frame picture of the first key frame, so as to obtain a change judgment result, and the change judgment result is used to indicate whether the frame picture between the first key frame and the second key frame has changed. In addition, the event causing the frame picture change can be at least one of a person, an animal, a vehicle, an object, etc. that breaks into the frame picture.

[0075] Reference Figure 2 In some embodiments, the step S120, performing frame picture change judgment on the first key frame according to the second key frame to obtain a change judgment result, includes:

[0076] A1. Obtaining a preset first threshold value;

[0077] A2. Calculate the frame difference of the first key frame according to the second key frame to obtain a frame difference value;

[0078] A3. Compare the first threshold value and the frame image difference value to obtain the change judgment result.

[0079] In the embodiment of the present application, first, a preset first threshold value can be obtained, which is a threshold value of the difference between the frame images of the first key frame and the second key frame, and the specific value of the first threshold value can be set according to the actual situation. Step A2 can be to calculate the difference between the frame image of the second key frame and the frame image of the first key frame, and the frame image difference value can be determined based on the mean square error (MSE), the structural similarity index (SSIM), the mutual information (MI), etc.

[0080] It can be understood that step A3 can be to compare the size relationship between the first threshold value and the frame picture difference value to obtain the change judgment result. Specifically, if the frame picture difference value is determined based on the mean square error (MSE), the larger the frame picture difference value, the greater the difference between the first key frame and the second key frame. On the contrary, the smaller the frame picture difference value, the smaller the difference between the first key frame and the second key frame. At this time, when the frame picture difference value is greater than the first threshold value, a change judgment result representing that the frame picture has changed can be obtained; or, when the frame picture difference value is less than or equal to the first threshold value, a change judgment result representing that the frame picture has not changed can be obtained. The frame picture difference values ​​of other acquisition methods are similar, and can be simply deduced by analogy, and this application will not repeat them here.

[0081] Step S130: If the change judgment result is that the frame image has changed, data of the original video stream is cached according to the first key frame to obtain a first intermediate video stream, wherein the first key frame of the first intermediate video stream is the first key frame;

[0082] In an embodiment of the present application, if the change judgment result is that the frame image has changed, the original video stream can be cached based on the timestamp information of the first key frame to obtain a first intermediate video stream, which includes the first key frame and the frame image after the first key frame, while retaining the timestamp information of the second key frame.

[0083] It can be understood that if the change judgment result is that the frame image has not changed, the current first key frame and the second key frame can be updated. Specifically, the current second key frame can be used as the first key frame in the next frame image change judgment process, and the next key frame of the current second key frame can be used as the second key frame in the next frame image change judgment process, and then the process returns to step S110; or, the next key frame of the current two key frames can be used as the first key frame in the next frame image change judgment process, and then the process returns to step S110.

[0084] Step S140: Perform video event recognition on the first intermediate video stream to obtain a third key frame, where the third key frame is a key frame that successfully recognizes the event type that causes frame image change in the first intermediate video stream;

[0085] In an embodiment of the present application, the video event recognition of step S140 is used for the event type of intrusion into the frame picture in the first intermediate video stream. Specifically, when an event such as a person, a car, or an object intrudes into the frame picture, the frame picture of the event in the first intermediate video stream will gradually become complete. Therefore, in an embodiment of the present application, step S140 can be to perform frame-by-frame recognition of each frame image in the first intermediate video stream through intelligent AI. If the intelligent AI successfully identifies the event type of the intrusion event in a certain frame image, the frame image is determined as the third key frame, and the third key frame is a key frame that successfully identifies the event type that causes the frame picture change in the first intermediate video stream.

[0086] Reference Figure 3 In some embodiments, the step S140 of performing video event recognition on the first intermediate video stream to obtain a third key frame includes:

[0087] B1. Obtain a current video frame in the first intermediate video stream;

[0088] B2. Perform event type recognition on the current video frame to obtain an event type recognition result;

[0089] B3. If the event type recognition result is that the event type recognition is unsuccessful, the current video frame is updated, and then the step of returning to execute the step of obtaining the current video frame in the first intermediate video stream;

[0090] Or, B4: if the event type recognition result is that the event type recognition is successful, the current video frame is determined as the third key frame.

[0091] In an embodiment of the present application, the current video frame may be a frame image currently recognized by the intelligent AI in the frame-by-frame recognition process. Specifically, for a certain frame image recognition of the intelligent AI, step B2 may be based on video AI recognition and analysis technology, extracting the frame features of the current video frame, and matching the extracted frame features with the known event types in the database to obtain an event type recognition result. Specifically, if the current video frame matches the corresponding event type, the obtained event type recognition result may be a successful event type recognition; or, if the current video frame does not match the corresponding event type, the obtained event type recognition result may be an unsuccessful event type recognition.

[0092] It is understandable that if the event type recognition result is that the event type recognition is unsuccessful, the current video frame can be updated, specifically, the next frame image of the current video frame can be determined as the current video frame in the next frame image recognition process, and then return to step B1. Alternatively, if the event type recognition result is that the event type recognition is successful, the current video frame can be determined as the third key frame.

[0093] Reference Figure 4 and Figure 5 , further, the method also includes:

[0094] B11. Performing a first frame picture comparison on the first key frame according to the current video frame to obtain a first frame picture comparison result;

[0095] B12. If the comparison result of the first frame is that the frames are consistent, the current video frame is determined as the fifth key frame, and the first intermediate video stream is cleared according to the fifth key frame;

[0096] Alternatively, B13, if the comparison result of the first frame is that the frames are inconsistent, returning to the step of performing event type recognition on the current video frame to obtain an event type recognition result;

[0097] The fifth key frame is a key frame having the same frame picture as the first key frame in the first intermediate video stream before the key frame that successfully identifies the event type causing the frame picture change in the first intermediate video stream.

[0098] In an embodiment of the present application, after executing step B1 and before executing step B2, the frame picture of the first key frame can also be compared with the frame picture of the current video frame to obtain the first frame picture comparison result; then, if the first frame picture comparison result is that the frame picture of the current video frame is consistent with the frame picture of the first key frame, it means that before the intelligent AI successfully identifies the event type of the event, the monitoring picture of the first intermediate video stream has been restored. At this time, the current video frame can be determined as the fifth key frame, and then based on the timestamp information of the fifth key frame, the cached first intermediate video stream can be cleared to reduce the number of invalid event reminders in the video surveillance system.

[0099] It can be understood that if the comparison result of the first frame is that the frame images of the current video frame and the first key frame are inconsistent, it means that the monitoring image of the first intermediate video stream has not been restored before the intelligent AI successfully identifies the event type of the event. At this time, it can return to execute step B2 to realize the event type identification of the current video frame.

[0100] Step S150: Perform video event termination recognition on the first intermediate video stream according to the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream, wherein the fourth key frame is a key frame that is after the third key frame in the first intermediate video stream and has the same frame picture as the first key frame;

[0101] In the embodiment of the present application, the video event termination identification in step S150 is used to determine whether the event is terminated after it occurs, and it can specifically identify each frame image in the first intermediate video stream frame by frame to obtain the fourth key frame. Specifically, if the event is terminated after it occurs, a key frame that is the same as the frame image of the first key frame can be found in the first intermediate video stream, and the found key frame is determined as the fourth key frame. At the same time, the second intermediate video stream can also be determined based on the first key frame and the fourth key frame, and the second terminated video stream contains all video content data from the occurrence of the event to the termination.

[0102] Reference Figure 6 In some embodiments, the step S150 of performing video event termination identification on the first intermediate video stream according to the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream includes:

[0103] C1. According to the third key frame, obtain an intermediate video frame in the first intermediate video stream, where the intermediate video frame is a key frame after the third key frame;

[0104] In an embodiment of the present application, for a frame recognition in the first intermediate video stream, step C1 may be to obtain an intermediate video frame in the first intermediate video stream, where the intermediate video frame is a key frame corresponding to the frame recognition process and after the third key frame.

[0105] C2. performing a second frame picture comparison on the first key frame according to the intermediate video frame to obtain a second frame picture comparison result;

[0106] Reference Figure 7 Further, the step C2, performing a second frame picture comparison on the first key frame according to the intermediate video frame to obtain a second frame picture comparison result, includes:

[0107] C21. Generate a second threshold value according to the first threshold value, where the second threshold value is smaller than the first threshold value;

[0108] C22. Perform frame similarity calculation on the second key frame according to the intermediate video frame to obtain frame similarity;

[0109] C23. Compare the second threshold value with the frame image similarity to obtain the second frame image comparison result.

[0110] In the embodiment of the present application, step C21 may randomly generate a specific value less than the first threshold value as the second threshold value, or may preset a specific value less than the first threshold value as the second threshold value. Step C22 may calculate the similarity between the intermediate video frame and the second key frame, which may be determined based on gray-level matching, histogram matching, image similarity evaluation based on deep learning, etc., so as to obtain the frame similarity.

[0111] It is understandable that step C23 may be to compare the magnitude relationship between the second threshold value and the frame picture similarity. If the frame picture similarity is greater than the second threshold value, a second frame picture comparison result representing the consistency of the frame pictures may be obtained; or, if the frame picture similarity is less than or equal to the second threshold value, a second frame picture comparison result representing the inconsistency of the frame pictures may be obtained. It is worth mentioning that the contents of the first frame picture comparison and the second frame picture comparison in the embodiment of the present application are similar and can be simply deduced by analogy.

[0112] C3. If the comparison result of the second frame is that the frames are consistent, the intermediate video frame is determined as a fourth key frame, and data of the first intermediate video stream is intercepted according to the fourth key frame to obtain a second intermediate video stream;

[0113] Alternatively, C4, if the comparison result of the second frame is that the frames are inconsistent, the intermediate video frame is updated, and then the step of obtaining the intermediate video frame in the first intermediate video stream according to the third key frame is returned.

[0114] In an embodiment of the present application, if the comparison result of the second frame is that the frames are consistent, it means that after the intelligent AI successfully identifies the event type of the event, the monitoring screen of the first intermediate video stream has been restored. At this time, the intermediate video frame can be determined as the fourth key frame, and then based on the timestamp information of the fourth key frame and the timestamp information of the first key frame, the cached first intermediate video stream is intercepted, thereby intercepting the second intermediate data stream that records all frame images between the first key frame to the fourth key frame;

[0115] It can be understood that if the comparison result of the second frame is inconsistent, it means that after the intelligent AI successfully identifies the event type of the event, the monitoring screen of the first intermediate video stream has not been restored. At this time, the intermediate video frame can be updated. Specifically, the next key frame of the intermediate video frame can be used as the intermediate video frame in the next second frame comparison process, and then return to execute step C1.

[0116] Step S160: Perform event time correction on the second intermediate video stream according to the second key frame, the third key frame and the fourth key frame to obtain video event data.

[0117] In an embodiment of the present application, after obtaining a second intermediate data stream recording all frame images between the first key frame to the fourth key frame, the second intermediate data stream records all video content data from the occurrence of an event to its termination. Based on the respective timestamp information of the second key frame, the third key frame and the third key frame, the video data related to non-event identification in the front section of the cache of the second intermediate data stream can be corrected and removed to obtain the corrected video event data.

[0118] Reference Figure 8 In some embodiments, the step S160 of performing event time correction on the second intermediate video stream according to the second key frame, the third key frame and the fourth key frame to obtain video event data includes:

[0119] D1, obtaining a first threshold value and a second threshold value;

[0120] D2. Performing event time calibration processing on the timestamp information of the second key frame, the timestamp information of the third key frame, and the timestamp information of the fourth key frame according to the first threshold value and the second threshold value to obtain an event calibration time;

[0121] D3. Calibrate and correct the second intermediate video stream according to the event calibration time to obtain the video event data.

[0122] In the embodiment of the present application, the first threshold value in step D1 may be the first threshold value in the aforementioned step A1, and the second threshold value in step D1 may be the second threshold value in the aforementioned step C21. The event time calibration process in step D2 may be to calculate the event calibration time of the second intermediate video stream, and the event calibration time may be determined by a preset weighted linear algorithm. Specifically, the equivalent expression of the event calibration time may be:

[0123] T=y1×(t2-t0)-y2×(t1-t0)+b

[0124] Among them, T is the event calibration time of the second intermediate video stream; y1 is the first threshold value; t2 is the timestamp information of the fourth key frame; t0 is the timestamp information of the second key frame; y2 is the second threshold value; t1 is the timestamp information of the third key frame; b is a preset bias item.

[0125] It can be understood that step D3 can be based on the acquired event calibration time, and remove and correct the front-end non-event recognized video frame images in the second intermediate video stream to obtain a corrected second intermediate video stream (i.e., video event data), which can specifically be to remove the video frame images of the second intermediate video stream before the event calibration time, and retain the video frame images of the second intermediate video stream after the event calibration time.

[0126] Fig. 9 A schematic diagram of a framework of a video event data processing system provided in an embodiment of the present application includes:

[0127] A first processing unit 810 is configured to obtain a first key frame and a second key frame in an original video stream, wherein the second key frame is a key frame subsequent to the first key frame;

[0128] The second processing unit 820 is used to perform frame picture change judgment on the first key frame according to the second key frame to obtain a change judgment result;

[0129] The third processing unit 830 is configured to cache data of the original video stream according to the first key frame to obtain a first intermediate video stream if the change judgment result is that the frame picture has changed, wherein the first key frame of the first intermediate video stream is the first key frame;

[0130] The fourth processing unit 840 is configured to perform video event recognition on the first intermediate video stream to obtain a third key frame, where the third key frame is a key frame that successfully recognizes an event type that causes a frame change in the first intermediate video stream;

[0131] a fifth processing unit 850, configured to perform video event termination recognition on the first intermediate video stream according to the third key frame and the first key frame, to obtain a fourth key frame and a second intermediate video stream, wherein the fourth key frame is a key frame that is after the third key frame in the first intermediate video stream and has the same frame picture as the first key frame;

[0132] The sixth processing unit 860 is used to perform event time correction on the second intermediate video stream according to the second key frame, the third key frame and the fourth key frame to obtain video event data.

[0133] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present application includes:

[0135] at least one processor 980;

[0136] At least one memory 920, used to store at least one program;

[0137] When the at least one program is executed by the at least one processor 980, the at least one processor 980 implements the methods described in the foregoing embodiments.

[0138] An embodiment of the present application also provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor 980, it is used to implement the methods described in the above embodiments.

[0139] Fig.10 Specifically, the computer device may be a user terminal or a server.

[0140] The present application embodiment takes the computer device as a user terminal as an example, and the details are as follows:

[0141] like Fig.10 As shown, the computer device 900 may include an RF (Radio Frequency) circuit 910, a memory 920 including one or more computer-readable storage media, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a WiFi module 970, a processor 980 including one or more processing cores, and a power supply 990. Those skilled in the art will appreciate that Fig.10The device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0142] The RF circuit 910 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is handed over to one or more processors 980 for processing; in addition, the data related to the uplink is sent to the base station. Generally, the RF circuit 910 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a user identity module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. In addition, the RF circuit 910 can also communicate with the network and other devices through wireless communication. Wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0143] The memory 920 can be used to store software programs and modules. The processor 980 executes various functional applications and data processing by running the software programs and modules stored in the memory 920. The memory 920 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device 900 (such as audio data, a phone book, etc.), etc. In addition, the memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 920 may also include a memory controller to provide the processor 980 and the input unit 930 with access to the memory 920. Although Fig.10 The RF circuit 910 is shown, but it is understandable that it is not an essential component of the device 900 and can be omitted as required without changing the essence of the invention.

[0144] The input unit 930 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 930 may include a touch-sensitive surface 932 and other input devices 931. The touch-sensitive surface 932, also known as a touch display screen or a touch pad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 932), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 932 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the touch-sensitive surface 932 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch-sensitive surface 932, the input unit 930 may further include other input devices 931. Specifically, the other input devices 931 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0145] The display unit 940 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the control 900, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 940 may include a display panel 941. Optionally, the display panel 941 may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 932 may be covered on the display panel 941. When the touch-sensitive surface 932 detects a touch operation on or near it, it is transmitted to the processor 980 to determine the type of touch event. Subsequently, the processor 980 provides a corresponding visual output on the display panel 941 according to the type of touch event. Although in Fig.10 In the embodiment, the touch-sensitive surface 932 and the display panel 941 are implemented as two independent components to implement input and output functions, but in some embodiments, the touch-sensitive surface 932 and the display panel 941 can be integrated to implement input and output functions.

[0146] The computer device 900 may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 941 and / or the backlight when the device 900 is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured in the device 900, they will not be repeated here.

[0147] The audio circuit 960, the speaker 961, and the microphone 962 can provide an audio interface between the user and the device 900. The audio circuit 960 can transmit the electrical signal converted from the received audio data to the speaker 961, which is converted into a sound signal for output; on the other hand, the microphone 962 converts the collected sound signal into an electrical signal, which is received by the audio circuit 960 and converted into audio data, and then the audio data is processed by the output processor 980 and sent to another control device through the RF circuit 910, or the audio data is output to the memory 920 for further processing. The audio circuit 960 may also include an earplug jack to provide communication between an external headset and the device 900.

[0148] The device 900 can transmit information with the wireless transmission module set on the fighting device through the WiFi module 970.

[0149] The processor 980 is the control center of the device 900. It uses various interfaces and lines to connect various parts of the entire control device. It executes various functions of the device 900 and processes data by running or executing software programs and / or modules stored in the memory 920, and calling data stored in the memory 920, so as to monitor the control device as a whole. Optionally, the processor 980 may include one or more processing cores; optionally, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 950.

[0150] The device 900 also includes a power supply 990 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 980 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 990 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0151] Although not shown, the device 900 may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0152] An embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the method described in the above embodiments.

[0153] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0154] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0159] The step numbers in the above method embodiment are only provided for the convenience of explanation and description, and no limitation is imposed on the order of the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0160] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the described embodiments. Technical personnel familiar with the field may make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for processing video event data, characterized in that: include: Acquire a first key frame and a second key frame in an original video stream, wherein the second key frame is a key frame subsequent to the first key frame; According to the second key frame, performing frame picture change judgment on the first key frame to obtain a change judgment result; If the change judgment result is that the frame image has changed, data of the original video stream is cached according to the first key frame to obtain a first intermediate video stream, and the first key frame of the first intermediate video stream is the first key frame; Performing video event recognition on the first intermediate video stream to obtain a third key frame, wherein the third key frame is a key frame that successfully recognizes the event type causing frame image change in the first intermediate video stream; According to the third key frame and the first key frame, video event termination recognition is performed on the first intermediate video stream to obtain a fourth key frame and a second intermediate video stream, wherein the fourth key frame is a key frame that is after the third key frame in the first intermediate video stream and has the same frame picture as the first key frame; According to the second key frame, the third key frame and the fourth key frame, event time correction is performed on the second intermediate video stream to obtain video event data.

2. The processing method according to claim 1, characterized in that: The step of performing frame picture change judgment on the first key frame according to the second key frame to obtain a change judgment result includes: Obtaining a preset first threshold value; According to the second key frame, performing frame picture difference calculation on the first key frame to obtain a frame picture difference value; The first threshold value and the frame picture difference value are compared to obtain the change judgment result.

3. The processing method according to claim 1, characterized in that: The performing video event recognition on the first intermediate video stream to obtain a third key frame includes: Obtaining a current video frame in the first intermediate video stream; Performing event type recognition on the current video frame to obtain an event type recognition result; If the event type recognition result is that the event type recognition is unsuccessful, the current video frame is updated, and then the step of obtaining the current video frame in the first intermediate video stream is returned; or, if the event type recognition result is that the event type recognition is successful, the current video frame is determined as the third key frame.

4. The processing method according to claim 3, characterized in that: The method further comprises: According to the current video frame, performing a first frame picture comparison on the first key frame to obtain a first frame picture comparison result; If the first frame comparison result is that the frames are consistent, the current video frame is determined as the fifth key frame, and the first intermediate video stream is cleared according to the fifth key frame; or, if the first frame comparison result is that the frames are inconsistent, the process returns to the step of performing event type recognition on the current video frame to obtain an event type recognition result; The fifth key frame is a key frame having the same frame picture as the first key frame in the first intermediate video stream before the key frame that successfully identifies the event type causing the frame picture change in the first intermediate video stream.

5. The processing method according to claim 1, characterized in that: The performing video event termination identification on the first intermediate video stream according to the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream comprises: According to the third key frame, an intermediate video frame in the first intermediate video stream is obtained, where the intermediate video frame is a key frame after the third key frame; According to the intermediate video frame, performing a second frame picture comparison on the first key frame to obtain a second frame picture comparison result; If the second frame picture comparison result is that the frames are consistent, the intermediate video frame is determined as the fourth key frame, and according to the fourth key frame, data of the first intermediate video stream is intercepted to obtain the second intermediate video stream; or, if the second frame picture comparison result is that the frames are inconsistent, the intermediate video frame is updated, and then the step of obtaining the intermediate video frame in the first intermediate video stream according to the third key frame is returned.

6. The processing method according to claim 5, characterized in that: The step of performing a second frame picture comparison on the first key frame according to the intermediate video frame to obtain a second frame picture comparison result includes: Generate a second threshold value according to the first threshold value, wherein the second threshold value is smaller than the first threshold value; According to the intermediate video frame, performing frame picture similarity calculation on the second key frame to obtain frame picture similarity; The second threshold value is compared with the frame picture similarity to obtain the second frame picture comparison result.

7. The processing method according to claim 1, characterized in that: The performing event time correction on the second intermediate video stream according to the second key frame, the third key frame and the fourth key frame to obtain video event data includes: Obtain a first threshold value and a second threshold value; According to the first threshold value and the second threshold value, performing event time calibration processing on the timestamp information of the second key frame, the timestamp information of the third key frame, and the timestamp information of the fourth key frame to obtain an event calibration time; According to the event calibration time, the second intermediate video stream is calibrated and corrected to obtain the video event data.

8. A video event data processing system, characterized in that: include: A first processing unit is used to obtain a first key frame and a second key frame in an original video stream, where the second key frame is a key frame subsequent to the first key frame; A second processing unit is used to perform frame picture change judgment on the first key frame according to the second key frame to obtain a change judgment result; A third processing unit is configured to cache data of the original video stream according to the first key frame to obtain a first intermediate video stream if the change judgment result is that the frame picture has changed, wherein the first key frame of the first intermediate video stream is the first key frame; a fourth processing unit, configured to perform video event recognition on the first intermediate video stream to obtain a third key frame, wherein the third key frame is a key frame that successfully recognizes an event type that causes a frame image change in the first intermediate video stream; a fifth processing unit, configured to perform video event termination recognition on the first intermediate video stream according to the third key frame and the first key frame, to obtain a fourth key frame and a second intermediate video stream, wherein the fourth key frame is a key frame that is after the third key frame in the first intermediate video stream and has the same frame picture as the first key frame; The sixth processing unit is used to perform event time correction on the second intermediate video stream according to the second key frame, the third key frame and the fourth key frame to obtain video event data.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images