A method, system, device, and medium for processing video event data.

Abstract

This application discloses a method, system, device, and medium for processing video event data. The method involves determining frame changes in a first keyframe based on a second keyframe to obtain a change determination result. If the change determination result indicates a frame change, the original video stream is buffered based on the first keyframe to obtain a first intermediate video stream. Video event recognition is performed on the first intermediate video stream to obtain a third keyframe. Based on the third and first keyframes, video event termination recognition is performed on the first intermediate video stream to obtain a fourth keyframe and a second intermediate video stream. Event time correction is performed on the second intermediate video stream based on the second, third, and fourth keyframes to obtain video event data. This method can improve the accuracy of event occurrence time determination and enhance the completeness of event content in the video event data. This application relates to the field of data processing technology.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, system, device and medium for processing video event data. Background Technology

[0002] Video surveillance systems are systems that use cameras and other equipment to record video of a specific area in real time or at set intervals. They have gained attention because they can analyze and identify video surveillance data using intelligent AI.

[0003] Currently, existing technologies typically use AI to analyze and identify video surveillance data. When the AI ​​identifies an event, it retains a few frames of video surveillance images as video event data for storage. This method has poor accuracy in determining the time of the event, and the completeness of the event content in the retained video event data is unsatisfactory.

[0004] Therefore, the problems existing in the current technology still need to be solved and optimized. Summary of the Invention

[0005] To address at least one of the aforementioned technical problems, this application provides a method, system, device, and medium for processing video event data. The processing method can improve the accuracy of determining the occurrence time of events and enhance the completeness of the event content in the video event data.

[0006] According to a first aspect of this application, a method for processing video event data is provided, comprising:

[0007] Obtain the first keyframe and the second keyframe from the original video stream, where the second keyframe is the keyframe following the first keyframe.

[0008] Based on the second keyframe, the first keyframe is subjected to frame change judgment to obtain the change judgment result;

[0009] If the change determination result is that the frame has changed, then the original video stream is cached according to the first key frame to obtain the first intermediate video stream, and the first key frame of the first intermediate video stream is the first key frame.

[0010] Video event recognition is performed on the first intermediate video stream to obtain a third key frame. The third key frame is a key frame that successfully identifies the type of event in the first intermediate video stream that causes changes in the frame.

[0011] Based on the third keyframe and the first keyframe, video event termination recognition is performed on the first intermediate video stream to obtain the fourth keyframe and the second intermediate video stream. The fourth keyframe is a keyframe that is the same as the frame of the first keyframe after the third keyframe in the first intermediate video stream.

[0012] Based on the second keyframe, the third keyframe, and the fourth keyframe, the second intermediate video stream is modified for event timing to obtain video event data.

[0013] In some embodiments, the step of determining frame changes in the first key frame based on the second key frame to obtain a change determination result includes:

[0014] Obtain the preset first threshold value;

[0015] Based on the second keyframe, the frame difference is calculated for the first keyframe to obtain the frame difference value;

[0016] The change judgment result is obtained by comparing the first threshold value with the frame difference value.

[0017] In some embodiments, the step of performing video event recognition on the first intermediate video stream to obtain a third keyframe includes:

[0018] Obtain the current video frame in the first intermediate video stream;

[0019] Perform event type identification on the current video frame to obtain the event type identification result;

[0020] If the event type identification result is that the event type identification is unsuccessful, then the current video frame is updated, and then the process returns to the step of obtaining the current video frame in the first intermediate video stream; or, if the event type identification result is that the event type identification is successful, then the current video frame is determined as the third keyframe.

[0021] In some embodiments, the method further comprises:

[0022] Based on the current video frame, perform a first frame comparison on the first key frame to obtain the first frame comparison result;

[0023] If the comparison result of the first frame is that the frames are consistent, then the current video frame is determined as the fifth key frame, and the first intermediate video stream is cleared according to the fifth key frame; or, if the comparison result of the first frame is that the frames are inconsistent, then the process returns to the step of performing event type identification on the current video frame to obtain the event type identification result.

[0024] The fifth keyframe is a keyframe in the first intermediate video stream that has the same frame as the first keyframe before the keyframe that successfully identifies the event type that causes the frame change in the first intermediate video stream.

[0025] In some embodiments, the step of performing video event termination recognition on the first intermediate video stream based on the third keyframe and the first keyframe to obtain the fourth keyframe and the second intermediate video stream includes:

[0026] Based on the third keyframe, an intermediate video frame is obtained from the first intermediate video stream, wherein the intermediate video frame is a keyframe following the third keyframe.

[0027] Based on the intermediate video frames, the first key frame is compared with the second frame to obtain the second frame comparison result.

[0028] If the comparison result of the second frame is that the frames are consistent, then the intermediate video frame is determined as the fourth key frame, and data is extracted from the first intermediate video stream according to the fourth key frame to obtain the second intermediate video stream; or, if the comparison result of the second frame is that the frames are inconsistent, then the intermediate video frame is updated, and then the process returns to the step of obtaining the intermediate video frame in the first intermediate video stream according to the third key frame.

[0029] In some embodiments, the step of performing a second-frame comparison on the first keyframe based on the intermediate video frame to obtain a second-frame comparison result includes:

[0030] A second threshold is generated based on the first threshold, and the second threshold is less than the first threshold.

[0031] Based on the intermediate video frames, perform frame similarity calculations on the second key frame to obtain the frame similarity score.

[0032] The second threshold and the frame similarity are compared to obtain the second frame comparison result.

[0033] In some embodiments, the step of performing event timing correction on the second intermediate video stream based on the second keyframe, the third keyframe, and the fourth keyframe to obtain video event data includes:

[0034] Obtain the first threshold and the second threshold;

[0035] Based on the first threshold and the second threshold, the timestamp information of the second key frame, the timestamp information of the third key frame, and the timestamp information of the fourth key frame are subjected to event time calibration processing to obtain the event calibration time;

[0036] The second intermediate video stream is calibrated and corrected according to the event calibration time to obtain the video event data.

[0037] According to a second aspect of this application, a video event data processing system is provided, comprising:

[0038] The first processing unit is used to acquire the first keyframe and the second keyframe in the original video stream, wherein the second keyframe is the keyframe following the first keyframe.

[0039] The second processing unit is used to perform frame change judgment on the first key frame based on the second key frame and obtain the change judgment result.

[0040] The third processing unit is used to perform data caching on the original video stream according to the first key frame if the change judgment result is that the frame image has changed, so as to obtain a first intermediate video stream, wherein the first key frame of the first intermediate video stream is the first key frame.

[0041] The fourth processing unit is used to perform video event recognition on the first intermediate video stream to obtain a third key frame. The third key frame is a key frame that successfully identifies the event type that causes frame changes in the first intermediate video stream.

[0042] The fifth processing unit is used to perform video event termination recognition on the first intermediate video stream based on the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream. The fourth key frame is a key frame that is the same as the frame of the first key frame after the third key frame in the first intermediate video stream.

[0043] The sixth processing unit is used to perform event timing correction on the second intermediate video stream based on the second keyframe, the third keyframe, and the fourth keyframe to obtain video event data.

[0044] According to a third aspect of this application, a computer device is provided, comprising:

[0045] At least one processor;

[0046] At least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any of the above aspects.

[0048] According to a fourth aspect of this application, a computer-readable storage medium is provided, wherein a processor-executable program is stored, which, when executed by the processor, is used to implement the method as described in any of the preceding aspects.

[0049] The beneficial effects of the technical solutions provided in this application are:

[0050] This application provides a method, system, device, and medium for processing video event data. The method involves acquiring a first keyframe and a second keyframe from an original video stream, where the second keyframe is the keyframe following the first keyframe. Based on the second keyframe, a frame change judgment is performed on the first keyframe to obtain a change judgment result. If the change judgment result indicates a frame change, the original video stream is buffered based on the first keyframe to obtain a first intermediate video stream, where the first keyframe of the first intermediate video stream is the first keyframe. Video event recognition is performed on the first intermediate video stream to obtain a third keyframe, where the third keyframe is the keyframe that successfully identifies the event type causing the frame change in the first intermediate video stream. Based on the third keyframe and the first keyframe, video event termination recognition is performed on the first intermediate video stream to obtain a fourth keyframe and a second intermediate video stream, where the fourth keyframe is the keyframe following the third keyframe in the first intermediate video stream and has the same frame as the first keyframe. Event time correction is performed on the second intermediate video stream based on the second keyframe, the third keyframe, and the fourth keyframe to obtain video event data. This method uses the third and first keyframes to perform video event termination recognition on the first intermediate video stream, resulting in a second intermediate video stream with higher event content completeness. In addition, the method also uses the second, third, and fourth keyframes to correct the event occurrence time of the second intermediate video stream, which can effectively improve the accuracy of event occurrence time determination while ensuring the completeness of event content. Attached Figure Description

[0051] Figure 1 This application provides a flowchart of a method for processing video event data.

[0052] Figure 2 A detailed flowchart of step S120 is provided for an embodiment of this application;

[0053] Figure 3 A detailed flowchart of step S140 provided for an embodiment of this application;

[0054] Figure 4 This is a schematic flowchart of one optional method for processing video event data provided in an embodiment of this application;

[0055] Figure 5 A logical diagram of step S140 provided for an embodiment of this application;

[0056] Figure 6 A detailed flowchart of step S150 provided for an embodiment of this application;

[0057] Figure 7 A detailed flowchart of step C2 provided for an embodiment of this application;

[0058] Figure 8 A detailed flowchart of step S160 provided for an embodiment of this application;

[0059] Figure 9 A schematic diagram of the framework of a video event data processing system provided in this application embodiment;

[0060] Figure 10 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0061] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0062] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0064] Currently, existing technologies typically use AI to analyze and identify video surveillance data. When the AI ​​identifies an event, it retains a few frames of video surveillance images as video event data. This method usually determines the time when the AI ​​identifies the event type as the event occurrence time and retains the adjacent frames as video event data. However, because AI identification of the specific event type requires processing time, and because there are cases where the changes in adjacent frames of video surveillance data are relatively small when an event occurs, this method cannot accurately determine the event occurrence time. Its accuracy in determining the event occurrence time is poor, and the completeness of the event content in the retained video event data is unsatisfactory.

[0065] In view of this, embodiments of this application provide a method, system, device, and medium for processing video event data. The processing method performs video event termination recognition on a first intermediate video stream based on a third keyframe and a first keyframe, which can obtain a second intermediate video stream with higher event content completeness. In addition, the method also corrects the event occurrence time of the second intermediate video stream based on a second keyframe, a third keyframe, and a fourth keyframe, which can effectively improve the accuracy of determining the event occurrence time while ensuring the completeness of the event content.

[0066] This application provides a method, system, device, and medium for processing video event data, which can be specifically described through the following embodiments. First, a method for processing video event data in this application embodiment is described.

[0067] The video event data processing method provided in this application can be applied to video surveillance application scenarios. In video surveillance application scenarios, video surveillance service providers can use the processing method provided in this application to analyze and identify events in video surveillance data, which can improve the completeness of the retained video event data and effectively improve the accuracy of determining the time of event occurrence.

[0068] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0069] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0070] Reference Figure 1 , Figure 1 This is an optional flowchart of a video event data processing method provided in the embodiments of this application, which may include, but is not limited to, steps S110 to S160.

[0071] Step S110: Obtain the first keyframe and the second keyframe in the original video stream, wherein the second keyframe is the keyframe following the first keyframe.

[0072] In this embodiment of the application, the original video stream can be video surveillance data acquired in real time by a camera or other video recording device. The first keyframe and the second keyframe can be two adjacent frame images in the original video stream. The first keyframe is the previous frame image among the two adjacent frame images, and the second keyframe is the next frame image among the two adjacent frame images.

[0073] Step S120: Based on the second keyframe, perform frame change judgment on the first keyframe to obtain the change judgment result;

[0074] In this embodiment, the frame change determination in step S120 is used to determine whether a frame change has occurred in the second key frame compared to the first key frame, thereby obtaining a change determination result. This change determination result is used to characterize whether a frame change has occurred between the first key frame and the second key frame. Furthermore, the event causing the frame change can be at least one of a person, animal, vehicle, object, etc., that intrudes into the frame.

[0075] Reference Figure 2 In some embodiments, step S120, determining frame changes in the first keyframe based on the second keyframe to obtain a change determination result, includes:

[0076] A1. Obtain the preset first threshold value;

[0077] A2. Based on the second keyframe, perform frame difference calculation on the first keyframe to obtain the frame difference value;

[0078] A3. Compare the first threshold value with the frame difference value to obtain the change judgment result.

[0079] In this embodiment, a preset first threshold can first be obtained, which is a threshold for the difference between the frames of the first keyframe and the second keyframe. The specific value of the first threshold can be set according to the actual situation. Step A2 can be to calculate the difference between the frame of the second keyframe and the frame of the first keyframe. Specifically, the frame difference value can be determined based on mean squared error (MSE), structural similarity index (SSIM), mutual information (MI), etc.

[0080] It is understandable that step A3 can be a comparison between the first threshold and the frame difference value to obtain a change judgment result. Specifically, if the frame difference value is determined based on the mean squared error (MSE), the larger the frame difference value, the greater the difference between the first keyframe and the second keyframe; conversely, the smaller the frame difference value, the smaller the difference between the first keyframe and the second keyframe. In this case, when the frame difference value is greater than the first threshold, a change judgment result indicating that the frame has changed can be obtained; or, when the frame difference value is less than or equal to the first threshold, a change judgment result indicating that the frame has not changed can be obtained. The same logic applies to the frame difference values ​​obtained in other methods, and can be easily deduced by analogy. This application will not elaborate further here.

[0081] Step S130: If the change judgment result is that the frame has changed, then according to the first key frame, the original video stream is cached to obtain the first intermediate video stream, and the first key frame of the first intermediate video stream is the first key frame.

[0082] In this embodiment of the application, if the change determination result is that the frame image has changed, the original video stream can be cached based on the timestamp information of the first key frame to obtain the first intermediate video stream. The first intermediate video stream includes the first key frame and the frame images after the first key frame, while retaining the timestamp information of the second key frame.

[0083] Understandably, if the change judgment result is that the frame image has not changed, the current first keyframe and second keyframe can be updated. Specifically, the current second keyframe can be used as the first keyframe in the next frame image change judgment process, and the next keyframe of the current second keyframe can be used as the second keyframe in the next frame image change judgment process, and then return to step S110; or, the next keyframe of the current second keyframe can be used as the first keyframe in the next frame image change judgment process, and then return to step S110.

[0084] Step S140: Perform video event recognition on the first intermediate video stream to obtain a third key frame. The third key frame is a key frame that successfully identifies the event type that causes frame changes in the first intermediate video stream.

[0085] In this embodiment, the video event recognition in step S140 is used to identify the intrusion frame event type in the first intermediate video stream. Specifically, when an event such as a person, vehicle, or object intrudes into a frame, the frame of the event in the first intermediate video stream gradually becomes complete. Therefore, in this embodiment, step S140 can be performed frame-by-frame recognition of each frame image in the first intermediate video stream using intelligent AI. If the intelligent AI successfully identifies the intrusion event type in a certain frame image, then that frame image is determined as the third key frame. This third key frame is the key frame that successfully identifies the event type that causes the frame change in the first intermediate video stream.

[0086] Reference Figure 3 In some embodiments, step S140, performing video event recognition on the first intermediate video stream to obtain a third keyframe, includes:

[0087] B1. Obtain the current video frame in the first intermediate video stream;

[0088] B2. Perform event type identification on the current video frame to obtain the event type identification result;

[0089] B3. If the event type identification result is that the event type identification is unsuccessful, then update the current video frame and then return to the step of obtaining the current video frame in the first intermediate video stream;

[0090] Alternatively, B4, if the event type identification result is successful, then the current video frame is determined as the third keyframe.

[0091] In this embodiment, the current video frame can be the frame image currently being identified by the intelligent AI during frame-by-frame recognition. Specifically, for the recognition of a certain frame image by the intelligent AI, step B2 can be based on video AI recognition and analysis technology to extract the frame features of the current video frame, and match the extracted frame features with known event types in the database to obtain the event type recognition result. Specifically, if the current video frame matches a corresponding event type, the obtained event type recognition result can be "event type recognition successful"; or, if the current video frame does not match a corresponding event type, the obtained event type recognition result can be "event type recognition unsuccessful".

[0092] Understandably, if the event type identification result is unsuccessful, the current video frame can be updated. Specifically, the next frame of the current video frame can be determined as the current video frame in the next frame image identification process, and then the process can return to step B1. Alternatively, if the event type identification result is successful, the current video frame can be determined as the third keyframe.

[0093] Reference Figure 4 and Figure 5 Furthermore, the method also includes:

[0094] B11. Based on the current video frame, perform a first frame comparison on the first key frame to obtain the first frame comparison result.

[0095] B12. If the comparison result of the first frame is that the frames are consistent, then the current video frame is determined as the fifth key frame, and the first intermediate video stream is cleared according to the fifth key frame.

[0096] Alternatively, B13, if the comparison result of the first frame is that the frames are inconsistent, then return to the step of performing event type identification on the current video frame and obtaining the event type identification result;

[0097] The fifth keyframe is a keyframe in the first intermediate video stream that has the same frame as the first keyframe before the keyframe that successfully identifies the event type that causes the frame change in the first intermediate video stream.

[0098] In this embodiment of the application, after executing step B1 and before executing step B2, the frame of the first key frame can be compared with the frame of the current video frame to obtain the frame comparison result of the first frame. Then, if the frame comparison result of the first frame is consistent with the frame of the current video frame and the frame of the first key frame, it means that the monitoring screen of the first intermediate video stream has been restored before the intelligent AI successfully identifies the event type of the event. At this time, the current video frame can be identified as the fifth key frame, and then the cached first intermediate video stream can be cleared based on the timestamp information of the fifth key frame to reduce the number of invalid event reminders in the video surveillance system.

[0099] Understandably, if the comparison result of the first frame shows that the current video frame and the first key frame are inconsistent, it means that the monitoring screen of the first intermediate video stream has not been restored before the intelligent AI successfully identifies the event type. At this time, it can return to step B2 to realize the event type identification of the current video frame.

[0100] Step S150: Based on the third keyframe and the first keyframe, perform video event termination recognition on the first intermediate video stream to obtain a fourth keyframe and a second intermediate video stream. The fourth keyframe is a keyframe that is the same as the frame of the first keyframe after the third keyframe in the first intermediate video stream.

[0101] In this embodiment, the video event termination identification in step S150 is used to determine whether the event terminated after it occurred. Specifically, it can identify each frame in the first intermediate video stream frame by frame to obtain the fourth keyframe. Specifically, if the event terminated after it occurred, a keyframe with the same frame as the first keyframe can be found in the first intermediate video stream, and this found keyframe is identified as the fourth keyframe. Simultaneously, a second intermediate video stream can be determined based on the first and fourth keyframes. This second terminated video stream contains all video content data from the occurrence to the termination of the event.

[0102] Reference Figure 6 In some embodiments, step S150, which involves performing video event termination recognition on the first intermediate video stream based on the third keyframe and the first keyframe to obtain a fourth keyframe and a second intermediate video stream, includes:

[0103] C1. Based on the third keyframe, obtain the intermediate video frame in the first intermediate video stream, wherein the intermediate video frame is a keyframe after the third keyframe.

[0104] In this embodiment of the application, for the identification of a certain frame in the first intermediate video stream, step C1 may be to obtain an intermediate video frame in the first intermediate video stream, wherein the intermediate video frame is a key frame that corresponds to the frame identification process and is after the third key frame.

[0105] C2. Based on the intermediate video frames, perform a second frame comparison on the first key frame to obtain the second frame comparison result;

[0106] Reference Figure 7 Further, step C2, comparing the first keyframe with the intermediate video frame to obtain the second frame comparison result, includes:

[0107] C21. Generate a second threshold based on the first threshold, wherein the second threshold is less than the first threshold.

[0108] C22. Based on the intermediate video frames, perform frame similarity calculation on the second key frame to obtain the frame similarity.

[0109] C23. Compare the second threshold value with the frame similarity to obtain the second frame comparison result.

[0110] In this embodiment, step C21 can be either randomly generating a specific value less than the first threshold as the second threshold, or pre-setting a specific value less than the first threshold as the second threshold. Step C22 can be calculating the similarity between the intermediate video frame and the second key frame, specifically determined based on gray-level matching, histogram matching, deep learning-based image similarity evaluation, etc., to obtain the frame similarity.

[0111] It is understandable that step C23 can be a comparison between the second threshold and the frame similarity. If the frame similarity is greater than the second threshold, a second frame comparison result indicating frame consistency can be obtained; or, if the frame similarity is less than or equal to the second threshold, a second frame comparison result indicating frame inconsistency can be obtained. It is worth noting that the first and second frame comparisons in this embodiment are similar and can be easily deduced by analogy.

[0112] C3. If the comparison result of the second frame is that the frames are consistent, then the intermediate video frame is determined as the fourth key frame, and the data of the first intermediate video stream is extracted according to the fourth key frame to obtain the second intermediate video stream.

[0113] Alternatively, C4, if the comparison result of the second frame is that the frames are inconsistent, then the intermediate video frame is updated, and then the process returns to the step of obtaining the intermediate video frame in the first intermediate video stream based on the third keyframe.

[0114] In this embodiment of the application, if the comparison result of the second frame is that the frames are consistent, it means that after the intelligent AI successfully identifies the event type of the event, the monitoring screen of the first intermediate video stream has been restored. At this time, the intermediate video frame can be determined as the fourth key frame. Then, based on the timestamp information of the fourth key frame and the timestamp information of the first key frame, the cached first intermediate video stream is intercepted to intercept the second intermediate data stream that records all frame images between the first key frame and the fourth key frame.

[0115] It is understandable that if the comparison result of the second frame is inconsistent, it means that after the intelligent AI successfully identified the event type, the monitoring screen of the first intermediate video stream has not been restored. At this time, the intermediate video frame can be updated. Specifically, the next key frame of the intermediate video frame can be used as the intermediate video frame in the next second frame comparison process, and then return to the execution step C1.

[0116] Step S160: Based on the second keyframe, the third keyframe, and the fourth keyframe, perform event timing correction on the second intermediate video stream to obtain video event data.

[0117] In this embodiment of the application, after obtaining a second intermediate data stream that records all frame images between the first keyframe and the fourth keyframe, the second intermediate data stream records all video content data from the occurrence to the termination of the event. Based on the timestamp information of the second keyframe, the third keyframe, and the third keyframe, the non-event recognition-related video data in the buffer of the second intermediate data stream can be corrected and removed to obtain the corrected video event data.

[0118] Reference Figure 8 In some embodiments, step S160, which involves performing event timing correction on the second intermediate video stream based on the second keyframe, the third keyframe, and the fourth keyframe to obtain video event data, includes:

[0119] D1. Obtain the first threshold and the second threshold;

[0120] D2. Based on the first threshold and the second threshold, perform event time calibration processing on the timestamp information of the second key frame, the timestamp information of the third key frame, and the timestamp information of the fourth key frame to obtain the event calibration time.

[0121] D3. Based on the event calibration time, calibrate and correct the second intermediate video stream to obtain the video event data.

[0122] In this embodiment, the first threshold value in step D1 can be the first threshold value in step A1, and the second threshold value in step D1 can be the second threshold value in step C21. The event time calibration process in step D2 can be the calculation of the event calibration time of the second intermediate video stream, which can be determined by a preset weighted linear algorithm. Specifically, the equivalent expression for the event calibration time can be:

[0123] T = y1 × (t2 - t0) - y2 × (t1 - t0) + b

[0124] Where T is the event calibration time of the second intermediate video stream; y1 is the first threshold; t2 is the timestamp information of the fourth keyframe; t0 is the timestamp information of the second keyframe; y2 is the second threshold; t1 is the timestamp information of the third keyframe; and b is a preset bias term.

[0125] It is understandable that step D3 can be based on the acquired event calibration time to remove and correct the front-end non-event-recognized video frame images in the second intermediate video stream, thereby obtaining the corrected second intermediate video stream (i.e., video event data). Specifically, it can be to remove the video frame images of the second intermediate video stream before the event calibration time and retain the video frame images of the second intermediate video stream after the event calibration time.

[0126] Figure 9 A schematic diagram of the framework of a video event data processing system provided in this application embodiment includes:

[0127] The first processing unit 810 is used to acquire the first keyframe and the second keyframe in the original video stream, wherein the second keyframe is the keyframe following the first keyframe.

[0128] The second processing unit 820 is used to perform frame change judgment on the first key frame based on the second key frame and obtain the change judgment result.

[0129] The third processing unit 830 is used to perform data caching on the original video stream according to the first key frame if the change judgment result is that the frame image has changed, so as to obtain a first intermediate video stream, wherein the first key frame of the first intermediate video stream is the first key frame.

[0130] The fourth processing unit 840 is used to perform video event recognition on the first intermediate video stream to obtain a third key frame. The third key frame is a key frame that successfully identifies the event type that causes frame changes in the first intermediate video stream.

[0131] The fifth processing unit 850 is used to perform video event termination recognition on the first intermediate video stream based on the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream. The fourth key frame is a key frame that is the same as the frame of the first key frame after the third key frame in the first intermediate video stream.

[0132] The sixth processing unit 860 is used to perform event timing correction on the second intermediate video stream based on the second keyframe, the third keyframe and the fourth keyframe to obtain video event data.

[0133] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0134] Figure 10 A schematic diagram of the structure of a computer device provided in this application embodiment includes:

[0135] At least one processor 980;

[0136] At least one memory 920 is used to store at least one program;

[0137] When the at least one program is executed by the at least one processor 980, the at least one processor 980 performs the method as described in the foregoing embodiments.

[0138] This application also provides a computer-readable storage medium storing a processor-executable program, which, when executed by the processor 980, is used to implement the methods described in the foregoing embodiments.

[0139] Figure 10 Specifically, computer equipment can be either a user terminal or a server.

[0140] This application uses a computer device as a user terminal as an example, as detailed below:

[0141] like Figure 10 As shown, the computer device 900 may include an RF (Radio Frequency) circuit 910, a memory 920 including one or more computer-readable storage media, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a WiFi module 970, a processor 980 including one or more processing cores, and a power supply 990, among other components. Those skilled in the art will understand that... Figure 10The device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0142] The RF circuit 910 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and hands it over to one or more processors 980 for processing; additionally, it transmits uplink data to the base station. Typically, the RF circuit 910 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, an LNA (Low Noise Amplifier), a duplexer, etc. Furthermore, the RF circuit 910 can also communicate wirelessly with networks and other devices. Wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), LTE (Long Term Evolution), email, SMS (Short Messaging Service), etc.

[0143] The memory 920 can be used to store software programs and modules. The processor 980 executes various functional applications and data processing by running the software programs and modules stored in the memory 920. The memory 920 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the device 900 (such as audio data, telephone directory, etc.). In addition, the memory 920 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 920 may also include a memory controller to provide access to the memory 920 for the processor 980 and the input unit 930. Although Figure 10 The RF circuit 910 is shown, but it is understood that it is not a necessary component of the device 900 and can be omitted as needed without changing the nature of the invention.

[0144] The input unit 930 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 930 may include a touch-sensitive surface 932 and other input devices 931. The touch-sensitive surface 932, also known as a touch display screen or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 932), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 932 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 980, and can receive and execute commands from the processor 980. In addition, the touch-sensitive surface 932 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 932, the input unit 930 may also include other input devices 931. Specifically, other input devices 931 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0145] Display unit 940 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces for controlling 900. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 940 may include display panel 941, optionally configured as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, touch-sensitive surface 932 may cover display panel 941. When touch-sensitive surface 932 detects a touch operation on or near it, it transmits the information to processor 980 to determine the type of touch event. Subsequently, processor 980 provides corresponding visual output on display panel 941 according to the type of touch event. Although in Figure 10 In this embodiment, the touch-sensitive surface 932 and the display panel 941 are implemented as two separate components to realize input and output functions. However, in some embodiments, the touch-sensitive surface 932 and the display panel 941 can be integrated to realize input and output functions.

[0146] The computer device 900 may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 941 according to the ambient light level, and the proximity sensor can turn off the display panel 941 and / or backlight when the device 900 is moved to the ear. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. Other sensors that the device 900 may be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0147] Audio circuitry 960, speaker 961, and microphone 962 provide an audio interface between the user and device 900. Audio circuitry 960 converts received audio data into electrical signals, which are then transmitted to speaker 961, where they are converted into sound signals for output. Conversely, microphone 962 converts collected sound signals into electrical signals, which are received by audio circuitry 960, converted back into audio data, and then processed by processor 980 before being transmitted via RF circuitry 910 to another control device, or output to memory 920 for further processing. Audio circuitry 960 may also include an earphone jack to facilitate communication between peripheral headphones and device 900.

[0148] Device 900 can transmit information with the wireless transmission module set up on the battle equipment via WiFi module 970.

[0149] Processor 980 is the control center of device 900. It connects various parts of the control device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 920, and by calling data stored in memory 920, it performs various functions of device 900 and processes data, thereby providing overall monitoring of the control device. Optionally, processor 980 may include one or more processing cores; optionally, processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into processor 950.

[0150] The device 900 also includes a power supply 990 (such as a battery) that supplies power to the various components. Preferably, the power supply can be logically connected to the processor 980 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 990 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0151] Although not shown, device 900 may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0152] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the methods described in the foregoing embodiments.

[0153] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0154] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0158] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] The step numbers in the above method embodiments are set only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0160] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A method for processing video event data, characterized in that, include: Obtain the first keyframe and the second keyframe from the original video stream, where the second keyframe is the keyframe following the first keyframe. Based on the second keyframe, the first keyframe is subjected to frame change judgment to obtain the change judgment result; If the change determination result is that the frame has changed, then the original video stream is cached according to the first key frame to obtain the first intermediate video stream, and the first key frame of the first intermediate video stream is the first key frame. Video event recognition is performed on the first intermediate video stream to obtain a third key frame. The third key frame is a key frame that successfully identifies the type of event in the first intermediate video stream that causes changes in the frame. Based on the third keyframe and the first keyframe, video event termination recognition is performed on the first intermediate video stream to obtain the fourth keyframe and the second intermediate video stream. The fourth keyframe is a keyframe that is the same as the frame of the first keyframe after the third keyframe in the first intermediate video stream. Based on the second keyframe, the third keyframe, and the fourth keyframe, the second intermediate video stream is modified for event timing to obtain video event data.

2. The processing method according to claim 1, characterized in that, The step of determining frame changes in the first keyframe based on the second keyframe to obtain a change determination result includes: Obtain the preset first threshold value; Based on the second keyframe, the frame difference is calculated for the first keyframe to obtain the frame difference value; The change judgment result is obtained by comparing the first threshold value with the frame difference value.

3. The processing method according to claim 1, characterized in that, The step of performing video event recognition on the first intermediate video stream to obtain the third keyframe includes: Obtain the current video frame in the first intermediate video stream; Perform event type identification on the current video frame to obtain the event type identification result; If the event type identification result is that the event type identification is unsuccessful, then the current video frame is updated, and then the process returns to the step of obtaining the current video frame in the first intermediate video stream; or, if the event type identification result is that the event type identification is successful, then the current video frame is determined as the third keyframe.

4. The processing method according to claim 3, characterized in that, The method further includes: Based on the current video frame, perform a first frame comparison on the first key frame to obtain the first frame comparison result; If the comparison result of the first frame is that the frames are consistent, then the current video frame is determined as the fifth key frame, and the first intermediate video stream is cleared according to the fifth key frame; or, if the comparison result of the first frame is that the frames are inconsistent, then the process returns to the step of performing event type identification on the current video frame to obtain the event type identification result. The fifth keyframe is a keyframe in the first intermediate video stream that has the same frame as the first keyframe before the keyframe that successfully identifies the event type that causes the frame change in the first intermediate video stream.

5. The processing method according to claim 1, characterized in that, The step of performing video event termination recognition on the first intermediate video stream based on the third keyframe and the first keyframe to obtain the fourth keyframe and the second intermediate video stream includes: Based on the third keyframe, an intermediate video frame is obtained from the first intermediate video stream, wherein the intermediate video frame is a keyframe following the third keyframe. Based on the intermediate video frames, the first key frame is compared with the second frame to obtain the second frame comparison result. If the comparison result of the second frame is that the frames are consistent, then the intermediate video frame is determined as the fourth key frame, and data is extracted from the first intermediate video stream according to the fourth key frame to obtain the second intermediate video stream; or, if the comparison result of the second frame is that the frames are inconsistent, then the intermediate video frame is updated, and then the process returns to the step of obtaining the intermediate video frame in the first intermediate video stream according to the third key frame.

6. The processing method according to claim 5, characterized in that, The step of performing a second-frame comparison on the first keyframe based on the intermediate video frames to obtain the second-frame comparison result includes: A second threshold is generated based on the first threshold, and the second threshold is less than the first threshold. Based on the intermediate video frames, perform frame similarity calculations on the second key frame to obtain the frame similarity score. The second threshold and the frame similarity are compared to obtain the second frame comparison result.

7. The processing method according to claim 1, characterized in that, The step of performing event timing correction on the second intermediate video stream based on the second keyframe, the third keyframe, and the fourth keyframe to obtain video event data includes: Obtain the first threshold and the second threshold; Based on the first threshold and the second threshold, the timestamp information of the second key frame, the timestamp information of the third key frame, and the timestamp information of the fourth key frame are subjected to event time calibration processing to obtain the event calibration time; The second intermediate video stream is calibrated and corrected according to the event calibration time to obtain the video event data.

8. A video event data processing system, characterized in that, include: The first processing unit is used to acquire the first keyframe and the second keyframe in the original video stream, wherein the second keyframe is the keyframe following the first keyframe. The second processing unit is used to perform frame change judgment on the first key frame based on the second key frame and obtain the change judgment result. The third processing unit is used to perform data caching on the original video stream according to the first key frame if the change judgment result is that the frame image has changed, so as to obtain a first intermediate video stream, wherein the first key frame of the first intermediate video stream is the first key frame. The fourth processing unit is used to perform video event recognition on the first intermediate video stream to obtain a third key frame. The third key frame is a key frame that successfully identifies the event type that causes frame changes in the first intermediate video stream. The fifth processing unit is used to perform video event termination recognition on the first intermediate video stream based on the third key frame and the first key frame to obtain a fourth key frame and a second intermediate video stream. The fourth key frame is a key frame that is the same as the frame of the first key frame after the third key frame in the first intermediate video stream. The sixth processing unit is used to perform event timing correction on the second intermediate video stream based on the second keyframe, the third keyframe, and the fourth keyframe to obtain video event data.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images