Alarm video generation method and device, electronic equipment and storage medium

Abstract

The invention provides an alarm video generation method and device, electronic equipment and a storage medium, a streaming media system sends target streaming media data to a detection system, a first target frame of the target streaming media data corresponds to a first timestamp, and the data type of the first timestamp is a 32-bit unsigned integer; full recorded video data is generated, a second target frame of the full recorded video data corresponds to a second timestamp, and the data type of the second timestamp is a 64-bit unsigned integer; alarm frame information generated by the detection system on the basis of anomaly detection on the target streaming media data is received, and the alarm frame information comprises a third timestamp corresponding to a frame needing to be alarmed; the data type of the third timestamp is a 32-bit unsigned integer; converting the third timestamp to obtain a fourth timestamp, wherein the data type of the fourth timestamp is a 64-bit unsigned integer; and according to the fourth timestamp, determining a target alarm frame from the total recorded video data, and generating a target alarm video.

Description

Technical Field

[0001] The present invention relates to the technical field of warning video generation, and in particular to a warning video generation method, a warning video generation device, an electronic device and a computer-readable storage medium. Background Art

[0002] In intelligent monitoring systems, algorithmic capabilities are used to detect abnormal situations and trigger alarms when abnormalities are detected, generating alarm frames. Alarm frames usually contain alarm information and timestamps, which are used to notify the streaming media system to record abnormal events.

[0003] However, due to machine clock asynchrony and network delay, there is a deviation between the timestamp of the alarm frame and the actual recording time, resulting in a mismatch between the alarm video and the alarm frame, affecting the accuracy and reliability of the monitoring system. Summary of the invention

[0004] In view of the above problems, a method for generating a warning video, a device for generating a warning video, an electronic device and a computer-readable storage medium are proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] A method for generating an alarm video is applied to a streaming media system, wherein the streaming media system is connected to a detection system, and the method comprises:

[0006] The streaming media system sends target streaming media data to the detection system, the first target frame of the target streaming media data corresponds to a first timestamp, and the data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data;

[0007] Generate a full amount of recorded video data, wherein a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer;

[0008] Receiving the alarm frame information generated by the detection system based on the abnormality detection of the target streaming media data, wherein the alarm frame information includes a third timestamp corresponding to the frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer;

[0009] Convert the third timestamp to obtain a fourth timestamp, where the data type of the fourth timestamp is a 64-bit unsigned integer;

[0010] According to the fourth timestamp, a target alarm frame is determined from the full amount of recorded video data, and a target alarm video is generated according to the target alarm frame.

[0011] Optionally, the method further comprises:

[0012] When the first timestamp is about to exceed a 32-bit unsigned integer, a new cycle is started, and a first timestamp is assigned to the target streaming media data starting from 0.

[0013] Optionally, converting the third timestamp to obtain a fourth timestamp includes:

[0014] Determine the target number of cycles;

[0015] The fourth timestamp is determined according to the target cycle number and the third timestamp.

[0016] Optionally, determining the target number of cycles includes:

[0017] A target number of cycles is determined according to the current second timestamp and the third timestamp.

[0018] Optionally, determining the fourth timestamp according to the target number of cycles and the third timestamp includes:

[0019] Calculating the product of the target number of cycles and a preset value to obtain a first product value;

[0020] The fourth timestamp is determined according to the first product value and the third timestamp.

[0021] Optionally, the full amount of recorded video data includes a plurality of recorded video segments, and determining the target alarm frame from the full amount of recorded video data according to the fourth timestamp includes:

[0022] Determining a target recorded video segment from the plurality of recorded video segments according to the fourth timestamp;

[0023] And, according to the fourth timestamp, determining the target alarm frame from the target recorded video segment.

[0024] Optionally, generating a target warning video according to the target warning frame includes:

[0025] A target warning video of a target length is generated according to a plurality of frames before and after the target warning frame and the target warning frame.

[0026] The embodiment of the present invention further provides a device for generating an alarm video, which is applied to a streaming media system, wherein the streaming media system is connected to a detection system, and the device comprises:

[0027] A sending module, used for the streaming media system to send target streaming media data to the detection system, wherein a first target frame of the target streaming media data corresponds to a first timestamp, and a data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data;

[0028] A first generating module is used to generate a full amount of recorded video data, wherein a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer;

[0029] A receiving module, used to receive the alarm frame information generated by the detection system based on the abnormal detection of the target streaming media data, wherein the alarm frame information includes a third timestamp corresponding to the frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer;

[0030] A conversion module, configured to convert the third timestamp to obtain a fourth timestamp, wherein a data type of the fourth timestamp is a 64-bit unsigned integer;

[0031] The second generating module is used to determine a target alarm frame from the full amount of recorded video data according to the fourth timestamp, and generate a target alarm video according to the target alarm frame.

[0032] Optionally, the device further comprises:

[0033] The loop module is used to start a new round of loop when the first timestamp is about to exceed a 32-bit unsigned integer, and assign a first timestamp starting from 0 to the target streaming media data.

[0034] Optionally, the conversion module is used to determine a target number of cycles; and determine the fourth timestamp according to the target number of cycles and the third timestamp.

[0035] Optionally, the conversion module is used to determine a target number of cycles according to the current second timestamp and the third timestamp.

[0036] Optionally, the conversion module is used to calculate the product of the target number of cycles and a preset value to obtain a first product value; and determine the fourth timestamp according to the first product value and the third timestamp.

[0037] Optionally, the full amount of recorded video data includes multiple recorded video clips, and the second generation module is used to determine the target recorded video clip from the multiple recorded video clips according to the fourth timestamp; and, according to the fourth timestamp, determine the target alarm frame from the target recorded video clip.

[0038] Optionally, the second generating module is used to generate a target warning video of a target length based on a plurality of frames before and after the target warning frame and the target warning frame.

[0039] An embodiment of the present invention further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the above-mentioned method for generating an alarm video when executed by the processor.

[0040] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for generating an alarm video as described above is implemented.

[0041] The embodiments of the present invention have the following advantages:

[0042] In an embodiment of the present invention, a streaming media system sends target streaming media data to a detection system, a first target frame of the target streaming media data corresponds to a first timestamp, and a data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data; generate a full amount of recorded video data, a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer; receive alarm frame information generated by the detection system based on the anomaly detection of the target streaming media data, the alarm frame information includes a third timestamp corresponding to a frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer; convert the third timestamp to obtain a fourth timestamp, and the data type of the fourth timestamp is a 64-bit unsigned integer; determine a target alarm frame from the full amount of recorded video data according to the fourth timestamp, and generate a target alarm video according to the target alarm frame.

[0043] The embodiments of the present invention can solve the problem that the alarm video does not match the alarm frame due to machine clock asynchrony and network delay, and the deviation between the timestamp of the alarm frame and the actual recording time, thereby affecting the accuracy and reliability of the monitoring system.

[0044] The streaming media system also performs unified processing on the timestamps of the received RTP video streams to prevent timestamp duplication caused by abnormal timestamps. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0046] Figure 1 It is a flowchart of the steps of a method for generating an alarm video according to an embodiment of the present invention;

[0047] Figure 2 is a flowchart of another method for generating an alarm video according to an embodiment of the present invention;

[0048] Figure 3 This is an overall flow chart of generating an alarm video according to an embodiment of the present invention;

[0049] Figure 4 This is an overall interactive flow chart of generating an alarm video according to an embodiment of the present invention;

[0050] Figure 5 is a flow chart of obtaining a target alarm frame according to an embodiment of the present invention;

[0051] Figure 6 is a schematic diagram of an RTP packet header according to an embodiment of the present invention;

[0052] Figure 7 It is a structural schematic diagram of a device for generating an alarm video according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In order to solve the problem that the alarm video does not match the alarm frame due to the deviation between the timestamp of the alarm frame and the actual recording time due to machine clock asynchronization and network delay, thereby affecting the accuracy and reliability of the monitoring system, an embodiment of the present invention proposes a method for generating an alarm video, which embeds an accurate timestamp (PTS, Presentation) in the alarm frame, and uses the timestamp to retrieve the corresponding video slice in the streaming media system to generate an alarm short video that accurately matches the time of the alarm frame.

[0055] For details, please refer to Figure 1 , showing a flowchart of the steps of a method for generating an alarm video according to an embodiment of the present invention.

[0056] like Figure 1 As shown, the method for generating the warning video may include the following steps:

[0057] Step 101: The streaming media system sends target streaming media data to the detection system. The first target frame of the target streaming media data corresponds to a first timestamp, and the data type of the first timestamp is a 32-bit unsigned integer. The detection system is used to perform anomaly detection on the target streaming media data.

[0058] In the embodiment of the present invention, the method for generating an alarm video can be applied to a streaming media system, which is a special network service system that allows users to receive and play audio, video or other multimedia data streams in real time through the network. The core technology of the streaming media system is streaming transmission, that is, data can be continuously transmitted to the client like running water, and is often used to transmit camera data or captured video images to the user side according to business needs.

[0059] Exemplarily, the streaming media system can be connected to the detection system, and the streaming media system can transmit the streaming media data to the detection system; the detection system can perform anomaly detection on the streaming media data, specifically, a preset time can be set; when the detection system detects a preset event in the streaming media data, it can notify the streaming media system to generate an alarm video.

[0060] Specifically, the streaming media system may send target streaming media data to the detection system; in practical applications, the streaming media system may send target streaming media data to the detection system based on RTP (Real-time Transport Protocol).

[0061] RTP is a network transmission protocol widely used for real-time transmission of audio, video and other media data. In real-time applications, it ensures that audio and video data can reach the terminal in a suitable order and rate. The real physical time that the RTP timestamp can represent is 13 hours; therefore, the data type of the first timestamp of the first target frame in the target streaming media data is a 32-bit unsigned integer.

[0062] The first target frame may be every frame in the target streaming media data, or some frames may be selected as the first target frames to set the first timestamp according to a preset rule, which is not limited in the embodiment of the present invention.

[0063] Exemplarily, when the streaming media system starts to send a video stream to the detection system, it generates an initial timestamp. This initial timestamp can be any value, but is usually a large random number to avoid conflicts with other streams.

[0064] The timestamp of each subsequent frame is calculated based on the timestamp of the previous frame plus a fixed increment. This increment depends on the frame rate of the video. For example, if the unit of the timestamp is 1 / 90000, then the increment is 1 / 90000 seconds, which is not limited in the embodiment of the present invention.

[0065] Step 102: Generate full recorded video data, wherein a second target frame of the full recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer.

[0066] On the other hand, the streaming media system can also record and generate a full amount of recorded video data, that is, all video data.

[0067] Among them, the second target frame of the full amount of recorded video data corresponds to a second timestamp; the second target frame can be each frame in the full amount of recorded video data, or some frames can be selected as second target frames to set the second timestamp according to preset rules, and the embodiment of the present invention is not limited to this.

[0068] In practical applications, in order to reduce the number of timestamp cycles in the streaming media system and avoid time duplication, the streaming media system can use uint64 to record the internal timestamp and start from 0. Specifically, the data type of the second timestamp of the full amount of recorded video data is a 64-bit unsigned integer. Compared with a 32-bit unsigned integer that can only represent 13 hours, a 64-bit unsigned integer can effectively reduce the number of cycles.

[0069] Exemplarily, the unit of the first timestamp and the second timestamp is 1 / 90000 seconds, and the maximum timestamp of a 32-bit unsigned integer corresponds to approximately 47721 seconds. The maximum timestamp of a 64-bit unsigned integer corresponds to approximately 204963823041 seconds, which is approximately 6500 years.

[0070] Step 103: Receive alarm frame information generated by the detection system based on abnormal detection of target streaming media data, where the alarm frame information includes a third timestamp corresponding to the frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer.

[0071] In actual applications, after receiving the target streaming media data, the detection system can perform anomaly detection on the target streaming media data, and when an anomaly is detected, generate alarm frame information for the frame corresponding to the anomaly. The alarm frame information may include a third timestamp corresponding to the frame that needs an alarm; since the alarm frame is determined from the frame in the target streaming media data, the data type of the third timestamp of the frame that needs an alarm is also a 32-bit unsigned integer.

[0072] After generating the alarm frame information, the detection system may send the alarm frame information to the streaming media system.

[0073] Step 104: Convert the third timestamp to obtain a fourth timestamp, where the data type of the fourth timestamp is a 64-bit unsigned integer.

[0074] In some feasible embodiments, after receiving the alarm frame information, the streaming media system can obtain a third timestamp therefrom, and convert the third timestamp to obtain a fourth timestamp whose data type is a 64-bit unsigned integer, so as to unify the data type of the time dimension.

[0075] Step 105: According to the fourth timestamp, determine the target alarm frame from the full amount of recorded video data, and generate the target alarm video according to the target alarm frame.

[0076] After determining the fourth timestamp of the alarm frame, the streaming media system may determine the target alarm frame from the full amount of recorded video data according to the fourth timestamp, and generate a target alarm video including the target alarm frame based on the target alarm frame.

[0077] In an embodiment of the present invention, a streaming media system sends target streaming media data to a detection system, a first target frame of the target streaming media data corresponds to a first timestamp, and a data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data; generate a full amount of recorded video data, a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer; receive alarm frame information generated by the detection system based on the anomaly detection of the target streaming media data, the alarm frame information includes a third timestamp corresponding to a frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer; convert the third timestamp to obtain a fourth timestamp, and the data type of the fourth timestamp is a 64-bit unsigned integer; determine a target alarm frame from the full amount of recorded video data according to the fourth timestamp, and generate a target alarm video according to the target alarm frame.

[0078] The embodiments of the present invention can solve the problem that the alarm video does not match the alarm frame due to machine clock asynchrony and network delay, and the deviation between the timestamp of the alarm frame and the actual recording time, thereby affecting the accuracy and reliability of the monitoring system.

[0079] The streaming media system also performs unified processing on the timestamps of the received RTP video streams to prevent timestamp duplication caused by abnormal timestamps.

[0080] Reference Figure 2 , shows a flowchart of another method for generating an alarm video according to an embodiment of the present invention, which may include the following steps:

[0081] Step 201: The streaming media system sends target streaming media data to the detection system.

[0082] The streaming media system can send target streaming media data to the detection system; in actual applications, the streaming media system can send target streaming media data to the detection system based on RTP.

[0083] In an embodiment of the present invention, the above method may further include the following steps:

[0084] When the first timestamp is about to exceed the 32-bit unsigned integer, a new cycle is started, and the first timestamp is assigned to the target streaming media data starting from 0.

[0085] In some feasible embodiments, the real physical time that can be represented by the RTP timestamp is 13 hours, and a cycle will occur if it exceeds 13 hours. In view of this, when the first timestamp is about to exceed the 32-bit unsigned integer, a new cycle can be restarted, and the first timestamp is assigned from 0 to the new frame of the target streaming media data.

[0086] Step 202: Generate full recorded video data, wherein a second target frame of the full recorded video data corresponds to a second timestamp.

[0087] On the other hand, the streaming media system can also record and generate a full amount of recorded video data, that is, all video data.

[0088] Among them, the second target frame of the full amount of recorded video data corresponds to a second timestamp; the second target frame can be each frame in the full amount of recorded video data, or some frames can be selected as second target frames to set the second timestamp according to preset rules, and the embodiment of the present invention is not limited to this.

[0089] In practical applications, in order to reduce the number of timestamp cycles in the streaming media system and thus avoid the problem of time duplication, the streaming media system can record the internal timestamp using uint64 and start from 0. Specifically, the data type of the second timestamp of the full amount of recorded video data is a 64-bit unsigned integer.

[0090] Step 203: Receive the alarm frame information generated by the detection system based on the abnormality detection of the target streaming media data.

[0091] In actual applications, after receiving the target streaming media data, the detection system can perform anomaly detection on the target streaming media data, and when an anomaly is detected, generate alarm frame information for the frame corresponding to the anomaly. The alarm frame information may include a third timestamp corresponding to the frame that needs an alarm; since the alarm frame is determined from the frame in the target streaming media data, the data type of the third timestamp of the frame that needs an alarm is also a 32-bit unsigned integer.

[0092] After generating the alarm frame information, the detection system may send the alarm frame information to the streaming media system.

[0093] Step 204: Determine the target number of cycles.

[0094] In practical applications, after receiving the alarm frame information, the target number of cycles of the current cycle may be determined first.

[0095] In one embodiment of the present invention, the target number of cycles may be determined by:

[0096] The target number of cycles is determined according to the current second timestamp and the third timestamp.

[0097] In some feasible embodiments, the target number of loops may be calculated based on the second timestamp of the current record of the full amount of recorded video data and the third timestamp.

[0098] Exemplarily, the second timestamp of the current record of the full amount of recorded video data may be divided by the third timestamp and rounded down to obtain the number of times the current RTP stream timestamp is cycled, ie, the target number of cycles.

[0099] Step 205: Determine a fourth timestamp according to the target cycle number and the third timestamp.

[0100] In some feasible embodiments, based on the target cycle number and the third timestamp, a fourth timestamp in a data format of a 64-bit unsigned integer may be calculated.

[0101] In an embodiment of the present invention, the fourth timestamp may be determined in the following manner, including:

[0102] The product of the target number of cycles and a preset value is calculated to obtain a first product value; and a fourth timestamp is determined according to the first product value and the third timestamp.

[0103] In some feasible embodiments, the product of the target number of cycles and a preset value may be calculated first to obtain a first product value; wherein the first product value is 47721.

[0104] Then, a fourth timestamp whose data format is determined to be a 64-bit unsigned integer may be calculated according to the first product value and the third timestamp.

[0105] Step 206: The full amount of recorded video data includes multiple recorded video segments. According to the fourth timestamp, a target recorded video segment is determined from the multiple recorded video segments.

[0106] In some feasible embodiments, the full amount of recorded video data may include multiple recorded video segments; after determining the fourth timestamp, the streaming media system may first determine the target recorded video segment from the multiple recorded video segments according to the fourth timestamp.

[0107] Exemplarily, each recorded video clip is provided with a timestamp of its starting frame, and the data format is a 64-bit unsigned integer.

[0108] After obtaining the fourth timestamp, it may be first detected between which two start frames the fourth timestamp is located; then, it may be determined to use the recorded video segment corresponding to the start frame with a smaller timestamp as the target recorded video segment.

[0109] Step 207: Determine a target alarm frame from the target recorded video segment according to the fourth timestamp.

[0110] After the target recorded video segment is determined, a target alarm frame may be determined from multiple frames of the target recorded video segment based on the fourth timestamp.

[0111] Specifically, after determining the target recorded video segment, the offset value of the frame requiring an alarm in the target recorded video segment can be determined based on the fourth timestamp and the timestamp of the starting frame of the target recorded video segment, and further based on the offset value, the frame at the position corresponding to the offset value can be determined from the target recorded video segment as the target alarm frame, that is, the frame requiring an alarm.

[0112] Step 208: Generate a target warning video of a target length according to the target warning frame and the multiple frames before and after the target warning frame.

[0113] After determining the target alarm frame, a target alarm video of a target length can be generated based on multiple frames before and after the target alarm frame and the target frame. The target length can be any length; specifically, the target length can be set according to actual business requirements, and the embodiment of the present invention does not limit this.

[0114] For example, Figure 3 As shown, the streaming media system receives the RTP stream to provide live streaming services; on the one hand, it can send the RTP stream to the AI ​​recognition / detection service of the detection system, and on the other hand, it can generate a full amount of recorded video data and report the full amount of recorded video data to the recording service.

[0115] The recording service of the streaming media system can receive the alarm frame information sent by the AI ​​recognition / detection service. Based on the alarm frame information, the recording service can determine the target alarm frame, generate the target alarm video and return it to the AI ​​recognition / detection service, and store it in the database.

[0116] like Figure 4As shown, the business system can send video detection tasks to the detection system in the business system; the computing power module in the detection system can pull the RTP stream from the streaming media system and call the algorithm capability to detect the RTP stream.

[0117] When the algorithm capability module detects an abnormal situation, the alarm detection module in the detection system triggers an alarm and generates alarm frame information.

[0118] The timestamp synchronization module in the detection system is responsible for transparently transmitting the PTS timestamp in the alarm frame information to the streaming media system.

[0119] In addition, the business system can also turn on video recording so that the streaming media recording module of the streaming media system can start recording to obtain a full amount of recorded video data.

[0120] After the streaming media system receives the PTS timestamp, the video slice retrieval module retrieves the corresponding video clip in the database.

[0121] The offset determination module of the streaming media system determines the offset of the target alarm frame in the recorded video segment according to the PTS timestamp.

[0122] The alarm short video generation module retrieves the frames before and after the target alarm frame according to the offset, and generates the target alarm video and returns it to the business system.

[0123] like Figure 5 As shown, after the RTP stream is transmitted to the streaming media system, the streaming media system can record the video internally and generate a full amount of recorded video data with a data type of a 64-bit unsigned integer; the full amount of recorded video data can include multiple recorded video clips (recorded video clip 1, recorded video clip 2, ..., recorded video clip n).

[0124] In addition, the detection system can pull the RTP stream with the data type of 32-bit unsigned integer from the streaming media system for parsing based on the RTP protocol; and when an abnormality is detected, send an alarm frame information to the streaming media system.

[0125] The streaming media system can start to retrieve the target alarm frame based on the alarm frame information; specifically, the second timestamp can be divided by the third timestamp to obtain the target number of loops; then the product of the target number of loops and a preset value is calculated, and based on the sum of the first product value and the third timestamp, the third timestamp of the PTS is mapped to the fourth timestamp of the streaming media system whose data type is a 64-bit unsigned integer.

[0126] Based on the fourth timestamp, the target recorded video segment can be determined; then based on the timestamp of the starting frame of the target recorded video segment and the fourth timestamp, the offset of the target alarm frame in the target recorded video segment is determined, thereby determining the position of the target alarm frame in the target recorded video segment, and then generating the target alarm video.

[0127] In order to prevent the timestamp of the upstream video stream from being irregular, the timestamp in the RTP stream needs to be rewritten first, such as Figure 6 , shows an RTP packet header of an embodiment of the present invention.

[0128] Among them: 0-1 bytes: Version, Padding, Extension, CSRC Count

[0129] In byte 0, bits 0-1 are the version (Version); bit 2 is the padding (Padding); bit 3 is the extension (Extension); bits 4-7 are the CSRC count (CSRC Count).

[0130] In 1 byte, bits 0-1 are the payload type (PayloadType); bit 2 is the marker (Marker); bytes 2-3: sequence number (SequenceNumber);

[0131] 4-7 bytes: Timestamp;

[0132] 8-11 bytes: SSRC Identifier;

[0133] 12 bytes and later: CSRC Identifiers; if there are CSRC identifiers, they will start at 12 bytes, with one identifier every 4 bytes.

[0134] 12+4xCC bytes: Extension Header. If an extension header exists, it starts after the CSRC identifier.

[0135] 12+4xCC bytes: Extension Header ID;

[0136] 14+4xCC bytes: Extension Header Length (ExtensionHeaderLength) - 16 bits;

[0137] 16+4xCC bytes: Extension Header Data (ExtensionHeaderData)

[0138] Version: indicates the RTP version number. The current version is 2.

[0139] Padding: Indicates whether there are padding bytes at the end of the packet.

[0140] Extension: indicates whether there is an RTP header extension.

[0141] CSRC Count: indicates the number of CSRC identifiers.

[0142] Payload Type: indicates the type of the payload.

[0143] Marker, used to mark important events, such as frame boundaries.

[0144] Sequence Number: increases by 1 for each RTP packet sent and is used to detect packet loss and sorting.

[0145] Timestamp: 32 bits, indicating the sampling time.

[0146] SSRC Identifier: identifies the synchronization source.

[0147] CSRC Identifiers, which identify the contributing sources.

[0148] Extension Header, contains additional header information.

[0149] Offsets refers to offset, Octet refers to byte, and bit refers to position.

[0150] In an embodiment of the present invention, a streaming media system sends target streaming media data to a detection system; generates a full amount of recorded video data, wherein a second target frame of the full amount of recorded video data corresponds to a second timestamp; receives alarm frame information generated by the detection system based on abnormal detection of the target streaming media data; determines a target number of cycles; determines a fourth timestamp based on the target number of cycles and a third timestamp; the full amount of recorded video data includes multiple recorded video clips, and a target recorded video clip is determined from the multiple recorded video clips based on a fourth timestamp; determines a target alarm frame from the target recorded video clip based on the fourth timestamp; generates a target alarm video of a target length based on multiple frames before and after the target alarm frame and the target alarm frame.

[0151] The embodiments of the present invention can solve the problem that the alarm video does not match the alarm frame due to machine clock asynchrony and network delay, and the deviation between the timestamp of the alarm frame and the actual recording time, thereby affecting the accuracy and reliability of the monitoring system.

[0152] The streaming media system also performs unified processing on the timestamps of the received RTP video streams to prevent timestamp duplication caused by abnormal timestamps.

[0153] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0154] Reference Figure 7 , shows a structural schematic diagram of a device for generating an alarm video according to an embodiment of the present invention. The device for generating an alarm video can be applied to a streaming media system, and the streaming media system is connected to a detection system.

[0155] like Figure 7 As shown, the device for generating the warning video may include the following modules:

[0156] The sending module 701 is used for the streaming media system to send target streaming media data to the detection system, wherein the first target frame of the target streaming media data corresponds to a first timestamp, and the data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data;

[0157] A first generating module 702 is used to generate a full amount of recorded video data, wherein a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer;

[0158] The receiving module 703 is used to receive the alarm frame information generated by the detection system based on the abnormal detection of the target streaming media data, wherein the alarm frame information includes a third timestamp corresponding to the frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer;

[0159] The conversion module 704 is used to convert the third timestamp to obtain a fourth timestamp, where the data type of the fourth timestamp is a 64-bit unsigned integer;

[0160] The second generating module 705 is used to determine the target alarm frame from the full amount of recorded video data according to the fourth timestamp, and generate the target alarm video according to the target alarm frame.

[0161] In an optional embodiment of the present invention, the device further comprises:

[0162] The loop module 704 is used to start a new round of loop when the first timestamp is about to exceed the 32-bit unsigned integer, and assign the first timestamp to the target streaming media data starting from 0.

[0163] In an optional embodiment of the present invention, the conversion module 704 is used to determine a target cycle number; and determine a fourth timestamp according to the target cycle number and the third timestamp.

[0164] In an optional embodiment of the present invention, the conversion module 704 is used to determine the target number of cycles according to the current second timestamp and the third timestamp.

[0165] In an optional embodiment of the present invention, the conversion module 704 is used to calculate the product of the target number of cycles and a preset value to obtain a first product value; and determine a fourth timestamp according to the first product value and the third timestamp.

[0166] In an optional embodiment of the present invention, the full amount of recorded video data includes multiple recorded video clips, and the second generation module 705 is used to determine the target recorded video clip from the multiple recorded video clips according to the fourth timestamp; and, according to the fourth timestamp, determine the target alarm frame from the target recorded video clip.

[0167] In an optional embodiment of the present invention, the second generating module 705 is used to generate a target warning video of a target length according to a plurality of frames before and after the target warning frame and the target warning frame.

[0168] In an embodiment of the present invention, a streaming media system sends target streaming media data to a detection system, a first target frame of the target streaming media data corresponds to a first timestamp, and a data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data; generate a full amount of recorded video data, a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer; receive alarm frame information generated by the detection system based on the anomaly detection of the target streaming media data, the alarm frame information includes a third timestamp corresponding to a frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer; convert the third timestamp to obtain a fourth timestamp, and the data type of the fourth timestamp is a 64-bit unsigned integer; determine a target alarm frame from the full amount of recorded video data according to the fourth timestamp, and generate a target alarm video according to the target alarm frame.

[0169] The embodiments of the present invention can solve the problem that the alarm video does not match the alarm frame due to machine clock asynchrony and network delay, and the deviation between the timestamp of the alarm frame and the actual recording time, thereby affecting the accuracy and reliability of the monitoring system.

[0170] The streaming media system also performs unified processing on the timestamps of the received RTP video streams to prevent timestamp duplication caused by abnormal timestamps.

[0171] An embodiment of the present invention further provides an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the method for generating an alarm video as described above is implemented.

[0172] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for generating an alarm video as described above is implemented.

[0173] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0174] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0175] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0176] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0180] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0181] The above provides a detailed introduction to a method for generating an alarm video, a device for generating an alarm video, an electronic device and a computer-readable storage medium. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for generating an alarm video, characterized in that: Applied to a streaming media system, the streaming media system is connected to a detection system, and the method comprises: The streaming media system sends target streaming media data to the detection system, the first target frame of the target streaming media data corresponds to a first timestamp, and the data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data; Generate a full amount of recorded video data, wherein a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer; Receiving the alarm frame information generated by the detection system based on the abnormality detection of the target streaming media data, wherein the alarm frame information includes a third timestamp corresponding to the frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer; Convert the third timestamp to obtain a fourth timestamp, where the data type of the fourth timestamp is a 64-bit unsigned integer; According to the fourth timestamp, a target alarm frame is determined from the full amount of recorded video data, and a target alarm video is generated according to the target alarm frame.

2. The method according to claim 1, characterized in that: The method further comprises: When the first timestamp is about to exceed a 32-bit unsigned integer, a new cycle is started, and a first timestamp is assigned to the target streaming media data starting from 0.

3. The method according to claim 2, characterized in that The converting the third timestamp to obtain a fourth timestamp includes: Determine the target number of cycles; The fourth timestamp is determined according to the target cycle number and the third timestamp.

4. The method according to claim 3, characterized in that Determining the target number of cycles includes: A target number of cycles is determined according to the current second timestamp and the third timestamp.

5. The method according to claim 4, characterized in that The determining the fourth timestamp according to the target number of cycles and the third timestamp includes: Calculating the product of the target number of cycles and a preset value to obtain a first product value; The fourth timestamp is determined according to the first product value and the third timestamp.

6. The method according to claim 1, characterized in that: The full amount of recorded video data includes a plurality of recorded video clips, and determining the target alarm frame from the full amount of recorded video data according to the fourth timestamp includes: Determining a target recorded video segment from the plurality of recorded video segments according to the fourth timestamp; And, according to the fourth timestamp, determining the target alarm frame from the target recorded video segment.

7. The method according to claim 1, characterized in that: The step of generating a target warning video according to the target warning frame comprises: A target warning video of a target length is generated according to a plurality of frames before and after the target warning frame and the target warning frame.

8. A device for generating an alarm video, characterized in that: Applied to a streaming media system, the streaming media system is connected to a detection system, and the device comprises: A sending module, used for the streaming media system to send target streaming media data to the detection system, wherein a first target frame of the target streaming media data corresponds to a first timestamp, and a data type of the first timestamp is a 32-bit unsigned integer; the detection system is used to perform anomaly detection on the target streaming media data; A first generating module is used to generate a full amount of recorded video data, wherein a second target frame of the full amount of recorded video data corresponds to a second timestamp, and a data type of the second timestamp is a 64-bit unsigned integer; A receiving module, used to receive the alarm frame information generated by the detection system based on the abnormal detection of the target streaming media data, wherein the alarm frame information includes a third timestamp corresponding to the frame requiring an alarm; the data type of the third timestamp is a 32-bit unsigned integer; A conversion module, configured to convert the third timestamp to obtain a fourth timestamp, wherein a data type of the fourth timestamp is a 64-bit unsigned integer; The second generating module is used to determine a target alarm frame from the full amount of recorded video data according to the fourth timestamp, and generate a target alarm video according to the target alarm frame.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for generating an alarm video as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating an alarm video as described in any one of claims 1 to 7 is implemented.

Images