A video synchronization method and an electronic device

Abstract

The application discloses a video synchronization method and an electronic device, the method comprising: detecting an event of a vehicle, determining a corresponding event trigger frame of the event in a video stream, and generating a unique global event identifier for the event; determining a target frame based on a first video frame before the event trigger frame and / or a second video frame after the event trigger frame in the video stream; writing semantic event time information corresponding to the target frame into a video coding stream corresponding to the target frame through a supplemental enhancement information frame; wherein the semantic event time information comprises the global event identifier, a timestamp of the event trigger frame, and a relative time offset of the target frame relative to the event trigger frame; and sending the video coding stream carrying the semantic event time information to a cloud end, so that the cloud end establishes an event mapping relationship according to the semantic event time information, and a playing terminal aligns multiple video streams according to the relative time offset of the video frames with the event trigger frame as a time axis origin.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a video synchronization method and an electronic device. Background Technology

[0002] Vehicle-mounted video recording equipment is widely used for security monitoring and accident review. Related technologies typically rely on file-level timestamps or NTP (Network Time Protocol) synchronization to add time information to videos, achieving time alignment across multiple video streams. However, in multi-vehicle, multi-timezone environments, vehicle local clocks may deviate due to manual adjustments or synchronization failures, resulting in inconsistent time stamps for the same physical moment in videos from different vehicles. Current technologies only record UTC (Coordinated Universal Time) in the file header or metadata, lacking frame-level time stamping capabilities, making it difficult to accurately align multiple video streams after an event. Therefore, when performing multi-vehicle synchronized playback, these technologies often require manual adjustments to the progress bar for image comparison, failing to achieve automated, frame-level synchronized playback based on business events.

[0003] Therefore, how to achieve event-based frame alignment of multi-vehicle video streams without relying on absolute clock synchronization is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a video synchronization method and an electronic device that can achieve frame-level alignment of multi-vehicle video streams based on events without relying on absolute clock synchronization.

[0005] To achieve the above objectives, this application provides a video synchronization method applied to an in-vehicle terminal, comprising: Detect vehicle events, determine the event trigger frame corresponding to the event in the video stream, and generate a unique global event identifier for the event; The target frame is determined based on a first video frame before the event-triggered frame and / or a second video frame after the event-triggered frame in the video stream. The semantic event timing information corresponding to the target frame is written into the video encoded stream corresponding to the target frame through supplementary enhancement information frames; wherein, the semantic event timing information includes the global event identifier, the timestamp of the event triggering frame, and the relative time offset of the target frame relative to the event triggering frame; The video encoded stream carrying the semantic event time information is sent to the cloud so that the cloud can establish an event mapping relationship based on the semantic event time information, and the playback terminal can align the multiple video streams with the event trigger frame in the multiple video streams as the time axis origin and the relative time offset of the video frames.

[0006] Optionally, a unique global event identifier is generated for the event, including: The integer part of the timestamp of the event trigger frame is obtained as the first component, the vehicle identifier is obtained as the second component, and the event identifier of the event on the vehicle is obtained as the third component. A global event identifier is generated based on the first component, the second component, and the third component.

[0007] Optionally, the semantic event timing information corresponding to the target frame is written into the video encoded stream corresponding to the target frame through supplementary enhancement information frames, including: Register a unique identifier for the semantic event time data; The unique identifier corresponding to the semantic event time data and the semantic event time information are serialized byte by byte and used as the payload of the supplementary enhancement information frame; The supplementary enhancement information frame is inserted before the network abstraction layer unit corresponding to the target frame.

[0008] Optionally, determining the target frame based on a first video frame preceding the event-triggered frame and / or a second video frame following the event-triggered frame in the video stream includes: The keyframes in the first video frame and / or the second video frame are determined as the target frame; The semantic event timing information of non-key frames in the first video frame and / or the second video frame is estimated by the cloud or the playback terminal based on the semantic event timing information of adjacent key frames of the non-key frames.

[0009] To achieve the above objectives, this application provides a video synchronization method applied to a cloud server, comprising: The system receives video encoded streams carrying semantic event timing information from multiple vehicles; wherein the semantic event timing information is written into the video encoded stream through supplementary enhancement information frames, and the semantic event timing information includes a global event identifier, a timestamp of the event trigger frame, and a relative time offset of the video frame relative to the event trigger frame; Parse the supplemental enhancement information frames in the video encoded stream to extract the semantic event time information; Based on the semantic event time information, global event identifiers of different vehicles that meet preset conditions are associated into the same event group; wherein, the preset conditions are that the timestamp difference of the event trigger frames is less than a preset time threshold, the distance between the event occurrence locations is less than a preset distance threshold, and the event types are the same; Receive the target global event identifier sent by the playback terminal; The playback terminal sends the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located, so that the playback terminal aligns the multiple video streams with the event trigger frame in the multiple video streams as the time axis origin and according to the relative time offset of the video frames.

[0010] Optionally, after extracting the semantic event time information, the method further includes: Display strategy information for screen display time is generated based on the timestamp of the event trigger frame and the time zone information of the playback terminal; wherein, the display strategy information is used to describe the conversion rules between the timestamp in the video stream and the local time of the playback terminal; Accordingly, the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located are sent to the playback terminal, including: The playback terminal sends the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located, as well as the display strategy information, so that the playback terminal can convert the timestamps in the video streams into local times and overlay them on the video screen according to the display strategy information.

[0011] To achieve the above objectives, this application provides a video synchronization method applied to a playback terminal, comprising: Send a target global event identifier to a cloud server and receive multiple video encoded streams from the cloud server; wherein, the video encoded streams carry semantic event time information through supplementary enhancement information frames, and the semantic event time information includes a global event identifier, a timestamp of the event trigger frame, and a relative time offset of the video frame relative to the event trigger frame; Parse the supplemental enhancement information frames in the video encoded stream to extract the semantic event time information; Using the event-triggered frames in the multiple video streams as the origin of the time axis, the multiple video streams are aligned according to the relative time offset of the video frames, and the aligned multiple video streams are played synchronously.

[0012] Optionally, after extracting the semantic event time information, the method further includes: If the non-key frames in the video encoded stream do not carry semantic event timing information, then the semantic event timing information of the non-key frames is estimated based on the semantic event timing information of the adjacent key frames based on the non-key frames.

[0013] Optionally, it also includes: Receive display strategy information regarding screen display time from the cloud server; During the synchronous playback of aligned multi-channel video streams, the timestamps in the video streams are converted into local times and then overlaid on the video screen according to the display strategy information.

[0014] To achieve the above objectives, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the video synchronization method described above.

[0015] To achieve the above objectives, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the video synchronization method described above.

[0016] The video synchronization method provided in this application embeds the globally unique identifier of an event, the timestamp of the event trigger frame, and the time offset of the target frame relative to the event trigger frame into the video encoded stream. This ensures that the time information of each frame no longer depends on the device's local clock or an external time synchronization system, but is described based on the event itself. This event-centric time annotation method provides a unified time reference system for videos recorded by different vehicles under the same business event. After receiving the video stream carrying the above information, the cloud can establish a mapping relationship between the event and the video frame. The playback terminal does not need to rely on absolute time alignment and can directly achieve frame-level synchronous playback of multiple videos based on the event trigger frame as the origin and relative time offset. Thus, this application avoids the problem of multi-screen asynchrony caused by clock deviation, time zone differences, or loss of time information, and realizes automated and accurate multi-vehicle video linkage playback based on business events. This application also discloses an electronic device and a computer-readable storage medium that can achieve the same technical effects.

[0017] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings: Figure 1 This is a flowchart illustrating a video synchronization method according to an exemplary embodiment; Figure 2 A flowchart illustrating another video synchronization method according to an exemplary embodiment; Figure 3 This is a flowchart illustrating yet another video synchronization method according to an exemplary embodiment; Figure 4 This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] This application discloses a video synchronization method that achieves frame-level alignment of multi-vehicle video streams based on events without relying on absolute clock synchronization.

[0023] See Figure 1 A flowchart illustrating a video synchronization method according to an exemplary embodiment is shown below. Figure 1 As shown, it includes: S101: Detect vehicle events, determine the event trigger frame corresponding to the event in the video stream, and generate a unique global event identifier for the event; In this embodiment, the execution subject is a vehicle. An event refers to a behavior or state with specific business significance that occurs during the operation of an industrial vehicle, such as a collision, emergency braking, boundary crossing, or pallet drop. An event trigger frame refers to the physical frame position in the video stream corresponding to the moment the event occurs; it serves as the reference point for subsequent calculations of the relative time offset. A global event identifier is an encoding used to uniquely identify a specific business event across the entire network, ensuring that events generated by different vehicles at different times are not confused.

[0024] In this embodiment, the in-vehicle system acquires raw video streams from the vehicle's surroundings, such as videos with a resolution of 1920×1080 and a frame rate of 25 or 30 frames per second. The system then preprocesses and compresses the raw video to generate a video bitstream compatible with standards such as H.264 / H.265. The in-vehicle system reserves an SEI (Supplemental Enhancement Information) extension write interface in the encoding process, providing an entry point for subsequently inserting semantic event timing information.

[0025] In this step, the in-vehicle system uses multi-source data, including vehicle driving data (such as vehicle speed and acceleration), sensor data (such as collision sensor data), and human operation signals, to detect in real time whether a predefined business event has occurred. Once an event is detected, the system immediately records the UTC timestamp of the event trigger and locks the corresponding frame in the video stream as the event trigger frame. Subsequently, a globally unique identifier is generated for the event to facilitate cross-vehicle event association and video synchronization in the cloud. This process achieves a precise mapping from physical events to video stream data.

[0026] To ensure time accuracy and uniqueness, UTC timestamps can be calculated using the following formula: ,in, This is a UTC timestamp, in microseconds. This is the integer second portion of the Unix timestamp, in seconds. This represents the microsecond portion of the current second, with a value range from 0 to 999999.

[0027] As a feasible implementation method, generating a unique global event identifier for the event includes: obtaining the integer part of the timestamp of the event trigger frame as a first component, the vehicle identifier as a second component, and the event identifier of the event local to the vehicle as a third component; and generating a global event identifier based on the first component, the second component, and the third component.

[0028] The integer part of the timestamp typically refers to the integer seconds portion of a Unix timestamp, used to identify the absolute starting point of the event. The vehicle identifier is a unique short code assigned or mapped by the platform, used to distinguish multiple events occurring on the same vehicle within a short period.

[0029] In practice, a composite coding structure is used to combine the above three components into a 64-bit unsigned integer global event identifier. The specific calculation method is as follows: ,in, As a global event identifier, The integer second portion of the UTC time of the event (as mentioned above) (Corresponding), the value range is determined by a 32-bit unsigned integer. A unique 16-bit short code identifier for the vehicle (uniformly assigned or mapped by the platform to avoid hash collisions), occupying 16 bits. This is the vehicle's local event sequence number, with a value ranging from 0 to 65535. This indicates a left shift operation. This indicates a bitwise OR operation.

[0030] As can be seen, this implementation constructs a globally unique event fingerprint by combining time, space (vehicle), and sequence number, which solves the identification conflict problem in multi-vehicle concurrent scenarios. Moreover, the calculation is simple and efficient, making it suitable for operation on the resource-constrained vehicle edge.

[0031] S102: Determine the target frame based on the first video frame before the event trigger frame and / or the second video frame after the event trigger frame in the video stream; The first video frame refers to the video frame preceding the event trigger frame, typically originating from the pre-record buffer of a vehicle-mounted DVR (Digital Video Recorder), used to record the scene before the event occurs. The second video frame refers to the video frame following the event trigger frame, used to record the evolution process after the event occurs. The target frame refers to the video frame into which semantic event timing information needs to be embedded, and can be related frames before and after the event trigger frame.

[0032] In this step, a time window is defined based on a preset traceability duration (e.g., 5 seconds before the event to 5 seconds after the event), and the target frame is determined based on the video frames within this window. In this way, the complete sequence of images before and after the event can be associated with the event itself, providing a complete contextual image for subsequent multi-vehicle synchronous playback.

[0033] Let the traceback duration be seconds, video frame rate The number of traceback frames that need to be updated is: ,in, For the frames that need to be updated, To trace the duration, the unit is seconds. This refers to the video frame rate, measured in frames per second.

[0034] S103: Write the semantic event time information corresponding to the target frame into the video encoding stream corresponding to the target frame through supplementary enhancement information frames; wherein, the semantic event time information includes the global event identifier, the timestamp of the event triggering frame, and the relative time offset of the target frame relative to the event triggering frame; Semantic event timing information refers to a data set describing the timing attributes of video frames based on business events, including fields such as global event identifier, Coordinated Universal Time (UTC) timestamp of the event-triggered frame, and relative time offset of the target frame relative to the event-triggered frame. Supplemental enhancement information frames are network abstraction layer units defined in the H.264 or H.265 video coding standards, used to carry auxiliary information related to the video stream without affecting the normal decoding of the standard decoder. Relative time offset refers to the time difference between the target frame and the event-triggered frame, measured in microseconds; a positive number indicates the target frame is after the event-triggered frame, and a negative number indicates it is before. For the first... The time offset of a trace frame (indices in the buffer) relative to the event-triggered frame can be written as: ,in, For the first The event-relative time offset of each trace frame, in microseconds. For the first The original acquisition time of each trace frame. This is the original acquisition time of the event trigger frame.

[0035] In this step, an SEI message is constructed in the video encoder. The aforementioned semantic event timing information is serialized and written into the SEI's payload. This makes timing information an intrinsic attribute of the video stream, tightly integrated with the video data. Regardless of how the video file is transcoded or encapsulated, as long as the SEI data is retained, the timing information will not be lost, thus achieving high-fidelity transmission of timing information.

[0036] As a feasible implementation, the semantic event timing information corresponding to the target frame is written into the video encoded stream corresponding to the target frame through a supplementary enhancement information frame, including: registering a unique identifier corresponding to the semantic event timing data; serializing the unique identifier corresponding to the semantic event timing data and the semantic event timing information by bytes as the payload of the supplementary enhancement information frame; and inserting the supplementary enhancement information frame before the network abstraction layer unit corresponding to the target frame.

[0037] The unique identifier typically refers to a universally unique identifier used to distinguish SEI data from different applications or vendor-defined data. The Network Abstraction Layer unit (NAL unit) is the basic building block of the video encoded stream.

[0038] In this step, a dedicated UUID (e.g., a specific 16-byte sequence) is first registered for the semantic event timing data to ensure it does not conflict with other SEI data. Then, this UUID is used as a prefix and concatenated with the serialized semantic event timing information (such as version number, flag bits, timestamps, etc.) to form the complete SEI payload. Finally, using the encoder interface, the encapsulated SEI NAL unit is inserted before the NAL unit of the target video frame. This UUID-based registration mechanism allows the playback end to quickly identify and parse the timing information, while standard decoders, if they do not recognize the UUID, will automatically ignore the SEI data, ensuring compatibility with existing devices.

[0039] As a preferred embodiment, determining the target frame based on a first video frame before the event-triggered frame and / or a second video frame after the event-triggered frame in the video stream includes: determining key frames in the first video frame and / or the second video frame as the target frame; wherein, the semantic event timing information of non-key frames in the first video frame and / or the second video frame is estimated by the cloud or the playback terminal based on the semantic event timing information of adjacent key frames of the non-key frames.

[0040] In video encoding, a keyframe is a frame that can be decoded independently without relying on other frames (such as an I-frame or some P-frames), and is usually the starting point of a Group of Pictures (GOP). A non-keyframe is a frame that requires reference to the preceding and following frames to be decoded (such as a B-frame).

[0041] In practical implementation, to reduce the bandwidth and storage space occupied by SEI data in the video stream, a sparse embedding strategy is adopted. Complete semantic event timing information is written only in keyframes, while supplementary enhancement information frames are not written for non-keyframes. The system identifies whether each target frame is a keyframe based on the video encoder's encoding structure, and only keyframes are identified as target frames for embedding semantic event timing information. For non-keyframes, the cloud or playback terminal estimates the relative time offset of the non-keyframe in subsequent processing using a linear interpolation method based on the relative time offset recorded in the two adjacent keyframes.

[0042] As can be seen, this implementation significantly reduces the write frequency of SEI data through a metadata compression strategy that combines keyframe marking and interpolation. While ensuring time accuracy, it keeps the amount of additional data to an extremely low level (typically having a negligible impact on the overall bitrate), effectively solving the problem of balancing storage cost and time accuracy under massive video data.

[0043] S104: Send the video encoded stream carrying the semantic event time information to the cloud so that the cloud can establish an event mapping relationship based on the semantic event time information, and allow the playback terminal to align the multiple video streams with the event trigger frame in the multiple video streams as the time axis origin and the relative time offset of the video frames.

[0044] Among them, the event mapping relationship refers to the association index between the global event identifier in the semantic event time information and the specific video clip, vehicle location and UTC time in the cloud.

[0045] In this step, after receiving the video stream carrying SEI information, the cloud parses the semantic event timing data within it, constructing a dual mapping from video frames to both the event timeline and the UTC timeline. When operators need to view a specific event (such as an accident), the cloud retrieves video clips of all relevant vehicles based on the global event ID, and uses the event trigger frames of these clips as the timeline origin (t=0). During playback, the terminal no longer relies on absolute time (such as local or server time), which is prone to errors, but instead precisely calculates the display timing of each frame based on the relative time offset recorded in the SEI. For example, when the player's timeline is set to t=-2 seconds, the video feeds of all vehicles will synchronously display the scene from 2 seconds prior to the event.

[0046] As can be seen, this step, by establishing an event-centric mapping relationship, completely solves the problem of inconsistent time bases across time zones and devices. The playback terminal can achieve precise frame-level synchronization, enabling operators to intuitively compare the same business moment from different perspectives, greatly improving the efficiency of accident analysis and multi-vehicle collaboration.

[0047] The video synchronization method provided in this application embeds the globally unique identifier of an event, the timestamp of the event trigger frame, and the time offset of the target frame relative to the event trigger frame into the video encoding stream. This ensures that the time information of each frame no longer depends on the device's local clock or an external time synchronization system, but is described based on the event itself. This event-centric time annotation method provides a unified time reference system for videos recorded by different vehicles under the same business event. After receiving the video stream carrying the above information, the cloud can establish a mapping relationship between the event and the video frame. The playback terminal does not need to rely on absolute time alignment and can directly use the event trigger frame as the origin to achieve frame-level synchronous playback of multiple videos according to the relative time offset. Therefore, this application avoids the problem of multi-screen asynchrony caused by clock deviation, time zone differences, or loss of time information, and realizes automated and accurate multi-vehicle video linkage playback based on business events.

[0048] This application discloses a video synchronization method. See also... Figure 2 A flowchart illustrating another video synchronization method according to an exemplary embodiment, such as... Figure 2 As shown, it includes: S201: Receive a video encoded stream carrying semantic event timing information from multiple vehicles; wherein the semantic event timing information is written into the video encoded stream through supplementary enhancement information frames, and the semantic event timing information includes a global event identifier, a timestamp of the event trigger frame, and a relative time offset of the video frame relative to the event trigger frame. In this embodiment, the execution entity is a cloud server. In this step, the cloud server acts as a central service platform, receiving video encoded streams uploaded from different vehicles and regions via a network interface. These video encoded streams are generated by the onboard units of each vehicle after event detection, and semantic event timing information is embedded during the encoding process through supplementary enhancement information frames. The cloud server identifies and stores the received video streams, recording metadata such as the vehicle identifier, upload time, and video duration for each video stream. By receiving video encoded streams carrying semantic event timing information from multiple vehicles, the cloud can aggregate event-related video data scattered across different vehicles, providing a data foundation for subsequent cross-vehicle event correlation analysis.

[0049] S202: Parse the supplemental enhancement information frames in the video encoded stream and extract the semantic event time information; In this step, the cloud server decapsulates each received video encoded stream, traversing each Network Abstraction Layer (NET) unit in the video stream. When a NET unit of type Supplemental Enhancement (SA) frame is detected, its payload is further read. The cloud first checks whether the first 16 bytes of the payload match a dedicated universally unique identifier registered for semantic event timing data. If they match, the cloud sequentially parses the semantic event timing information fields, such as the global event identifier, the timestamp of the event trigger frame, and the relative time offset, from the payload according to a predefined data structure. The cloud associates this parsed information with the corresponding video frames and stores it in the database, establishing a mapping relationship between video frame indexes and semantic event timing information. By parsing SA frames to extract semantic event timing information, the cloud can obtain the time position of each video frame relative to the event trigger frame, providing a time reference for subsequent event association and multi-channel video synchronization.

[0050] S203: Based on the semantic event time information, associate the global event identifiers of different vehicles that meet the preset conditions into the same event group; wherein, the preset conditions are that the timestamp difference of the event trigger frames is less than a preset time threshold, the distance between the event occurrence locations is less than a preset distance threshold, and the event types are the same; The preset time threshold is a pre-defined upper limit for the time difference, used to determine whether events recorded by different vehicles are close in time; for example, it can be set to 5 seconds. The preset distance threshold is a pre-defined upper limit for the spatial distance, used to determine whether events recorded by different vehicles are close in location; for example, it can be set to 100 meters. The event type refers to the specific category of the event, such as collision, emergency braking, or driving beyond the designated boundary; the corresponding event type code can be obtained from the flag field of the supplementary enhancement information frame. An event group refers to a collection formed by aggregating multiple related events from different vehicles, representing the recording of the same physical event from different vehicle perspectives.

[0051] In this step, after the cloud server analyzes and obtains the semantic event time information of each video stream, it performs clustering analysis on the global event identifiers from different vehicles. For every two global event identifiers from different vehicles, the cloud respectively obtains the Coordinated Universal Time (UTC) timestamp of the event trigger frame corresponding thereto, the vehicle position coordinates (which can be obtained from the video stream metadata or vehicle-reported information), and the event type code. The cloud calculates the absolute value of the difference between the two event trigger frame timestamps. If this difference is less than the preset time threshold, it continues to determine whether the distance between the two event occurrence locations is less than the preset distance threshold. If this is satisfied, it further determines whether the event type codes are the same. When all of the above three conditions are satisfied, the cloud associates these two global event identifiers into the same event group. The cloud continues to traverse all unassociated global event identifiers, repeats the above judgment process, classifies all global event identifiers that meet the conditions into the corresponding event groups, and generates a unique event group identifier for each event group. By associating the global event identifiers of different vehicles that meet the preset conditions into the same event group, the cloud can identify the same physical event scattered in the multi-vehicle videos and establish an event-level association relationship for subsequent multi-vehicle joint playback.

[0052] S204: Receive the target global event identifier sent by the playback terminal; Wherein, the target global event identifier refers to the global event identifier corresponding to an event that the playback terminal sends to the cloud after the user selects a certain event from the event list.

[0053] In this step, the playback terminal displays the event list associated by the cloud on the user interface, and each event corresponds to one or more global event identifiers. When the user selects a certain event for playback by clicking or other interaction methods, the playback terminal uses the global event identifier corresponding to this event as a request parameter and sends it to the cloud server through a network protocol such as the Hypertext Transfer Protocol or the Real-Time Streaming Protocol. After receiving this request, the cloud server extracts the target global event identifier from it. By receiving the target global event identifier sent by the playback terminal, the cloud can know the specific event that the user hopes to playback, so as to accurately return the relevant multi-channel video streams.

[0054] S205: Send the video coding streams corresponding to all global event identifiers in the event group where the target global event identifier is located to the playback terminal, so that the playback terminal aligns the multi-channel video streams with the relative time offset of the video frames with the event trigger frame in the multi-channel video streams as the time axis origin.

[0055] In this step, the cloud server, based on the received target global event identifier, searches the database for the event group to which the global event identifier belongs, and obtains a list of all global event identifiers in that event group. For each global event identifier in the list, the cloud looks up its corresponding video encoded stream storage address and obtains the complete video encoded stream data. The cloud packages these video encoded streams and returns them to the playback terminal. After receiving the multiple video streams, the playback terminal parses the semantic event timing information carried in the supplemental enhancement information frames of each video stream, and performs frame-level alignment based on the relative time offset of each frame, using the event trigger frame in each video stream as the time axis origin, to achieve synchronous playback of multiple vehicle perspectives. By sending the video encoded streams corresponding to all global event identifiers in the event group to the playback terminal, the cloud provides the user with complete multi-vehicle video data, enabling the playback terminal to achieve frame-level synchronous playback of multiple videos based on the event time axis.

[0056] In a preferred embodiment, after extracting the semantic event time information, the method further includes: generating display strategy information for screen display time based on the timestamp of the event trigger frame and the time zone information of the playback terminal; wherein, the display strategy information is used to describe the conversion rules between the timestamp in the video stream and the local time of the playback terminal; correspondingly, sending the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located to the playback terminal includes: sending the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located and the display strategy information to the playback terminal, so that the playback terminal converts the timestamp in the video stream to the local time according to the display strategy information and then overlays it on the video screen.

[0057] The screen display time refers to the time watermark superimposed on the video frame, used to show viewers the actual time corresponding to that frame. Display strategy information refers to the set of rules used to guide the playback terminal in time conversion, which may include the Coordinated Universal Time (UTC) timestamp of the event trigger frame, the viewer's time zone offset, and the time format (such as 24-hour or 12-hour format). Time zone information refers to the offset of the playback terminal's location relative to UTC, which can be obtained by the playback terminal based on the Internet Protocol address or manually configured by the user.

[0058] In this step, after parsing and obtaining the semantic event time information, the cloud obtains the Coordinated Universal Time (UTC) timestamp of the event trigger frame. When the playback terminal sends the target global event identifier to the cloud, the cloud simultaneously obtains the playback terminal's timezone information. This timezone information can be carried by the playback terminal in the request or obtained by the cloud through geolocation resolution based on the playback terminal's Internet Protocol address. The cloud generates display policy information based on the UTC timestamp of the event trigger frame and the playback terminal's timezone offset. This information defines the conversion rules between the UTC time of each frame in the video stream and the playback terminal's local time. When returning the video encoded stream to the playback terminal, the cloud sends the display policy information as additional metadata. After receiving the display policy information, the playback terminal calculates the UTC time of the current playback frame based on the relative time offset during playback, converts it to local time based on the timezone offset, and finally displays the converted local time as a watermark on the video screen. By generating and sending display strategy information, the cloud enables the playback terminal to dynamically display the local time according to the viewer's time zone, eliminating the time zone conversion problems in cross-regional collaborative analysis. Viewers in different regions see time tags that conform to their local habits when watching the same video.

[0059] Let the UTC timestamp of the event trigger frame be... The relative event time read in the current frame SEI is (unit: microseconds), then the first... The UTC time corresponding to the frame is approximately: ,in, For the first The frame corresponds to the UTC time in seconds. The event trigger frame's UTC timestamp, in seconds. For the first The relative time of a frame to the frame that triggered the event, in microseconds, given the time offset of the current viewer's time zone relative to UTC. Then, UTC time can be converted to the viewer's local time: ,in, For the first The frame corresponds to the viewer's local time, in seconds. This represents the time offset of the viewer's time zone relative to UTC, in seconds (positive values ​​indicate the Eastern Time Zone, negative values ​​indicate the Western Time Zone, determined by the playback device based on the viewer's IP or user configuration).

[0060] The video synchronization method provided in this embodiment allows the cloud to receive and parse video encoded streams carrying semantic event time information from multiple vehicles, extracting embedded global event identifiers, event trigger frame timestamps, and relative time offsets. This enables the cloud to obtain the precise time position of each video frame relative to the event trigger frame from the encoding layer. Based on this, the cloud automatically associates globally generated event identifiers from different vehicles into the same event group according to preset conditions such as timestamp differences, location distances, and event types, achieving cross-vehicle and cross-time zone event aggregation. When the playback terminal sends a target global event identifier, the cloud returns the video encoded streams corresponding to all global event identifiers in that event group, allowing the playback terminal to perform frame-level alignment based on relative time offsets, using the event trigger frames in each video stream as the time axis origin. Therefore, automated synchronized playback of multi-vehicle videos based on business events can be achieved without relying on absolute clock synchronization or manual comparison, significantly improving the efficiency of multi-view accident analysis and coordinated monitoring.

[0061] This application discloses a video synchronization method. See also... Figure 3 A flowchart illustrating another video synchronization method according to an exemplary embodiment, such as... Figure 3 As shown, it includes: S301: Send a target global event identifier to the cloud server and receive multiple video encoded streams from the cloud server; wherein, the video encoded streams carry semantic event time information through supplementary enhancement information frames, and the semantic event time information includes a global event identifier, a timestamp of the event trigger frame, and a relative time offset of the video frame relative to the event trigger frame; In this embodiment, the execution entity is the playback terminal. In this step, the playback terminal displays a list of events associated with the cloud on its user interface, with each event corresponding to one or more global event identifiers. When a user selects an event for playback via clicking or other interactive methods, the playback terminal sends the corresponding global event identifier as a request parameter to the cloud server via network protocols such as Hypertext Transfer Protocol or Real-Time Streaming Protocol. The cloud server locates the event group to which the target global event identifier belongs and returns the video encoded streams corresponding to all global event identifiers in that event group. The playback terminal receives the multiple video encoded streams returned from the cloud and temporarily stores them in a local buffer, preparing for subsequent synchronized playback. By sending the target global event identifier to the cloud and receiving multiple video encoded streams, the playback terminal can obtain complete video data of the same business event from different vehicle perspectives.

[0062] S302: Parse the supplemental enhancement information frames in the video encoded stream and extract the semantic event time information; In this step, the playback terminal decapsulates each received video encoded stream, traversing each Network Abstraction Layer (NET) unit in the video stream. When a NET unit of type Supplemental Enhancement Frame (SEFF) is detected, its payload is further read. The playback terminal first checks whether the first 16 bytes of the payload match a dedicated universally unique identifier registered for semantic event timing data. If they match, the version number, flag, global event identifier, Coordinated Universal Time (UTC) timestamp of the event trigger frame, relative time offset, recording time zone offset, and frame sequence number are sequentially parsed from the payload according to a predefined data structure. The playback terminal associates this parsed information with the corresponding video frames, constructing a mapping table from video frames to the event timeline, which is stored in memory for subsequent playback. By parsing the SEFF to extract semantic event timing information, the playback terminal can obtain the precise time position of each video frame relative to the event trigger frame, providing a time reference for the synchronization and alignment of multiple video streams.

[0063] In a preferred embodiment, after extracting the semantic event timing information, the method further includes: if the non-key frames in the video encoded stream do not carry semantic event timing information, then estimating the semantic event timing information of the non-key frames based on the semantic event timing information of the adjacent key frames based on the non-key frames.

[0064] In practice, when parsing semantic event timing information, the playback terminal discovers that some non-key frames' corresponding supplementary enhancement information frames do not carry complete semantic event timing information. For these non-key frames, the playback terminal estimates the relative time offset of the non-key frame using a linear interpolation method based on the relative time offset recorded in the two adjacent key frames before and after it.

[0065] Suppose the known index is and The two frames were written with the complete relative event time. and The index of a certain frame in the middle is ,satisfy Then the relative event time of the frame can be estimated using the following formula: ,in, This represents the relative event time of the intermediate frame, in seconds. , As an index for two adjacent frames, and with complete relative time information written, in terms of bandwidth and storage overhead analysis, the video bitrate is assumed to be... (Unit: bps), frame rate The SEI data size for each frame is In bytes, the proportion of SEI data in the total bitrate can be written as: ,in, This represents the percentage of SEI data. The size of each SEI data frame, in bytes. This refers to the video frame rate, measured in frames per second. This refers to the video bitrate, measured in bits per second.

[0066] S303: Using the event-triggered frame in the multi-channel video stream as the origin of the time axis, align the multi-channel video stream according to the relative time offset of the video frames, and play the aligned multi-channel video stream synchronously.

[0067] In this step, after obtaining the semantic event timing information of each video stream, the playback terminal constructs an event timeline using the event trigger frame as a common time origin. For each video stream, the playback terminal maps the frame onto the event timeline based on its relative time offset. During playback, the playback terminal uses time t on the event timeline as the primary control variable, and for each video stream, it finds the frame with the closest relative time offset to t for playback. When the user drags the progress bar or controls playback, the playback terminal updates the time value on the current event timeline and synchronously updates the playback frames of each video stream. This alignment method does not rely on a fixed frame rate assumption and can effectively handle situations where different vehicle video streams have inconsistent frame rates, dropped frames, or jitter, achieving frame-level synchronous playback of multiple video streams. By aligning multiple video streams with the event trigger frame as the origin and relative time offset, the playback terminal allows users to simultaneously view the complete process before and after the same event from multiple vehicle perspectives without manually adjusting the progress bar for image comparison.

[0068] In absolute time alignment mode, all video streams are aligned according to their UTC timestamps. Assume there exists... Road video, number The first video of the road Frame UTC timestamp is Given the target playback time It is necessary to find the frame index closest to the target time for each video stream. , can be represented as: ,in, For the first The optimal frame index corresponding to the video stream. For the first Road Video No. The frame's UTC timestamp, in seconds or microseconds (must be consistent). To synchronize reference time, the units and Consistent.

[0069] In event alignment mode, all video streams are aligned using the same global event ID and relative event time. For the same Let the relative time of the event that is currently desired to be played be... (Unit: microseconds), then the first... The frames that should be played in the video stream are those that meet the following conditions. : ,in, For the first The current frame index of the video that should be played. For the first Road Video No. The relative event time recorded in the SEI frame.

[0070] The playback terminal uses time on the event timeline As the main control variable, search for the video streams. closest This method does not rely on a fixed frame rate assumption and can effectively handle frame drops or jitter issues. In multi-event scenarios, the player can switch to different frames. The corresponding timeline is played back synchronously.

[0071] As a preferred embodiment, it further includes: receiving display strategy information of screen display time from the cloud server; and during the synchronous playback of aligned multi-channel video streams, converting the timestamps in the video streams into local time according to the display strategy information and then overlaying them onto the video screen.

[0072] In practice, the playback terminal receives display strategy information from the cloud server simultaneously with the video encoded stream. This information includes the Coordinated Universal Time (UTC) timestamp of the event-triggered frame, the playback terminal's timezone offset, and the time display format. During synchronized playback, the playback terminal calculates the UTC timestamp of the current playback frame based on its relative time offset. Then, it calculates the local time based on the timezone offset in the display strategy information. Finally, the converted local time is rendered as a watermark in a specified time format (e.g., YYYY-MM-DD HH:MM:SS) and overlaid on a designated position on the video screen, such as the upper left or lower right corner. By receiving display strategy information and dynamically converting the displayed local time, the playback terminal ensures that viewers in different time zones see time stamps that conform to their respective local habits when watching the same video, facilitating cross-regional collaborative analysis without requiring viewers to manually perform timezone conversions.

[0073] The video synchronization method provided in this embodiment allows the playback terminal to send a target global event identifier to the cloud, receive multiple video encoded streams carrying semantic event time information, and parse supplementary enhancement information frames to extract the global event identifier, event trigger frame timestamps, and relative time offsets, thereby obtaining the precise coordinates of each video frame on the event timeline. Based on this, the playback terminal uses the event trigger frames in each video stream as a common origin and performs frame-level matching according to the relative time offsets of the video frames, achieving precise synchronized playback of multiple video streams without relying on absolute clock alignment or manual adjustment of the progress bar. For non-critical frames that do not carry semantic event time information, the playback terminal estimates the time by linear interpolation of adjacent critical frames, ensuring that all video frames can obtain event time coordinates and guaranteeing the integrity of synchronized playback. Furthermore, the playback terminal dynamically converts the Coordinated Universal Time (UTC) of the video frames to the viewer's local time and overlays it according to the display strategy information sent from the cloud, eliminating the time conversion problems associated with cross-timezone viewing. Therefore, this embodiment achieves automated frame-level synchronized playback of multi-vehicle video based on business events, significantly improving the user experience and efficiency of accident analysis and multi-view linkage.

[0074] To implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 4 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 4 As shown, the electronic device includes: Communication interface 1 enables information exchange with other devices, such as network devices; Processor 2 is connected to communication interface 1 to enable information interaction with other devices and to execute the video synchronization method provided by one or more of the above-mentioned technical solutions when running a computer program. The computer program is stored in memory 3.

[0075] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 4 The general will label all buses as Bus System 4.

[0076] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0077] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0078] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0079] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0080] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by a processor 2 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0081] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0082] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method of video synchronization, the method comprising: Applications in automotive applications include: Detect vehicle events, determine the event trigger frame corresponding to the event in the video stream, and generate a unique global event identifier for the event; The target frame is determined based on a first video frame before the event-triggered frame and / or a second video frame after the event-triggered frame in the video stream. The semantic event timing information corresponding to the target frame is written into the video encoded stream corresponding to the target frame through supplementary enhancement information frames; wherein, the semantic event timing information includes the global event identifier, the timestamp of the event triggering frame, and the relative time offset of the target frame relative to the event triggering frame; The video encoded stream carrying the semantic event time information is sent to the cloud so that the cloud can establish an event mapping relationship based on the semantic event time information, and the playback terminal can align the multiple video streams with the event trigger frame in the multiple video streams as the time axis origin and the relative time offset of the video frames.

2. The video synchronization method of claim 1, wherein, Generate a unique global event identifier for the event, including: The integer part of the timestamp of the event trigger frame is obtained as the first component, the vehicle identifier is obtained as the second component, and the event identifier of the event on the vehicle is obtained as the third component. A global event identifier is generated based on the first component, the second component, and the third component.

3. The video synchronization method of claim 1, wherein, Writing the semantic event timing information corresponding to the target frame into the video encoded stream corresponding to the target frame through supplementary enhancement information frames includes: Register a unique identifier for the semantic event time data; The unique identifier corresponding to the semantic event time data and the semantic event time information are serialized byte by byte and used as the payload of the supplementary enhancement information frame; The supplementary enhancement information frame is inserted before the network abstraction layer unit corresponding to the target frame.

4. The video synchronization method of claim 1, wherein, Determining the target frame based on a first video frame preceding the event-triggered frame and / or a second video frame following the event-triggered frame in the video stream includes: The keyframes in the first video frame and / or the second video frame are determined as the target frame; The semantic event timing information of non-key frames in the first video frame and / or the second video frame is estimated by the cloud or the playback terminal based on the semantic event timing information of adjacent key frames of the non-key frames.

5. A method of video synchronization, the method comprising: Applications in cloud servers, including: The system receives video encoded streams carrying semantic event timing information from multiple vehicles; wherein the semantic event timing information is written into the video encoded stream through supplementary enhancement information frames, and the semantic event timing information includes a global event identifier, a timestamp of the event trigger frame, and a relative time offset of the video frame relative to the event trigger frame; Parse the supplemental enhancement information frames in the video encoded stream to extract the semantic event time information; Based on the semantic event time information, global event identifiers of different vehicles that meet preset conditions are associated into the same event group; wherein, the preset conditions are that the timestamp difference of the event trigger frames is less than a preset time threshold, the distance between the event occurrence locations is less than a preset distance threshold, and the event types are the same; Receive the target global event identifier sent by the playback terminal; The playback terminal sends the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located, so that the playback terminal aligns the multiple video streams with the event trigger frame in the multiple video streams as the time axis origin and according to the relative time offset of the video frames.

6. The video synchronization method of claim 5, wherein, After extracting the semantic event time information, the method further includes: Display strategy information for screen display time is generated based on the timestamp of the event trigger frame and the time zone information of the playback terminal; wherein, the display strategy information is used to describe the conversion rules between the timestamp in the video stream and the local time of the playback terminal; Accordingly, the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located are sent to the playback terminal, including: The playback terminal sends the video encoded streams corresponding to all global event identifiers in the event group where the target global event identifier is located, as well as the display strategy information, so that the playback terminal can convert the timestamps in the video streams into local times and overlay them on the video screen according to the display strategy information.

7. A method of video synchronization, the method comprising: Applied to playback terminals, including: Send a target global event identifier to a cloud server and receive multiple video encoded streams from the cloud server; wherein, the video encoded streams carry semantic event time information through supplementary enhancement information frames, and the semantic event time information includes a global event identifier, a timestamp of the event trigger frame, and a relative time offset of the video frame relative to the event trigger frame; Parse the supplemental enhancement information frames in the video encoded stream to extract the semantic event time information; Using the event-triggered frames in the multiple video streams as the origin of the time axis, the multiple video streams are aligned according to the relative time offset of the video frames, and the aligned multiple video streams are played synchronously.

8. The video synchronization method of claim 7, wherein, After extracting the semantic event time information, the method further includes: If the non-key frames in the video encoded stream do not carry semantic event timing information, then the semantic event timing information of the non-key frames is estimated based on the semantic event timing information of the adjacent key frames based on the non-key frames.

9. The video synchronization method of claim 7, wherein, Also includes: Receive display strategy information regarding screen display time from the cloud server; During the synchronous playback of aligned multi-channel video streams, the timestamps in the video streams are converted into local times and then overlaid on the video screen according to the display strategy information.

10. An electronic device, comprising: include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the video synchronization method as described in any one of claims 1 to 9.

Images