An event information fusion method and an electronic device

By using time range overlap as a matching criterion in edge-cloud collaborative intelligent security playback scenarios, a fused event carrying event information and video file information is generated, solving the problem of event and video matching failure and achieving accurate matching and playback capabilities.

CN123123528APending Publication Date: 2026-11-17BEIJING QIREN TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611272772.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In edge-cloud collaborative intelligent security playback scenarios, existing technologies cannot effectively handle the time offset and complex overlap between events and videos, resulting in the failure to match events with videos. Users cannot review the corresponding recording content through events, thus losing the ability to play back and verify.

Method used

By acquiring a list of events and videos to be matched, and using time range overlap as the matching criterion, a fused event carrying event information, playable time range information, and the original time range information of the underlying video file is generated, thus achieving accurate matching between events and videos.

Benefits of technology

It achieves accurate matching of events and videos under time offset or complex overlap conditions, improves the robustness of matching, and ensures the replayability of fused events, enabling users to accurately locate and play the corresponding recorded content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123528A_ABST
    Figure CN123123528A_ABST
Patent Text Reader

Abstract

This application discloses an event information fusion method and electronic device. The method includes: obtaining a list of events to be matched and a list of videos to be matched; the list of events to be matched includes multiple events ordered chronologically, and the list of videos to be matched includes multiple videos ordered chronologically, each video carrying a time range and the original time range of the corresponding underlying video file; traversing the list of events to be matched and the list of videos to be matched, and determining a match between the target event and the target video if their time ranges overlap; generating a fused event associated with the target video based on the type of the underlying video file corresponding to the target video; the fused event carrying event information of the target event, playable time range information of the target video, and the original time range information of the underlying video file. This application achieves accurate matching of events and videos while retaining playable information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of surveillance technology, and more specifically, to an event information fusion method and an electronic device. Background Technology

[0002] In edge-cloud collaborative intelligent security playback scenarios, it's necessary to associate local video with events generated by cloud analytics so users can review corresponding recordings. Current technologies often use fixed timestamp matching to associate events and recordings; a match is only determined when the timestamp of an event perfectly matches the timestamp of a recording segment. However, due to network jitter and reporting delays, there's often a time offset between cloud events and local video recordings. Events and videos may only partially overlap, or even span multiple recording segments. In such cases, fixed timestamp matching can only identify instances where the timestamps match perfectly, failing to handle time offsets and complex overlaps. This leads to event-video matching failures, preventing the event from being associated with a playable recording, and ultimately, the user cannot review the recording, losing playback verification capabilities. Furthermore, if the association is based solely on cloud events, the playable time range information carried by the local video and the original time range information of the underlying recording file are discarded. This results in the association results lacking playable intervals and file location information, making it impossible for users to play back the video after clicking, and thus failing to achieve playback verification.

[0003] Therefore, how to achieve accurate matching of events and videos while retaining playable information when there is a time offset or complex overlap between the events and videos is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide an event information fusion method and an electronic device that achieves accurate matching of events and videos and retains playable information when there is a time offset or complex overlap between the events and videos.

[0005] To achieve the above objectives, this application provides an event information fusion method, comprising: Obtain a list of events to be matched and a list of videos to be matched; wherein, the list of events to be matched includes multiple events sorted in chronological order, and the list of videos to be matched includes multiple videos sorted in chronological order, each video carrying the time range of the video and the original time range of the corresponding underlying recording file; Traverse the list of events to be matched and the list of videos to be matched. If the time range of the target event and the time range of the target video overlap, output the target event and the target video as a matching pair. Based on the type of the underlying recording file corresponding to the target video, a fusion event associated with the target video is generated; wherein, the fusion event carries event information of the target event, playable time range information of the target video, and original time range information of the underlying recording file.

[0006] Optionally, before generating the fusion event associated with the target video based on the type of the underlying recording file corresponding to the target video, the method further includes: The type of the underlying video file is determined based on the generation method of the underlying video file corresponding to the target video; wherein, the type of the underlying video file includes files generated by trigger-based recording and slice files generated by continuous recording. The files generated by trigger-based recording are video files generated in response to a trigger signal, and the slice files generated by continuous recording are slice files generated by dividing continuous recordings into slices of a preset duration.

[0007] Optionally, if the underlying video file is a triggered recording file, then generating the fusion event associated with the target video includes: The time range of the target video is determined as the playable time range of the target video; Generate a fusion event associated with the target event as a fusion event associated with the target video.

[0008] Optionally, if the underlying video file is a slice file generated from continuous video recording, then the generation of the fusion event associated with the target video includes: The intersection of the time range of the target event and the time range of the target video is determined as the playable time range of the target video; Generate a fusion event associated with the target event as a fusion event associated with the target video.

[0009] Optionally, if the type of the underlying video file is a slice file generated by splitting continuous video recordings, and the target event matches multiple target videos, then generating a fusion event associated with the target video includes: The intersection of the time range of the target event and the time range of each target video is determined as the playable time range of each target video; For each target video that matches the target event, a fusion event associated with the target event is generated as a fusion event associated with each target video.

[0010] Optionally, if the underlying video file is a triggered recording file, then before generating the fusion event associated with the target video, the method further includes: Determine whether at least one of the midpoint of the time range of the target video and the midpoint of the time range of the target event falls within the time range of the other. If so, then the step of generating a fusion event associated with the target video is performed.

[0011] Optionally, if the underlying video file is a file generated by triggered recording, the method further includes: After traversing the list of videos to be matched and the list of events to be matched, determine the unmatched videos in the list of videos to be matched that do not match any event; A placeholder event is generated for each of the unmatched videos; wherein the placeholder event carries event information of the event corresponding to the unmatched video, the time range of the unmatched video, and the original time range information of the underlying video file.

[0012] Optionally, the step of traversing the list of events to be matched and the list of videos to be matched, and outputting the target event and the target video as a matching pair when the time range of the target event overlaps with the time range of the target video, includes: Iterate through the list of events to be matched and the list of videos to be matched, and determine the current event to be matched and the current video to be matched during the iteration process; During the matching process between the current event to be matched and the current video to be matched, if the time range of the current event to be matched overlaps with the time range of the current video to be matched, then the current event to be matched and the current video to be matched are output as a matching pair, and the next event to be matched and the next video to be matched are determined to continue matching the next event to be matched and the next video to be matched; wherein, the overlap includes any of the following: the time range of the current video to be matched is completely contained within the time range of the current event to be matched, the end time of the current video to be matched falls within the time range of the current event to be matched, the start time of the current video to be matched falls within the time range of the current event to be matched, the time range of the current event to be matched is completely contained within the time range of the current video to be matched, and the endpoints of the time range of the current event to be matched and the time range of the current video to be matched are connected; When the time range of the current event to be matched does not overlap with the time range of the current video to be matched, the next event to be matched and the next video to be matched are directly determined to continue matching the next event to be matched and the next video to be matched.

[0013] Optionally, determining the next event to be matched and the next video to be matched includes: If the end time of the time range of the current video to be matched is later than the end time of the time range of the current event to be matched, the next event of the current event to be matched in the list of events to be matched is determined as the next event to be matched, and the current video to be matched is determined as the next video to be matched. If the end time of the time range of the current video to be matched is earlier than the end time of the time range of the current event to be matched, the next video of the current video to be matched in the list of videos to be matched is determined as the next video to be matched, and the current event to be matched is determined as the next event to be matched. If the end time of the time range of the current video to be matched is equal to the end time of the time range of the current event to be matched, the next event of the current event to be matched in the list of events to be matched is determined as the next event to be matched, and the next video of the current video to be matched in the list of videos to be matched is determined as the next video to be matched.

[0014] Optionally, obtaining the list of events to be matched includes: Obtain the original event list, record the original sorting direction of the events in the original event list, and sort the events in the original event list in chronological order to obtain the event list to be matched; Accordingly, after generating a fusion event associated with the target video based on the type of the underlying recording file corresponding to the target video, the process further includes: The fusion events are sorted according to the original sorting direction and then output.

[0015] To achieve the above objectives, this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the event information fusion method described above.

[0016] The event information fusion method provided in this application achieves accurate matching of events and videos while retaining playable information by using overlapping time ranges as the matching criterion. Since complex temporal relationships such as time offsets and partial overlaps between events and videos can be uniformly determined by whether their time ranges overlap, matching can be completed even if the timestamp endpoints of the event and video are not completely consistent, as long as their time ranges overlap. This avoids missed matches caused by inconsistencies in timestamp endpoints in related technologies, improving the robustness of event-video matching. Simultaneously, the fused event generated by this application carries event information, playable time range information, and the original time range information of the underlying video file. This allows the fused event to retain the semantic information of the event itself while also possessing the playable recording interval and the information needed to locate the underlying video file. Users can accurately locate and play the corresponding recording content based on the fused event, avoiding the problem in related technologies where the lack of a playable interval in the association result prevents users from playing back the video. Therefore, this application achieves accurate matching of events and videos while preserving playable information, even when there is a time offset or complex overlap between the events and videos, thus improving the robustness of the matching and ensuring the replayability of the merged events. This application also discloses an electronic device that can achieve the same technical effects.

[0017] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings: Figure 1 This is a structural diagram illustrating an event information fusion system according to an exemplary embodiment; Figure 2 This is a schematic diagram of a user interface for a playback display terminal according to an exemplary embodiment; Figure 3 This is a flowchart illustrating an event information fusion method according to an exemplary embodiment; Figure 4 This is a flowchart illustrating another event information fusion method according to an exemplary embodiment; Figure 5This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] The event information fusion system used in this application is described below. See also... Figure 1 A schematic diagram of the structure of an event information fusion system according to an exemplary embodiment is shown, such as... Figure 1 As shown, the system includes: acquisition equipment, storage equipment, cloud server, fusion processing equipment, and playback display terminal.

[0023] The acquisition device is used to acquire video streams from the monitored area. The acquisition device can be a network camera, dome camera, or other device with video acquisition capabilities; this application embodiment does not limit the specific type of acquisition device. A storage device is communicatively connected to the acquisition device and is used to store the video streams acquired by the acquisition device. The underlying recording files are stored in the storage device. The storage device can be located at the device end or at the edge, for example, it can be a built-in storage medium or an edge recorder. The underlying recording files are the original files of the video stream recordings, files generated by triggered recordings, and slice files generated by continuous recording segmentation are all stored in the storage device. A cloud server is communicatively connected to the acquisition device or storage device and is used to perform intelligent analysis on the acquired video streams, generate events carrying semantic information, and maintain an event list. The information carried by the events may include semantic fields such as human detection results, vehicle detection results, and risk levels. A fusion processing device is communicatively connected to both the storage device and the cloud server, and is used to obtain the event list and video list, and execute the event information fusion method provided in this application to generate fused events. The playback display terminal communicates with the fusion processing device to receive and display fusion events output by the device, enabling functions such as playback, card-based retrieval, and cloud-stored event lists. The playback display terminal can show the fused event list in chronological order. See also... Figure 2 , Figure 2 This is a schematic diagram of a user interface for a playback display terminal according to an exemplary embodiment. The interface lists multiple fused event entries in a timeline format. Each entry can display: the event occurrence time, a thumbnail / preview of the playable video associated with the fused event, event semantic tags (e.g., human figure, patrol, etc.), and the corresponding acquisition device identifier (e.g., c8s, c5l). Because the fused events simultaneously retain the cloud-based event semantic fields, the playable time range of the local video, and the original time range of the underlying video file, users can directly click on a fused event on this interface to search, filter, and verify playback, thus intuitively demonstrating the output effect of event information fusion.

[0024] This application discloses an event information fusion method that achieves accurate matching of events and videos while retaining playable information.

[0025] See Figure 3 A flowchart illustrating an event information fusion method according to an exemplary embodiment is shown below. Figure 3 As shown, it includes: S101: Obtain a list of events to be matched and a list of videos to be matched; wherein, the list of events to be matched includes multiple events sorted in chronological order, and the list of videos to be matched includes multiple videos sorted in chronological order, each video carrying the time range of the video and the original time range of the corresponding underlying recording file; The event list to be matched refers to the list of events to be matched with the video. These events originate from the analysis results generated by the cloud server after intelligent analysis of the videos collected by the device. Each event carries semantic fields such as event type, risk level, and confidence level. The video list to be matched refers to the list of local videos to be matched with the events. These videos originate from the device or edge storage. The video's time range refers to the start and end times of the video segment on the timeline. The original time range of the underlying recording file refers to the original start and end times (rawStartTimeSec and rawEndTimeSec) of the underlying recording file containing the video on the timeline. This original time range is used to locate the underlying recording file. The video's time range and the underlying recording file's original time range may be the same or different. For example, when the playable video segment is only a part of the underlying recording file, the video's time range is smaller than the underlying recording file's original time range.

[0026] In this step, based on the device identifier and the requested time window, the events of the device within the requested time window are obtained from the cloud server to form a list of events to be matched, and the local videos of the device within the requested time window are obtained from the device or edge storage to form a list of videos to be matched. Furthermore, after obtaining the list of videos to be matched, the validity of the videos can be verified, and only videos with both start and end times greater than 0 and a start time earlier than an end time are accepted. If the obtained list of videos to be matched is empty, an empty result is returned directly, and no further steps are executed.

[0027] As a possible implementation method, obtaining the list of events to be matched includes: obtaining an original event list, recording the original sorting direction of the events in the original event list, and sorting the events in the original event list in chronological order to obtain the list of events to be matched; The original event list refers to the list of events that have been obtained but not sorted. The original sorting direction refers to the order in which the events in the original event list were arranged when they were obtained, such as sorted in ascending or descending order of time.

[0028] In practical implementation, firstly, the original event list is obtained, and the original sorting direction of the events in the original event list is recorded. Then, the events in the original event list are sorted chronologically to obtain the event list to be matched, so that it can be compared in an orderly manner with the video list to be matched later. The video list to be matched is processed in a similar way, that is, the original sorting direction of the original video list is recorded, and the videos in the original video list are sorted chronologically. It can be seen that this implementation method, by recording the original sorting direction, provides a basis for subsequently outputting fused events according to the original sorting direction, ensuring that the output results are consistent with the interactive experience of the upper-layer business.

[0029] S102: Traverse the list of events to be matched and the list of videos to be matched. If the time range of the target event and the time range of the target video overlap, output the target event and the target video as a matching pair. Here, a target event refers to an event whose time range overlaps with that of a video during the current traversal, and a target video refers to a video whose time range overlaps with that of an event during the current traversal. Overlapping time ranges mean that the time ranges of the event and the video intersect on the timeline, that is, the time period corresponding to the event overlaps with the time period corresponding to the video. A matching pair refers to the pairing relationship between a target event and a target video that have overlapping time ranges, and one matching pair corresponds to one event and video to be merged.

[0030] In this step, the list of events to be matched and the list of videos to be matched are traversed, and each event is compared with a video one by one. When the time range of an event overlaps with the time range of a video, it indicates that the event and the video are related in time. The event is then identified as the target event and the video as the target video, and the two are output as a matching pair. Since the determination of overlapping time ranges does not require the event time and the video time endpoint to be strictly equal, a matching relationship can still be established even if there is a time offset between the event and the video, or if the event spans multiple video slices.

[0031] As can be seen, this step improves the robustness of event and video matching by replacing the matching method based on the overlap of time ranges in the existing technology, which is based on the matching method based on the equality of fixed timestamp endpoints, and avoids event loss due to time offset or cross-slice.

[0032] As a feasible implementation, the step of traversing the list of events to be matched and the list of videos to be matched, and determining that the target event and the target video are matched when their time ranges overlap, to obtain a matching pair of the target event and the target video, includes: traversing the list of events to be matched and the list of videos to be matched, and determining the current event to be matched and the current video to be matched during the traversal; during the matching process of the current event to be matched and the current video to be matched, if the time range of the current event to be matched overlaps with the time range of the current video to be matched, then outputting the current event to be matched and the current video to be matched as a matching pair, and determining the next event to be matched and the next video to be matched, so as to continue matching the next event to be matched... The process then proceeds to match the next video to be matched. The overlap includes any of the following: the time range of the current video to be matched is completely contained within the time range of the current event to be matched; the end time of the current video to be matched falls within the time range of the current event to be matched; the start time of the current video to be matched falls within the time range of the current event to be matched; the time range of the current event to be matched is completely contained within the time range of the current video to be matched; or the endpoints of the time ranges of the current event to be matched and the current video to be matched are connected. When the time ranges of the current event to be matched and the current video to be matched do not overlap, the next event to be matched and the next video to be matched are directly determined to continue matching the next event to be matched and the next video to be matched.

[0033] Here, the currently matched event refers to the event currently participating in the matching process during traversal, and the currently matched video refers to the video currently participating in the matching process during traversal. It should be noted that the overlap determination includes any of the following: the time range of the currently matched video is completely contained within the time range of the currently matched event, i.e., the start time of the currently matched video is not earlier than the start time of the currently matched event, and the end time of the currently matched video is not later than the end time of the currently matched event; the end time of the currently matched video falls within the time range of the currently matched event, i.e., the end time of the currently matched video is later than the start time of the currently matched event, but not later than the end time of the currently matched event; the start time of the currently matched video falls within the time range of the currently matched event, i.e., the start time of the currently matched video is later than the start time of the currently matched event, but not later than the end time of the currently matched event; the start time of the currently matched video falls within the time range of the currently matched event, i.e., the start time of the currently matched video is later than the start time of the currently matched event, and the end time of the currently matched video is later than the start time of the currently matched event. The start time of the current event to be matched is no earlier than the start time and earlier than the end time of the current event to be matched; the time range of the current event to be matched is completely contained within the time range of the current video to be matched, meaning the start time of the current event to be matched is no earlier than the start time of the current video to be matched, and the end time of the current event to be matched is no later than the end time of the current video to be matched; the time range of the current event to be matched and the time range of the current video to be matched coincide, meaning the end time of the current event to be matched is the same as the start time of the current video to be matched, or the end time of the current video to be matched is the same as the start time of the current event to be matched. If any of the above conditions are met, it indicates that the current event to be matched and the current video to be matched have temporal overlap, constituting a valid match, and the two will be output as a matching pair.

[0034] In practice, the system iterates through the list of events to be matched and the list of videos to be matched. During this process, the current event to be matched and the current video to be matched are determined. If the time ranges of the two events and videos overlap, they are output as a matching pair, and the next event to be matched and the next video to be matched are determined to continue the matching process. If the time ranges of the two events and videos do not overlap, it means that the current event to be matched and the current video to be matched are not related in time. The next event to be matched and the next video to be matched are determined to continue the matching process until the list of events to be matched or the list of videos to be matched has been traversed.

[0035] It should be noted that this implementation does not limit the implementation method of traversal. As one feasible implementation, a two-pointer traversal method can be used, that is, assigning event pointers and video pointers to the list of events to be matched and the list of videos to be matched, respectively. The event pointer points to the current event to be matched, and the video pointer points to the current video to be matched. The next event to be matched and the next video to be matched are determined by advancing the event pointer and / or the video pointer. As another feasible implementation method, a loop traversal method can also be used, that is, enumerating the events in the list of events to be matched one by one and comparing them with the videos in the list of videos to be matched one by one. It can be seen that this implementation outputs events and videos with overlapping times as matching pairs, and covers various overlapping forms such as inclusion, falling in, and endpoint conjunction. As long as there is a temporal intersection between the event and the video, a match is established, avoiding the missed matches caused by strict endpoint equality matching.

[0036] As one possible implementation, determining the next event to be matched and the next video to be matched includes: if the end time of the time range of the current video to be matched is later than the end time of the time range of the current event to be matched, determining the next event of the current event to be matched in the list of events to be matched as the next event to be matched, and determining the current video to be matched as the next video to be matched; if the end time of the time range of the current video to be matched is earlier than the end time of the time range of the current event to be matched, determining the next video of the current video to be matched in the list of videos to be matched as the next video to be matched, and determining the current event to be matched as the next event to be matched; if the end time of the time range of the current video to be matched is equal to the end time of the time range of the current event to be matched, determining the next event of the current event to be matched in the list of events to be matched as the next event to be matched, and determining the next video of the current video to be matched in the list of videos to be matched as the next video to be matched.

[0037] In practice, the next event to be matched and the next video to be matched are determined by comparing the end time of the time range of the current video to be matched with the end time of the time range of the current event to be matched. If the end time of the current video to be matched is later than the end time of the current event to be matched, it means the end time of the current event to be matched is earlier. The current event to be matched cannot overlap with any subsequent video in time. Therefore, the next event in the list of events to be matched is determined as the next event to be matched, and the current video to be matched is determined as the next video to be matched. If the end time of the current video to be matched is earlier than the end time of the current event to be matched, it means the end time of the current video to be matched is earlier. The current video to be matched cannot overlap with any subsequent event in time. Therefore, the next video in the list of videos to be matched is determined as the next video to be matched, and the current event to be matched is determined as the next event to be matched. If the end time of the current video to be matched is equal to the end time of the current event to be matched, it means both end simultaneously. The next event in the list of events to be matched is determined as the next event to be matched, and the next video in the list of videos to be matched is determined as the next video to be matched.

[0038] In the implementation using a two-pointer traversal, the process of determining the next event to be matched and the next video to be matched is essentially the process of advancing the event pointer and / or the video pointer: if the end time of the current video to be matched is later than the end time of the current event to be matched, only the event pointer is advanced; if the end time of the current video to be matched is earlier than the end time of the current event to be matched, only the video pointer is advanced; if the end times are equal, both the event pointer and the video pointer are advanced simultaneously. That is, let the time range of the current event to be matched be [eS, eE], and the time range of the current video to be matched be [vS, vE], then: if vE is greater than eE, advance the event pointer; if vE is less than eE, advance the video pointer; if vE equals eE, advance both the event pointer and the video pointer simultaneously. Since both the list of events to be matched and the list of videos to be matched are sorted in chronological order, and the pointer only moves forward and does not go back, each event and video to be matched is compared at most once. The complexity of the traversal process is O(n+m), where n is the number of events and m is the number of videos. This ensures that the matching process is linear and efficient, and that no possible matching event and video combinations are missed.

[0039] As can be seen, this implementation method determines the next event and video to be matched by comparing the end time, ensuring that no matching relationship with overlapping time is missed, while achieving linear and efficient traversal matching.

[0040] S103: Generate a fusion event associated with the target video based on the type of the underlying recording file corresponding to the target video; wherein the fusion event carries event information of the target event, playable time range information of the target video, and original time range information of the underlying recording file.

[0041] Among them, the fused event refers to a composite data structure that carries both event semantic fields and video playback interval information. It is the result data generated after binding event information with video playability information based on event and video matching. The playable time range information of the target video refers to the information of the time interval that can be played when the target video is played, which is used to determine the recording segment that can be played after the user clicks. The original time range information of the underlying recording file refers to the information of the original start and end time of the underlying recording file that carries the target video on the timeline, which is used to locate the underlying recording file.

[0042] In this step, a fusion event associated with the target video is generated based on the type of the underlying recording file corresponding to the target video. The fusion event carries the event information of the target event, the playable time range information of the target video, and the original time range information of the underlying recording file. This allows the fusion event to retain the semantic information of the target event while binding the playable time range of the target video and the original time range of the underlying recording file. Users can use the fusion event to perform semantic searches and filters, and directly locate and play the corresponding recording content. It should be noted that the method for determining the playable time range differs for different types of underlying recording files. This step only limits the generation of the fusion event; the specific determination method is described in subsequent implementation methods.

[0043] As can be seen, this step generates a fused event that carries event information, playable time range information, and the original time range information of the underlying video file, thereby binding event semantics with playable video information and providing a data foundation for event playback verification.

[0044] As a feasible implementation, after generating the fusion event associated with the target video based on the type of the underlying recording file corresponding to the target video, the method further includes: sorting each of the fusion events according to the original sorting direction and then outputting them.

[0045] In practical implementation, each fused event is sorted according to its original sorting direction before being output. The original sorting direction is the direction recorded when the original event list was obtained. Since fused events are generated by copying target events, restoring the sorting of fused events based on their corresponding original sorting directions ensures that the output result is consistent with the interaction order when the upper-layer business logic was passed in. Therefore, this implementation method ensures consistency with the interaction experience of the upper-layer business logic by restoring the output order according to the original sorting direction.

[0046] The event information fusion method provided in this application uses the overlap of time ranges as the matching criterion between events and videos to achieve accurate matching of events and videos while retaining playable information. Since complex time relationships such as time offsets and partial overlaps between events and videos can be uniformly determined by whether their time ranges overlap, even if the timestamp endpoints of the event and video are not completely consistent, matching can be completed as long as their time ranges overlap. This avoids missed matching caused by inconsistencies in timestamp endpoints in related technologies, improving the robustness of event-video matching. Simultaneously, the fused event generated by this application carries event information, playable time range information, and the original time range information of the underlying video file. This allows the fused event to retain the semantic information of the event itself while also possessing the playable recording interval and the information needed to locate the underlying video file. Users can accurately locate and play the corresponding recording content based on the fused event, avoiding the problem in related technologies where the lack of a playable interval in the association result prevents users from playing back the video. Therefore, this application embodiment achieves accurate matching of events and videos while retaining playable information when there is a time offset or complex overlap between the events and videos, thus improving the robustness of the matching and ensuring the replayability of the fused events.

[0047] This application discloses an event information fusion method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically: See Figure 4 A flowchart illustrating another event information fusion method according to an exemplary embodiment, such as... Figure 4 As shown, it includes: S201: Obtain a list of events to be matched and a list of videos to be matched; wherein, the list of events to be matched includes multiple events sorted in chronological order, and the list of videos to be matched includes multiple videos sorted in chronological order, each video carrying the time range of the video and the original time range of the corresponding underlying recording file; S202: Traverse the list of events to be matched and the list of videos to be matched. If the time range of the target event and the time range of the target video overlap, output the target event and the target video as a matching pair. S203: Determine the type of the underlying video file according to the generation method of the underlying video file corresponding to the target video; wherein, the type of the underlying video file includes files generated by trigger-based recording and slice files generated by continuous recording. The files generated by trigger-based recording are video files generated in response to a trigger signal, and the slice files generated by continuous recording are slice files generated by dividing continuous recordings into slices of a preset duration. Among them, trigger-based recording files refer to files generated when recording is initiated by triggering conditions. Trigger signals can include event triggering, alarm triggering, manual triggering, etc. For example, when the device detects a moving target, it triggers recording, and the resulting recording file is a trigger-based recording file. Such files usually correspond to an independent playable recording segment, and the time range of the video is usually consistent with the original time range of the underlying recording file. Slice files generated by continuous recording segmentation refer to files generated after dividing continuous recording into equal-length segments according to a preset duration. For example, 7×24-hour continuous recording is divided into slice files every 5 minutes. The time range of such files is consistent with the original time range of the underlying recording file, but an event may span multiple slice files.

[0048] In this step, the type of the underlying recording file is determined based on how it was generated. This information is then used to determine the playable time range and generate fusion events in different ways depending on the type of the underlying recording file. Therefore, this step, by determining the type of the underlying recording file, provides a basis for generating fusion events in different ways for different types.

[0049] S204: If the type of the underlying video file is a file generated by trigger-based recording, then the time range of the target video is determined as the playable time range of the target video, and a fusion event associated with the target event is generated as a fusion event associated with the target video; wherein, the fusion event carries the event information of the target event, the playable time range information of the target video, and the original time range information of the underlying video file; In this step, when the underlying recording file is a triggered recording file, the target event is copied to obtain the fused event. The reason for copying the target event instead of directly modifying it is that one event may match multiple videos. To avoid the target event being overwritten by multiple videos during one-to-many mapping, the target event needs to be copied so that each matching video corresponds to an independent fused event. Since this type of file is generated by a trigger signal to start recording, it itself corresponds to a complete playable recording segment. The time range of the target video is the complete playable interval. Therefore, the time range of the target video is directly determined as the playable time range of the target video. The event information of the target event is written into the event field of the fused event. The playable time range information of the target video and the original time range information of the underlying recording file are written into the video information (videoInfo) field of the fused event. The video information may include the device identifier, playable start and end times, event type, and the original start and end times of the underlying recording file: rawStartTimeSec and rawEndTimeSec.

[0050] As can be seen, this step directly uses the time range of the target video as the playable time range, enabling the fusion event generated in the triggered recording scenario to accurately locate the playable recording segment, thus ensuring the reliability of event playback verification.

[0051] As a feasible implementation, if the type of the underlying video file is a triggered recording file, before determining the time range of the target video as the playable time range of the target video and generating a fusion event associated with the target event as a fusion event associated with the target video, the method further includes: determining whether at least one of the midpoint of the time range of the target video and the midpoint of the time range of the target event falls within the other's time range; if so, then performing the step of determining the time range of the target video as the playable time range of the target video and generating a fusion event associated with the target event as a fusion event associated with the target video.

[0052] The midpoint of the target video's time range refers to the middle moment of the target video's time range, while the midpoint of the target event's time range refers to the middle moment of the target event's time range.

[0053] In practice, for cases where the underlying recording file corresponding to the target video is a triggered recording file, a midpoint filtering check is performed before generating the fusion event: First, it is determined whether the midpoint of the target video's time range falls within the target event's time range. If not, before the target video is matched, it is determined whether the midpoint of the target event's time range falls within the target video's time range. If the midpoint of the target video's time range falls within the target event's time range, or vice versa, it indicates that the event and video have substantial time overlap, and the step of generating the fusion event is executed. If neither midpoint falls within the other's time range, it indicates that the event and video only briefly overlap at the boundary, which is a mismatch, and no binding relationship is established. The process continues to traverse the list of events to be matched and the list of videos to be matched.

[0054] As can be seen, this implementation method eliminates mismatches caused by events and videos only briefly overlapping at the boundary through midpoint filtering, further improving the accuracy of matching.

[0055] As a feasible implementation, if the underlying video file is a triggered recording file, this embodiment further includes: after traversing the list of videos to be matched and the list of events to be matched, determining the unmatched videos in the list of videos to be matched that do not match any event; generating a placeholder event for each unmatched video; wherein the placeholder event carries event information of the event corresponding to the unmatched video, the time range of the unmatched video, and the original time range information of the underlying video file.

[0056] Among them, unmatched videos refer to videos in the list of videos to be matched that do not match any event, that is, videos that have not been identified as any target videos after traversal; placeholder events refer to standardized events generated for unmatched videos to fill the blanks in the list. Their event identifier (eid) is empty, the event time is the start time of the unmatched video, the event type list adopts the original valid event types of the local video, the cover image is empty, and the video information is bound to the unmatched video itself.

[0057] In practice, after traversing the list of videos to be matched and the list of events to be matched, unmatched videos in the list that do not match any events are identified. A placeholder event is generated for each unmatched video, carrying the event information of the corresponding event, the time range of the unmatched video, and the original time range information of the underlying recording file. Because cloud events may be missing due to network jitter, model omissions, etc., some local videos may lack corresponding events. If these videos are discarded directly, the user interface will display empty spaces where local videos exist but no corresponding events. By generating placeholder events for unmatched videos, this gap can be eliminated, allowing users to see and click to play these recordings. It should be noted that a missing video compensation switch (needMissedVideo) can also be set for unmatched videos. When the missing video compensation switch is on, a placeholder event is generated for the unmatched video; when the missing video compensation switch is off, no placeholder event is generated, and only the log of the unmatched video is recorded.

[0058] As can be seen, this implementation eliminates the empty display of local videos in the user interface without corresponding events by generating placeholder events for unmatched videos, thus preventing users from mistakenly judging that there are no alarm records on that day. At the same time, it retains the location information of the underlying video file, ensuring that placeholder events can also be played.

[0059] S205: If the type of the underlying video file is a slice file generated by continuous video segmentation, then the intersection of the time range of the target event and the time range of the target video is determined as the playable time range of the target video, and a fusion event associated with the target event is generated as a fusion event associated with the target video; wherein, the fusion event carries the event information of the target event, the playable time range information of the target video, and the original time range information of the underlying video file.

[0060] In this step, when the underlying video file is a slice file generated from continuous video recording, the target event is copied to obtain the fusion event. Since continuous video recording is divided into multiple slice files according to a preset duration, and the target video is only one slice, the time range of the target event may only cover a portion of that slice. Therefore, the intersection of the target event's time range and the target video's time range—that is, the time period covered by both the time period corresponding to the target event and the time period corresponding to the target video—is determined as the playable time range of the target video. The event information of the target event, the playable time range information of the target video, and the original time range information of the underlying video file are then written into the fusion event. Here, the playable time range is the time period corresponding to the aforementioned intersection, and the original time range is the original start and end time of the slice file on the timeline. For example, if the target event's time range is 10:02 to 10:08, and the target video is a segment file with a time range of 10:00 to 10:05, and the intersection of the two is 10:02 to 10:05, then the playable time range of the target video is 10:02 to 10:05. When the user clicks play, the segment file will start playing from 10:02.

[0061] As can be seen, this step generates a precise playable time range by intersecting and cropping, while preserving the original time range of the underlying video file. This achieves both precise event location and playback, and ensures that the underlying video file can be located, thus avoiding erroneous truncation or file location failure during playback.

[0062] As a feasible implementation, if the type of the underlying video file is a slice file generated by continuous video segmentation, and the target event matches multiple target videos, then the intersection of the time range of the target event and the time range of each target video is determined as the playable time range of each target video; for each target video that matches the target event, a fusion event associated with the target event is generated as a fusion event associated with each target video.

[0063] In practical implementation, when a target event matches multiple target videos (i.e., when the time range of the target event spans multiple consecutive video clips), the intersection of the time range of the target event and the time range of each target video is determined as the playable time range of each target video. For each target video that matches the target event, a fusion event associated with the target event is output as the fusion event associated with each target video. For example, if the time range of the target event spans from 10:02 to 10:08, and it overlaps with clips with time ranges from 10:00 to 10:05 and 10:05 to 10:10, then two fusion events are generated, with playable time ranges of 10:02 to 10:05 and 10:05 to 10:08, respectively. The business event identifiers of each fusion event remain consistent, and the list item identifier is appended with the time boundary of the corresponding clip.

[0064] As can be seen, this implementation generates an independent fusion event for each matching slice, allowing a single event to be associated with multiple consecutive slices, thus achieving complete coverage of cross-slice events. During playback, each slice is located sequentially according to the time boundary, ensuring complete playback of cross-slice events.

[0065] Therefore, this embodiment determines the type of the underlying video file. For files generated by trigger-based recording, the time range of the target video is directly determined as the playable time range. For slice files generated by segmenting continuous recording, the intersection of the time range of the target event and the time range of the target video is determined as the playable time range. The original time range of the underlying video file is always preserved, ensuring that the playable interval in trigger-based recording scenarios is the complete video interval, and the playable interval in continuous recording scenarios is a precise sub-interval. Compared to the previous embodiment, this embodiment, while achieving accurate matching of events and videos and preserving playable information, provides differentiated fusion processing for different recording types, making it more applicable.

[0066] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 5 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 5 As shown, the electronic device includes: Communication interface 1 enables information exchange with other devices, such as network devices; Processor 2 is connected to communication interface 1 to enable information interaction with other devices and to implement the event information fusion method provided by one or more of the above technical solutions when executing computer programs.

[0067] Memory 3 stores the aforementioned computer program.

[0068] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 5 The general will label all buses as Bus System 4.

[0069] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0070] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0071] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0072] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0073] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An event information fusion method, characterized in that, include: Obtain a list of events to be matched and a list of videos to be matched; wherein, the list of events to be matched includes multiple events sorted in chronological order, and the list of videos to be matched includes multiple videos sorted in chronological order, each video carrying the time range of the video and the original time range of the corresponding underlying recording file; Traverse the list of events to be matched and the list of videos to be matched. If the time range of the target event and the time range of the target video overlap, output the target event and the target video as a matching pair. Based on the type of the underlying recording file corresponding to the target video, a fusion event associated with the target video is generated; wherein, the fusion event carries event information of the target event, playable time range information of the target video, and original time range information of the underlying recording file.

2. The event information fusion method according to claim 1, characterized in that, Before generating a fusion event associated with the target video based on the type of the underlying recording file corresponding to the target video, the method further includes: The type of the underlying video file is determined based on the generation method of the underlying video file corresponding to the target video; wherein, the type of the underlying video file includes files generated by trigger-based recording and slice files generated by continuous recording. The files generated by trigger-based recording are video files generated in response to a trigger signal, and the slice files generated by continuous recording are slice files generated by dividing continuous recordings into slices of a preset duration.

3. The event information fusion method according to claim 2, characterized in that, If the underlying video file is a triggered recording file, then the generation of the fusion event associated with the target video includes: The time range of the target video is determined as the playable time range of the target video; Generate a fusion event associated with the target event as a fusion event associated with the target video.

4. The event information fusion method according to claim 2, characterized in that, If the underlying video file is a slice file generated from continuous video recording, then the generation of the fusion event associated with the target video includes: The intersection of the time range of the target event and the time range of the target video is determined as the playable time range of the target video; Generate a fusion event associated with the target event as a fusion event associated with the target video.

5. The event information fusion method according to claim 2, characterized in that, If the underlying video file is a slice file generated from continuous video segmentation, and the target event matches multiple target videos, then generating a fusion event associated with the target video includes: The intersection of the time range of the target event and the time range of each target video is determined as the playable time range of each target video; For each target video that matches the target event, a fusion event associated with the target event is generated as a fusion event associated with each target video.

6. The event information fusion method according to claim 2, characterized in that, If the underlying video file is a triggered recording file, then before generating the fusion event associated with the target video, the process further includes: Determine whether at least one of the midpoint of the time range of the target video and the midpoint of the time range of the target event falls within the time range of the other. If so, then the step of generating a fusion event associated with the target video is performed.

7. The event information fusion method according to claim 2, characterized in that, If the underlying video file is a file generated by trigger-based recording, then the method further includes: After traversing the list of videos to be matched and the list of events to be matched, determine the unmatched videos in the list of videos to be matched that do not match any event; A placeholder event is generated for each of the unmatched videos; wherein the placeholder event carries event information of the event corresponding to the unmatched video, the time range of the unmatched video, and the original time range information of the underlying video file.

8. The event information fusion method according to claim 1, characterized in that, The step of traversing the list of events to be matched and the list of videos to be matched, and outputting the target event and the target video as a matching pair when the time range of the target event and the time range of the target video overlap, includes: Iterate through the list of events to be matched and the list of videos to be matched, and determine the current event to be matched and the current video to be matched during the iteration process; During the matching process between the current event to be matched and the current video to be matched, if the time range of the current event to be matched overlaps with the time range of the current video to be matched, then the current event to be matched and the current video to be matched are output as a matching pair, and the next event to be matched and the next video to be matched are determined to continue matching the next event to be matched and the next video to be matched; wherein, the overlap includes any of the following: the time range of the current video to be matched is completely contained within the time range of the current event to be matched, the end time of the current video to be matched falls within the time range of the current event to be matched, the start time of the current video to be matched falls within the time range of the current event to be matched, the time range of the current event to be matched is completely contained within the time range of the current video to be matched, and the endpoints of the time range of the current event to be matched and the time range of the current video to be matched are connected; When the time range of the current event to be matched does not overlap with the time range of the current video to be matched, the next event to be matched and the next video to be matched are directly determined to continue matching the next event to be matched and the next video to be matched. The step of determining the next event to be matched and the next video to be matched includes: If the end time of the time range of the current video to be matched is later than the end time of the time range of the current event to be matched, the next event of the current event to be matched in the list of events to be matched is determined as the next event to be matched, and the current video to be matched is determined as the next video to be matched. If the end time of the time range of the current video to be matched is earlier than the end time of the time range of the current event to be matched, the next video of the current video to be matched in the list of videos to be matched is determined as the next video to be matched, and the current event to be matched is determined as the next event to be matched. If the end time of the time range of the current video to be matched is equal to the end time of the time range of the current event to be matched, the next event of the current event to be matched in the list of events to be matched is determined as the next event to be matched, and the next video of the current video to be matched in the list of videos to be matched is determined as the next video to be matched.

9. The event information fusion method according to claim 1, characterized in that, The process of obtaining the list of events to be matched includes: Obtain the original event list, record the original sorting direction of the events in the original event list, and sort the events in the original event list in chronological order to obtain the event list to be matched; Accordingly, after generating a fusion event associated with the target video based on the type of the underlying recording file corresponding to the target video, the process further includes: The fusion events are sorted according to the original sorting direction and then output.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the event information fusion method as described in any one of claims 1 to 9.