Video event aggregation method and apparatus, device and storage medium

Abstract

Provided in the present disclosure are a video event aggregation method and apparatus, a device and a storage medium. The method comprises: acquiring video data within a target time period and event data associated therewith; and, on the basis of the video data and the event data, generating a timeline within a container view and video bars on the timeline, wherein different styles of video bars correspond to different event types, and the event types are obtained by aggregating the event data. In the solution of the present disclosure, on the basis of video data within a target time period and event data associated therewith, a timeline within a container view and video bars on the timeline can be generated, and different event types can be represented by different styles of video bars, thus facilitating quick viewing of video pictures corresponding to event types of interest by means of video bars.

Description

Video event aggregation methods, apparatus, devices, and storage media Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of surveillance technology, event listening technology, and video processing technology. Background Technology

[0002] When using surveillance cameras, the common display method is to show the video data captured by the camera in a 9x9 or 12x1 grid. This display method cannot quickly identify videos where special events have occurred, and it takes a considerable amount of time to filter out the content the user wants to see when reviewing the video. Summary of the Invention

[0003] This disclosure provides video event aggregation methods, apparatus, devices, and storage media to solve or alleviate one or more technical problems in the prior art.

[0004] Firstly, this disclosure provides a video event aggregation method, including:

[0005] Acquire video data and associated event data within a target time period;

[0006] Generate a timeline in the container view and video bars on the timeline based on the video data and the event data;

[0007] Different styles of video bars correspond to different event types, and the event types are obtained based on the aggregation of the event data.

[0008] Secondly, this disclosure provides a video event aggregation device, comprising:

[0009] The acquisition module is used to acquire video data and its associated event data within a target time period;

[0010] The generation module is used to generate a timeline in the container view and video bars on the timeline based on the video data and the event data;

[0011] Different styles of video bars correspond to different event types, and the event types are obtained based on the aggregation of the event data.

[0012] Thirdly, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] The memory is communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0016] Fourthly, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of the present disclosure.

[0017] Fifthly, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of the present disclosure.

[0018] The beneficial effects of the technical solution provided in this disclosure include at least the following: video data within a target time period and its associated event data, which can generate a timeline in a container view and video bars on that timeline, different event types can be represented by video bars of different styles, and the video bars facilitate quick viewing of video footage corresponding to the event type of interest.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments provided according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0021] Figure 1 is a flowchart illustrating a video event aggregation method according to an embodiment of the present disclosure;

[0022] Figure 2 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure;

[0023] Figure 3 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure;

[0024] Figure 4 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure;

[0025] Figure 5 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure;

[0026] Figure 6 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure;

[0027] Figure 7 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure;

[0028] Figure 8 is a schematic diagram of a user interface according to an embodiment of the present disclosure;

[0029] Figure 9 is a schematic flowchart of a data request according to an embodiment of the present disclosure;

[0030] Figure 10 is a schematic diagram of drawing the boundary of an interface area according to an embodiment of the present disclosure;

[0031] Figure 11 is a schematic flowchart of processing event data according to an embodiment of the present disclosure;

[0032] Figure 12 is a schematic diagram of a set of scale events according to an embodiment of the present disclosure;

[0033] Figure 13 is a schematic diagram of the time axis region according to an embodiment of the present disclosure;

[0034] Figure 14 is a schematic diagram of the structure of a video event aggregation device according to an embodiment of the present disclosure;

[0035] Figure 15 is a structural schematic diagram of a video event aggregation device according to another embodiment of the present disclosure;

[0036] Figure 16 is a block diagram of an electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0037] The present disclosure will now be described in further detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0038] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0039] Figure 1 is a flowchart illustrating a video event aggregation method according to an embodiment of the present disclosure. In one embodiment, the method includes:

[0040] S101. Obtain video data and related event data within the target time period;

[0041] S102. Generate a timeline in the container view and video bars on the timeline based on the video data and the event data;

[0042] Different styles of video bars correspond to different event types, and the event types are obtained based on the aggregation of the event data.

[0043] In this embodiment of the disclosure, the camera can generate video data and associated event data during video recording. For example, in a surveillance scenario, a smart network camera can capture video data within the monitored area and generate event data based on events occurring during the recording process. Video data is typically continuous over a period of time, while event data may be discrete over a period of time. For example, within a day, a face recognition event might occur during time period A, a person moving event during time period B, and a baby crying event during time period C. The target time period can be flexibly selected according to actual needs, such as a day, 12 hours, 6 hours, 3 hours, etc.

[0044] In this embodiment of the disclosure, a container view can be generated around the video frame generated from the video data, such as below or to the right. Then, a timeline and video bars on that timeline are generated within this container view based on the video data and event data. The timeline can be parallel to the video frame, perpendicular to the video frame, or have other positional relationships with the video frame. If the timeline is perpendicular to the video frame, it can be called a vertical timeline. If the timeline is parallel to the video frame, it can be called a horizontal timeline.

[0045] In this embodiment of the disclosure, the event data associated with the video data may include multiple types of events. Event types can be obtained by aggregating the event data. For example, within a day, the event data associated with the video data may include 100 events. Among them, 50 events are "person moving," 10 events are "face recognition," 20 events are "person coughing," and 20 events are "loud noise." The event types that can be clustered include: "person moving," "face recognition," "person coughing," and "loud noise." Different event types can be represented using different styles of video bars. For example, video bars of different colors represent different event types. Furthermore, video bars of different shapes represent different event types. Video bars can be called video blocks, video event bars, video event blocks, etc.

[0046] Video data within the target time period and its associated event data can generate a timeline and video bars on that timeline in the container view. Different styles of video bars can represent different event types, and the video bars make it easy to quickly view the video footage corresponding to the event type of interest.

[0047] Figure 2 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure. The method may include one or more features of the video event aggregation method described above. In one embodiment, step S101 involves acquiring video data and associated event data within a target time period, including:

[0048] S201. Request the video data from the end time to the start time of the target time period in reverse order;

[0049] S202, request the event data associated with the video data from the end time to the start time of the target time period in reverse order.

[0050] In this embodiment, video data and event data can be requested in reverse chronological order, that is, the most recently generated video data and event data from the camera can be requested first. The data generated by the camera can be initially stored in a memory such as a hard drive, memory card, or cloud storage. Since video data and event data can be stored separately, video data and its associated event data can be requested in reverse chronological order. When a video bar needs to be drawn, video data and event data for the target time period can be requested to be read from the memory into a cache for processing. For example, if the target time period is the most recent 12 hours, the video data and event data of the last 12 hours stored in the memory can be read backwards from the end time of the latest data until all 12 hours of data have been read. That is, reading stops after reaching the start time of the last 12 hours of data. As another example, if the target time period is from 8:00 AM to 12:00 PM, reading can start from 12:00 PM and proceed backwards until the data from 8:00 AM is read. As yet another example, if the target time period is a specific day, reading can start from 11:59:59 PM of that day and proceed backwards until the data from 12:00 AM to 12:00 AM is read.

[0051] In one implementation, requesting the video data from the end time to the start time of the target time period in reverse order includes:

[0052] If the process of requesting video data or its associated event data in reverse order is interrupted, the process of requesting the video data or its associated event data in reverse order from the time of interruption to the time of start continues.

[0053] In the case of requesting video data or its associated event data at the aforementioned start time, the process of requesting video data or its associated event data in reverse order is completed.

[0054] In this embodiment of the disclosure, the process of requesting data in reverse order may be interrupted. In this case, data can be requested backward from the moment of interruption until all the data required for the target time period has been read. For example, if the process of requesting video data in reverse order is interrupted, the process of requesting video data from the moment of interruption to the moment of resumption continues in reverse order; if the video data at the moment of resumption is obtained, the process of requesting video data or its associated event data in reverse order is completed. As another example, if the process of requesting event data in reverse order is interrupted, the process of requesting event data from the moment of interruption to the moment of resumption continues in reverse order; if the event data at the moment of resumption is obtained, the process of requesting event data in reverse order is completed. Requesting data in reverse order facilitates the acquisition of more recently generated video data and event data, allows for more timely display of video bars corresponding to newer events, and makes it easier to view events that occurred more recently.

[0055] In this embodiment of the disclosure, the process of requesting data in reverse order may occur at one or more interrupted time periods. In this case, data can be requested backward from the beginning of one or more interrupted time periods until all the data required for the target time period has been read.

[0056] Figure 3 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure. The method may include one or more features of the video event aggregation method described above. In one embodiment, step S102 generates a timeline in a container view and video bars on the timeline based on the video data and the event data, including:

[0057] S301. Target video data and target event data are obtained by filtering from the video data and the event data according to the filtering conditions; wherein, the filtering conditions include at least one of the display time period and the display event type;

[0058] S302. Generate the timeline in the container view based on the first target video data and the first target event data obtained by filtering from the video data and the event data according to the usage display time period;

[0059] S303. Generate a video bar on the timeline in the container view based on the second target video data and the second target event data obtained by filtering from the video data and the event data according to the type of display event.

[0060] In this embodiment of the disclosure, the video duration and number of events within the target time period may be large. Filtering conditions can be used to display content of greater user interest in the container view. Video data meeting the filtering conditions can be considered target video data, and event data meeting the filtering conditions can be considered event data. From all the read data, a subset of more relevant data is selected. Based on the selected data, a timeline and video bars on that timeline are generated, displaying event types and video footage of greater user interest. For example, if the target time period is one day, the filtering conditions include a display period from 12:00 to 2:00. Video data from 12:00 to 2:00 and its associated event data can be filtered from the data of one day. Based on the filtered data, a timeline from 12:00 to 2:00 is displayed in the container view, and video bars corresponding to all event types within this period are generated on that timeline. As another example, if the target time period is 12 hours, the filtering conditions include a display period from 1:00 to 2:00, and the event type is "person moving." Video data from 1:00 to 2:00 and event data related to "person moving" can be filtered from the data of one day. Based on the filtered data, the system displays a timeline from 12:00 to 2:00 in the container view and generates video bars corresponding to events where someone moves during this time on the timeline.

[0061] In this embodiment, the filtering criteria may include a display time period and a display event type. The display event period may be determined by the user through dragging commands in the container view and unit time scales; the display time period may represent the time period containing content of interest to the user. The display event type may include preset display event types and user-defined display event types; the display event type may represent the event type corresponding to the content of interest to the user. Based on the display time period, first target video data and first target event data within that time period can be obtained, and a timeline corresponding to that display time period can be generated based on the first target video data and first target event data. Based on the display event type, second target video data and second target event data within a display time period can be obtained, and a video axis on the timeline corresponding to the display time period can be generated based on the second target video data and second target event data.

[0062] Figure 4 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure. The method may include one or more features of the video event aggregation method described above. In one embodiment, the method further includes:

[0063] S401. Determine the boundary of the container view based on the time of the playback marker, the unit time scale, and the height of the container view. The boundary of the container view includes the top boundary time and / or the bottom boundary time.

[0064] In this embodiment, the playback marker's time can include the currently selected reference time, which determines the display time period. The unit time scale can represent the displayed time scale divisions and can be determined according to the scaling ratio of the container view. The container view's height can represent the height of the container on the display page and can be arbitrarily set according to requirements. The top and bottom boundary times of the container view can be obtained by calculating based on the playback marker's time, the unit time scale, and the container view's height. One calculation method is as follows: topTimestamp = currentTime + viewHeight / 9 * oneUnitWidthTime bottomTimestamp = currentTime - (viewHeight - viewHeight / 9) * oneUnitWidthTime

[0065] Among them, topTimestamp can represent the top boundary time of the container view (maximum drawable top boundary time), bottomTimestamp can represent the bottom boundary time of the container view (maximum drawable bottom boundary time), viewHeight can represent the height of the container view, oneUnitWidthTime can represent the milliseconds occupied by a unit time scale, the interval value is different at different scaling ratios, and currentTime can represent the time of the play mark.

[0066] In one implementation, the event data includes multiple events, the video data includes multiple video segments, one event corresponds to one video segment, and the timeline includes one or more sub-timelines; generating the timeline in the container view based on first target video data and first target event data filtered from the video data and the event data using a display time period includes:

[0067] Based on the height of the container view, the boundary of the container view, the start and end times of the first target video data corresponding to the first target event data, and the unit time scale, determine the first directional boundary of the sub-time axis of the video segment corresponding to the current event;

[0068] The time axis is generated based on the first directional boundary of the one or more scale grids, the width of the time axis, and the second directional boundary of the time axis.

[0069] In this embodiment, the timeline may include one or more sub-timelines. A sub-timeline can represent an event and its corresponding video, such as a baby crying and its corresponding video, someone coughing and its corresponding video, no event and its corresponding video, etc. Several sub-timelines can form a single timeline. The boundary of the length direction (first direction) of the timeline can be calculated based on the height of the container view, the boundary of the container view, the start and end times of the first target video data corresponding to the first target event data event, and the unit time scale. One method for calculating a sub-timeline is as follows: drawStartY = viewHeight - (startTime - bottomTimestamp) / oneUnitWidthTime drawEndY = viewHeight - (endTime - bottomTimestamp) / oneUnitWidthTime allArea[type].push({startY:drawStartY,endY:drawEndY})

[0070] Here, `drawStartY` represents the first boundary of the first direction of the timeline, and `drawEndY` represents the second boundary of the first direction of the timeline. `viewHeight`, `oneUnitWidthTime`, and `bottomTimestamp` have the same meaning as in the previous example and will not be repeated here. `startTime` represents the start time of the first target video, and `endTime` represents the end time of the first target video. The `[]` in `allArea` can store the data of the object to be drawn on the timeline (e.g., the first target event). `type` represents the event type; here it is the default, indicating that the event type is no event or an empty event.

[0071] Specifically, based on the corresponding events and video durations of the sub-timelines, the length direction boundaries of the sub-timelines are generated. Then, based on the aforementioned length direction boundaries, the width of the timeline, the width direction (second direction) boundaries of the timeline, and the style of the timeline, a timeline comprising several sub-timelines can be generated. In one approach, several sub-timelines can be generated first, and then a complete timeline can be generated based on the sub-timelines.

[0072] For example, for a vertical timeline, based on the left and right widths of the timeline, the coordinates of the right boundary of the timeline, and the duration of the corresponding timeline, several vertical sub-timelines can be generated first. For example, the shape of the sub-timelines can be a small rectangle. Then, the generated sub-timelines can be combined sequentially in chronological order to obtain the vertical timeline.

[0073] For example, for a horizontal timeline, based on the duration of the sub-timelines, the vertical width of the timeline, and the coordinates of the lower boundary of the timeline, several horizontal sub-timelines can be generated first. For example, the shape of the sub-timelines can be a small rectangle. Then, the generated sub-timelines can be combined sequentially in chronological order to obtain the horizontal timeline.

[0074] In one implementation, generating a video bar on the timeline in the container view based on second target video data and second target event data obtained by filtering from the video data and the event data using display event types includes:

[0075] Based on the start and end times of the second target video data corresponding to the event in the second target event data, the height of the container view, the boundary of the container view, and the unit time scale, determine the first directional boundary of the video bar corresponding to the event in the second target event data;

[0076] Based on the first directional boundary, the second directional boundary, and the width of the video bar, as well as the color corresponding to the event type of the video bar, the video bar within the scale grid is generated.

[0077] In this embodiment of the disclosure, second target event data and corresponding second target video data can be obtained by filtering from a target time period or a display time period according to the display event type. The display event type may include a preset display event type or a user-defined display event type. Based on the events and videos corresponding to the content that users are more interested in, filtered according to the display event type, the length direction (first direction) boundary of the video bar can be calculated based on the start and end times of the video corresponding to the second target event, the height of the container view, the bottom boundary of the container view, and the unit time scale. One method of calculating the video bar is similar to the method of calculating the sub-timeline, but the meaning of the parameters in the formula is different, as shown in the following example: drawStartY = viewHeight - (startTime - bottomTimestamp) / oneUnitWidthTime drawEndY = viewHeight - (endTime - bottomTimestamp) / oneUnitWidthTime allArea[type].push({startY:drawStartY,endY:drawEndY})

[0078] Here, `drawStartY` can represent the first boundary of the first direction of the video axis, and `drawEndY` can represent the second boundary of the first direction of the video axis. `viewHeight`, `oneUnitWidthTime`, and `bottomTimestamp` have the same meaning as in the example above, and will not be repeated here. `startTime` can represent the start time of the second target video, `endTime` can represent the end time of the second target video, `allArea`'s `[]` can store the object data of the video axis to be drawn (such as the second target event), and `type` can represent the event type, which is the event type of the second target event here.

[0079] In this embodiment, video bars are generated based on their corresponding second target event data. Since the second target event data generally includes multiple data points and is discretely distributed, the corresponding video bars are also generally multiple discretely distributed data points. Based on the first directional boundary, the second directional boundary, and the width of the video bar, as well as the color corresponding to the event type of the video bar, several video bars can be generated on the timeline.

[0080] For example, if video bars are generated on a vertical timeline, several vertical video bars can be generated based on the left and right widths of the video bars, the coordinates of the right edge of the video bars, and the duration of the video bars. For example, the shape of the video bars can be a small rectangle.

[0081] For example, if video bars are generated on a horizontal timeline, several horizontal video bars can be generated based on the duration of the video bar, the vertical width of the video bar, and the coordinates of the lower boundary of the video bar. For example, the shape of the sub-timeline can be a small rectangle.

[0082] In one implementation, different event types correspond to different colors, and the color corresponding to the event type is used to determine the color of the video bar and / or prompt information corresponding to the event type.

[0083] In this embodiment of the disclosure, event types can be pre-aggregated based on stored event data. Priority configuration can be applied to the aggregated event types, either based on presets or user-defined configurations. Different styles can be configured for different event types. These styles can be used to generate timelines, video bars, and prompts, and can include colors, shapes, formats, etc.

[0084] Figure 5 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure. This method may include one or more features of the video event aggregation method described above. In one embodiment, it further includes:

[0085] S501. Generate event type prompt information corresponding to the video bar based on the event type.

[0086] In this embodiment of the disclosure, the event type prompt information may include an event aggregation bubble. An event aggregation bubble can be generated based on the event type corresponding to the second target event data filtered by the display time period and display time type. The event aggregation bubble can be used to prompt the user about the event types included in the video bar. The event aggregation bubble can be displayed as a bubble style including multiple event icons. One event icon corresponds to one event type. The color of the event icon for the same event type is the same as the color of the video bar.

[0087] Figure 6 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure. This method may include one or more features of the video event aggregation method described above. In one embodiment, S501, generating event type prompt information corresponding to the video bar based on the event type includes:

[0088] S601. Generate a playback marker line and an aggregated bubble at the playback marker position according to the event type; wherein, the aggregated bubble is used to represent event type prompt information, the event type prompt information includes event type icons of different colors, the color of the event type icon is the same as the color of the video bar corresponding to the event type, and the aggregated bubble displays a first number of event type icons;

[0089] S602. Display the time information of the playback marker position on one side of the time axis; wherein the style of the time information of the playback marker position is different from that of the time scale.

[0090] In this embodiment, a play marker line can be generated based on the location of the play marker. In one example, the play marker can be positioned at a specific location at one end of the container view. For example, a dashed line can be generated at that specific location as the play marker line. A prompt message, such as an aggregated bubble, can be generated based on the location of the play marker. In one example, filtered second target event data can be obtained, and the icon style corresponding to each event type can be determined based on the event type and its priority, and then displayed in the aggregated bubble. When the number of event types exceeds the first number that the aggregated bubble can display, icons corresponding to the first number of higher-priority event types are displayed first.

[0091] In this embodiment of the disclosure, the duration of a unit time scale can be determined based on the scaling ratio of the container view, and a scale grid and time scale information can be generated based on the duration of the unit time scale and the time of the playback marker. The scale grid includes the division of the displayed event segments, such as 10 segments, 20 segments, etc.; the time scale information includes the time information displayed on one side of the time axis. For example, if the duration of a unit time scale is 5 minutes and the time of the playback marker is 15:00, then the time scale information can be 14:55, 14:55, 14:45, 14:40, etc., and so on.

[0092] Figure 7 is a flowchart illustrating a video event aggregation method according to another embodiment of the present disclosure. The method may include one or more features of the video event aggregation method described above. In one embodiment, the method further includes:

[0093] S701. Generate a video jump button based on the video bar; wherein, the control button includes a previous event video jump button and / or a next event video jump button.

[0094] In this embodiment of the disclosure, video jump buttons can be generated at specific locations in the container view based on the generated video bars. The video jump buttons can be used to jump between videos corresponding to the video bars. The video jump buttons can include a previous event video jump button and a next event video jump button. The previous event video jump button can be used to jump to the video preceding the current event video. The next event video jump button can be used to jump to the video following the current event video.

[0095] In one implementation, the method further includes at least one of the following:

[0096] In response to a zoom command on the timeline, the display time period is adjusted, and at least one of the timeline, video bar, event type prompts, and video jump button within the container view is redrawn based on the adjusted display time period;

[0097] In response to a drag command on the timeline, adjust the playback markers and redraw at least one of the timeline, video bar, event type prompts, and video jump buttons within the container view based on the adjusted playback markers;

[0098] In response to a click command on the video jump button, jump to play the video corresponding to the previous or next event of the current event;

[0099] In response to a click command for event type prompts, a pop-up information box is drawn. Within the pop-up information box, some event types and their icons are displayed according to their priority, while others are hidden.

[0100] In response to a drag command on the inner area of ​​the pop-up information box, the hidden event type and event type icon in the pop-up information box are displayed.

[0101] In this embodiment, scaling the container view changes its scaling ratio, which in turn changes the unit time scale, ultimately altering the displayed time period. When the displayed time period changes, the corresponding event and video data can be read from the cache, and the steps described above can be repeated to generate a new timeline, video bar, event type prompts, and video jump button within the container view. Dragging the timeline changes the time of the playback marker, thus changing the displayed time period. Similarly, when the displayed time period changes, the corresponding event and video data can be read from the cache, and the steps described above can be repeated to generate a new timeline, video bar, event type prompts, and video jump button within the container view.

[0102] In this embodiment of the disclosure, clicking on a prompt message, such as a bubble, can display a bubble pop-up in the container view. Event types and their corresponding icons can be generated based on the event type corresponding to the second target event data and the icon style corresponding to the event type. If the total number of event types is greater than the second number, the second number of event types and their corresponding icons are displayed according to priority, while other undisplayed event types are hidden. When the bubble pop-up is displayed, if a click operation is detected outside the container view area of ​​the bubble pop-up, the bubble pop-up is closed. When the bubble pop-up is displayed, if a drag operation is detected within the bubble pop-up, the hidden event types and their corresponding icons can be displayed.

[0103] In one application scenario, video event aggregation methods can include a vertical timeline interaction method with bubble aggregation functionality. In this method, after entering the video display page, the user sees a vertical timeline on the screen, with various events marked by different colors and icons. Related events are automatically aggregated into bubbles near their corresponding time points. When the user clicks a bubble, it expands to display details of all included events. Click commands are received via buttons at the bottom to switch between videos. Furthermore, users can adjust the display density of the timeline using methods such as two-finger zoom to view information within specific time periods in greater detail. The following provides a detailed explanation of this method.

[0104] I. User Interface

[0105] As shown in Figure 8, the interface mainly includes a player section, date and event filters, and a timeline. The timeline itself includes a vertical timeline with aggregated bubbles, comprising a container view, playback markers, a time scale, a vertical video bar, aggregated bubbles, and bottom toggle buttons. Details are as follows:

[0106] Container view 801 is used to store timeline content and supports online scrolling to switch playback time, gesture zooming, and changing the scale precision.

[0107] Playback marker 802 is a bold mark on the selected time point or the playback ruler, accompanied by a playback divider line. It is used to mark the current playback time or the time point selected by the user while dragging. The time on the left is highlighted and updates in real time during playback.

[0108] The 803 time scale has each node representing a specific point in time. The precision is determined by the zoom level; for example, a 1-minute scale means the time span between two ticks is 1 minute. As the user drags, the zoom dynamically displays different content.

[0109] The vertical video bar is 804. Gray blocks represent video data, and different colors represent different events. It supports event priority, with higher priority events displayed first. It also supports event filtering, and the video bar is drawn differently depending on the filter option.

[0110] Bubble 805 indicates which events are present in the scale cell. Selecting a time point highlights the bubble, and clicking expands the bubble to view event details. Users can drag and zoom to aggregate, show, and hide the bubbles.

[0111] The bottom toggle button 806 allows you to switch between the previous and next videos.

[0112] II. Timeline Data

[0113] Due to the independence of the modules, video data and event data are separated. The data acquisition process is shown in Figure 9. Video data and event data are acquired and processed in parallel, and the returned data is in reverse order, that is, the most recently generated video data is returned first. The request start time is 0:00 on the request day, and the end time is the last second of the request day (23:59:59).

[0114] As shown in Figure 9, the data request process includes the following steps:

[0115] S901. Prepare request conditions. Request conditions may include video request conditions and event request conditions, and may also include the target time period in which the requested data is located. When requesting data, video request conditions and event request conditions can be generated separately to query video data and event data. Alternatively, a combined request condition that can query both video data and event data can be generated.

[0116] S902. Request and process data. Data can be requested based on request conditions. For example, video data for a target time period can be requested based on video request conditions, and event data for a target time period can be requested based on event request conditions. Alternatively, video data and event data can be requested in parallel based on comprehensive request conditions. The requested data can also be processed to obtain processed video data and event data.

[0117] S903. Determine whether the data retrieval for a certain day has been completed. If the retrieval is complete, end the request; if the retrieval is not complete, execute S904.

[0118] S904, Request parameters changed. Adjust the target time period in the request conditions based on the start time of the time period that was not retrieved this time. After adjustment, return to execute S902.

[0119] III. Event Priority Control

[0120] This module is used to control the priority of events and manage the priority of each event. When there are multiple events in a video, the events are displayed in color according to their priority.

[0121] IV. UI Design Process

[0122] The general process of designing a UI includes the following steps:

[0123] Step 1: Define the boundaries of the drawing area

[0124] Figure 10 shows the drawing boundaries of the interface area. `topTimestamp` is defined as the maximum drawable time point at the top, and `bottomTimestamp` is the minimum drawable time point at the bottom. `topTimestamp` and `bottomTimestamp` dynamically change depending on the currently selected time `currentTime`. It's important to note that the higher up the timeline, the more recent the corresponding event or video. `topTimestamp = currentTime + viewHeight / 9 * oneUnitWidthTime` `bottomTimestamp = currentTime - (viewHeight - viewHeight / 9) * oneUnitWidthTime`

[0125] Here, viewHeight represents the height of the view, oneUnitWidthTime represents the number of milliseconds occupied by the height of a video unit (the value varies depending on the scaling ratio), and currentTime represents the currently selected time point.

[0126] Step 2: Process video timeline data. `drawStartY = viewHeight - (startTime - bottomTimestamp) / oneUnitWidthTime` `drawEndY = viewHeight - (endTime - bottomTimestamp) / oneUnitWidthTime` `allArea[type].push({startY:drawStartY,endY:drawEndY})`

[0127] Here, viewHeight represents the height of the view, oneUnitWidthTime represents the number of milliseconds occupied by the video unit height (the value varies depending on the scaling ratio), startTime represents the video start time, endTime represents the video end time, allArea represents the object data storing the timeline to be drawn, and type represents the event type (default here, no event).

[0128] Step 3: Process event data to generate bubbles

[0129] As shown in Figure 11, the steps include:

[0130] S1101, Traverse the event data.

[0131] S1102. Determine if the event is out of bounds. If yes, execute S1101 and iterate to the next event data; otherwise, execute S1103.

[0132] S1103. Locate the video containing the incident.

[0133] S1104, Add event drawing.

[0134] S1105. Add an event to the bubble scale data.

[0135] S1106. Determine if the data traversal is complete. If yes, end the traversal; otherwise, execute S1101 to traverse to the next event data.

[0136] The video bar is processed in the same way as the timeline in step two, except for the event type, drawing position, and height.

[0137] The scale event set is used to draw aggregated bubbles. The following is an example of a scale spanning 10 minutes, as shown in Figure 12.

[0138] Step 4: Drawing

[0139] The drawing includes drawing the timeline area, selection indicator, ruler and bubble, and bottom button.

[0140] 1. Timeline Region Drawing: A rectangle was drawn using Scalable Vector Graphics (SVG) to depict a two-dimensional graphic, and its interior color was filled. The drawing method for a certain strip is shown in Figure 13. Here, x represents the x-coordinate of the right side of the timeline strip, startY represents the vertical start coordinate calculated from the video time in step two, endY represents the vertical end coordinate calculated from the video time in step two, d represents the width of the timeline strip, and path is the closed rectangular path enclosed by the four coordinates in the above figure.

[0141] The following is a method for drawing rectangular timeline color bars:<Shape d={path}fill={areaColors[item]}strokeWidth={1}key={item} / >

[0142] Here, Shape represents the shape, and in this example, a rectangle will be drawn. d represents the closed path to be drawn, path represents the video or event area, fill represents the color of the closed path, areaColors[item] represents the color of a specific event, stokeWidth represents the width of the line to be drawn, and key represents the key. Each shape is different.

[0143] 2. Select the indicator, including the highlight time and the horizontal line.

[0144] 3. Ruler and bubble.

[0145] It includes scales, scale time, and bubbles. The bubbles contain the set of events that exist within the current scale cell.

[0146] 4. Bottom button.

[0147] The two buttons on the left and right can receive click events.

[0148] V. Timeline Interaction and Response

[0149] 1. Drag

[0150] The timeline supports dragging. During dragging, the time bubble, time scale, video area bar, and currently selected time will all change as you drag. After dragging to a certain position and releasing, the video at the current time point will be retrieved and started playing.

[0151] 2. Scaling

[0152] It supports two-finger zoom. When you zoom in with two fingers, the time span becomes smaller, allowing you to accurately view events at a specific point in time and their surroundings. When you zoom out with two fingers, the time span becomes larger, allowing you to quickly view which events occurred within a certain period. The event collection within the bubble changes as you zoom in and out.

[0153] 3. Button Click

[0154] Click the button on the left to switch to the previous video. Click the button on the right to switch to the next video.

[0155] 4. Dynamic Indicators

[0156] The selected time indicator dynamically changes as the video plays and the user drags it, with the bubble next to the currently selected time highlighted and enlarged.

[0157] 5. Bubble Click

[0158] When there are too many events in a single cell, the extra events will be hidden. Clicking the bubble will bring up a pop-up window that displays all and specific events, including the event icon and event name.

[0159] 6. Bubble pop-up

[0160] Clicking the bubble will bring up a pop-up window. Clicking other locations, scrolling the timeline, clicking the current bubble again, or playing the video to other ticks will all cause the pop-up window to disappear and be hidden.

[0161] 7. Scrolling pop-up

[0162] The pop-up speech bubble can be viewed by scrolling up and down.

[0163] Figure 14 is a schematic diagram of a video event aggregation device according to an embodiment of the present disclosure. In one embodiment, the device includes:

[0164] The acquisition module 1401 is used to acquire video data and its associated event data within a target time period;

[0165] The generation module 1402 is used to generate a timeline in the container view and video bars on the timeline based on the video data and the event data;

[0166] Different styles of video bars correspond to different event types, and the event types are obtained based on the aggregation of the event data.

[0167] In one embodiment, the acquisition module 1401 is further configured to:

[0168] The video data is requested in reverse order from the end time to the start time of the target time period;

[0169] The request retrieves event data associated with the video data from the end time to the start time of the target time period in reverse order.

[0170] In one implementation, requesting the video data from the end time to the start time of the target time period in reverse order includes:

[0171] If the process of requesting video data or its associated event data in reverse order is interrupted, the process of requesting the video data or its associated event data in reverse order from the time of interruption to the time of start continues.

[0172] In the case of requesting video data or its associated event data at the aforementioned start time, the process of requesting video data or its associated event data in reverse order is completed.

[0173] In one embodiment, the generation module 1402 is further configured to:

[0174] Target video data and target event data are obtained by filtering the video data and event data according to the filtering criteria; wherein, the filtering criteria include at least one of the display time period and the display event type;

[0175] The timeline in the container view is generated based on the first target video data and the first target event data obtained by filtering from the video data and the event data during the display period.

[0176] The video bar on the timeline in the container view is generated based on the second target video data and the second target event data obtained by filtering from the video data and the event data using the event type.

[0177] Figure 15 is a schematic diagram of a video event aggregation apparatus according to another embodiment of the present disclosure. The apparatus may include one or more features of the video event aggregation apparatus described above. In one embodiment, the apparatus further includes:

[0178] The container view determination module 1501 is used to determine the boundary of the container view based on the time of the playback marker, the unit time scale, and the height of the container view. The boundary of the container view includes the top boundary time and / or the bottom boundary time.

[0179] In one implementation, the event data includes multiple events, the video data includes multiple video segments, one event corresponds to one video segment, and the timeline includes one or more sub-timelines; generating the timeline in the container view based on first target video data and first target event data filtered from the video data and the event data using a display time period includes:

[0180] Based on the height of the container view, the boundary of the container view, the start and end times of the first target video data corresponding to the first target event data, and the unit time scale, determine the first directional boundary of the sub-time axis of the video segment corresponding to the current event;

[0181] The time axis is generated based on the first directional boundary of the one or more scale grids, the width of the time axis, and the second directional boundary of the time axis.

[0182] In one implementation, generating a video bar on the timeline in the container view based on second target video data and second target event data obtained by filtering from the video data and the event data using display event types includes:

[0183] Based on the start and end times of the second target video data corresponding to the event in the second target event data, the height of the container view, the boundary of the container view, and the unit time scale, determine the first directional boundary of the video bar corresponding to the event in the second target event data;

[0184] Based on the first directional boundary, the second directional boundary, and the width of the video bar, as well as the color corresponding to the event type of the video bar, the video bar within the scale grid is generated.

[0185] In one implementation, different event types correspond to different colors, and the color corresponding to the event type is used to determine the color of the video bar and / or prompt information corresponding to the event type.

[0186] In one embodiment, as shown in FIG15, the device further includes:

[0187] The prompt information generation module 1502 is used to generate event type prompt information corresponding to the video bar according to the event type.

[0188] In one embodiment, the prompt information generation module 1502 is further configured to:

[0189] Based on the event type, a playback marker line and an aggregated bubble are generated at the playback marker position; wherein, the aggregated bubble is used to represent event type prompt information, the event type prompt information includes event type icons of different colors, the color of the event type icon is the same as the color of the video bar corresponding to the event type, and the aggregated bubble displays a first number of event type icons;

[0190] The time information of the playback marker position is displayed on one side of the time axis; wherein the style of the time information of the playback marker position is different from that of the time scale.

[0191] In one embodiment, as shown in FIG15, the device further includes:

[0192] The jump button generation module 1503 is used to generate video jump buttons based on the video bar; wherein, the control buttons include a previous event video jump button and / or a next event video jump button.

[0193] In one embodiment, the device further includes at least one of the following:

[0194] In response to a zoom command on the timeline, the display time period is adjusted, and at least one of the timeline, video bar, event type prompts, and video jump button within the container view is redrawn based on the adjusted display time period;

[0195] In response to a drag command on the timeline, adjust the playback markers and redraw at least one of the timeline, video bar, event type prompts, and video jump buttons within the container view based on the adjusted playback markers;

[0196] In response to a click command on the video jump button, jump to play the video corresponding to the previous or next event of the current event;

[0197] In response to a click command for event type prompts, a pop-up information box is drawn. Within the pop-up information box, some event types and their icons are displayed according to their priority, while others are hidden.

[0198] In response to a drag command on the inner area of ​​the pop-up information box, the hidden event type and event type icon in the pop-up information box are displayed.

[0199] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0200] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0201] Figure 16 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in Figure 16, the electronic device includes a memory 1610 and a processor 1620. The memory 1610 stores a computer program that can run on the processor 1620. The number of memories 1610 and processors 1620 can be one or more. The memory 1610 can store one or more computer programs, which, when executed by the electronic device, cause the electronic device to perform the method provided in the above-described method embodiments. The electronic device may further include a communication interface 1630 for communicating with external devices and performing data exchange and transmission.

[0202] If the memory 1610, processor 1620, and communication interface 1630 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in Figure 16, but this does not indicate that there is only one bus or one type of bus.

[0203] Optionally, in a specific implementation, if the memory 1610, processor 1620 and communication interface 1630 are integrated on a single chip, the memory 1610, processor 1620 and communication interface 1630 can communicate with each other through an internal interface.

[0204] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0205] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAMBUS RAM (DR RAM).

[0206] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, Bluetooth, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)). It is worth noting that the computer-readable storage media mentioned in this disclosure can be non-volatile storage media; in other words, it can be non-transient storage media.

[0207] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0208] In the description of the embodiments of this disclosure, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0209] In the description of the embodiments disclosed herein, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0210] In the description of embodiments of this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0211] The above description is merely an exemplary embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A video event aggregation method, comprising: Acquire video data and associated event data within a target time period; Generate a timeline in the container view and video bars on the timeline based on the video data and the event data; Different styles of video bars correspond to different event types, and the event types are obtained based on the aggregation of the event data.

2. The method according to claim 1, wherein, Acquire video data and its associated event data within the target time period, including: The video data is requested in reverse order from the end time to the start time of the target time period; The request retrieves event data associated with the video data from the end time to the start time of the target time period in reverse order.

3. The method according to claim 2, wherein, The reverse order request for the video data from the end time to the start time of the target time period includes: If the process of requesting video data or its associated event data in reverse order is interrupted, the process of requesting the video data or its associated event data in reverse order from the time of interruption to the time of start continues. In the case of requesting video data or its associated event data at the aforementioned start time, the process of requesting video data or its associated event data in reverse order is completed.

4. The method according to claim 1, wherein, Generating a timeline in the container view and video bars on the timeline based on the video data and the event data includes: Target video data and target event data are obtained by filtering the video data and event data according to the filtering criteria; wherein, the filtering criteria include at least one of the display time period and the display event type; The timeline in the container view is generated based on the first target video data and the first target event data obtained by filtering from the video data and the event data during the display period. The video bar on the timeline in the container view is generated based on the second target video data and the second target event data obtained by filtering from the video data and the event data using the display event type.

5. The method according to claim 4, wherein, The method further includes: The boundaries of the container view are determined based on the playback marker's time, the unit time scale, and the height of the container view. The boundaries of the container view include the top boundary time and / or the bottom boundary time.

6. The method according to claim 4 or 5, wherein, The event data includes multiple events, the video data includes multiple video segments, one event corresponds to one video segment, and the timeline includes one or more sub-timelines; generating the timeline in the container view based on first target video data and first target event data obtained by filtering from the video data and the event data using a display time period includes: Based on the height of the container view, the boundary of the container view, the start and end times of the first target video data corresponding to the first target event data, and the unit time scale, determine the first directional boundary of the sub-time axis of the video segment corresponding to the current event; The time axis is generated based on the first directional boundary of the one or more scale grids, the width of the time axis, and the second directional boundary of the time axis.

7. The method according to claim 6, wherein, Based on the second target video data and second target event data obtained by filtering from the video data and event data using the display event type, a video bar on the timeline in the container view is generated, including: Based on the start and end times of the second target video data corresponding to the event in the second target event data, the height of the container view, the boundary of the container view, and the unit time scale, determine the first directional boundary of the video bar corresponding to the event in the second target event data; Based on the first directional boundary, the second directional boundary, and the width of the video bar, as well as the color corresponding to the event type of the video bar, a video bar within the scale grid is generated.

8. The method according to any one of claims 1-7, wherein, Different event types correspond to different colors, and the color corresponding to the event type is used to determine the color of the video bar and / or prompt information corresponding to the event type.

9. The method according to any one of claims 1-8, further comprising: Based on the event type, generate event type prompt information corresponding to the video bar.

10. The method according to claim 9, wherein, Based on the event type, generate event type prompt information corresponding to the video bar, including: Based on the event type, a playback marker line and an aggregated bubble are generated at the playback marker position; wherein, the aggregated bubble is used to represent event type prompt information, the event type prompt information includes event type icons of different colors, the color of the event type icon is the same as the color of the video bar corresponding to the event type, and the aggregated bubble displays a first number of event type icons; The time information of the playback marker position is displayed on one side of the time axis; wherein the style of the time information of the playback marker position is different from that of the time scale.

11. The method according to any one of claims 1-10, further comprising: Based on the video bar, a video jump button is generated; wherein, the video jump button includes a previous event video jump button and / or a next event video jump button.

12. The method according to any one of claims 1-11, further comprising at least one of the following: In response to a zoom command on the timeline, the display time period is adjusted, and at least one of the timeline, video bar, event type prompts, and video jump button within the container view is redrawn based on the adjusted display time period; In response to a drag command on the timeline, adjust the playback markers and redraw at least one of the timeline, video bar, event type prompts, and video jump buttons within the container view based on the adjusted playback markers; In response to a click command on the video jump button, jump to play the video corresponding to the previous or next event of the current event; In response to a click command for event type prompts, a pop-up information box is drawn. Within the pop-up information box, some event types and their icons are displayed according to their priority, while others are hidden. In response to a drag command on the inner area of ​​the pop-up information box, the hidden event type and event type icon in the pop-up information box are displayed.

13. A video event aggregation device, comprising: The acquisition module is used to acquire video data and its associated event data within a target time period; The generation module is used to generate a timeline in the container view and video bars on the timeline based on the video data and the event data; Different styles of video bars correspond to different event types, and the event types are obtained based on the aggregation of the event data.

14. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-12.

Images