Video annotation method and apparatus, storage medium, electronic device, and program product
By using a timeline to synchronously play and annotate videos from multiple perspectives, the problem of low efficiency in perspective video annotation in existing technologies is solved, achieving efficient, accurate, and consistent annotation.
Patent Information
- Application Number
- PCT/CN2026/096267
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-16
- Filing Date
- 2026-05-11
- Publication Date
- 2026-11-19
AI Technical Summary
Existing video annotation schemes require users to annotate videos from each viewpoint separately, resulting in repetitive work and low efficiency, and making it difficult to guarantee the consistency and accuracy of video annotation results from different viewpoints.
By synchronously playing videos acquired from multiple different perspectives using a timeline, and responding to annotation commands triggered by the target timeline, video segments are identified and attribute information is annotated, thus achieving synchronous annotation of videos from multiple perspectives.
It improves the efficiency, accuracy, and consistency of annotation for videos from multiple perspectives, and further enhances annotation efficiency by batch annotating video clips.
Smart Images

Figure CN2026096267_19112026_PF_FP_ABST
Abstract
Description
Video annotation methods, devices, storage media, electronic devices and software products
[0001] This disclosure claims priority to Chinese Patent Application No. CN202510637741.9, filed with the State Intellectual Property Office of China on May 16, 2025, entitled "Video Annotation Method, Apparatus, Storage Medium, Electronic Device and Program Product", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This disclosure relates to data annotation technology in the field of intelligent driving, and in particular to a video annotation method, apparatus, storage medium, electronic device and program product. Background Technology
[0003] During the operation of intelligent driving vehicles, in order to obtain accurate information about the vehicle's surrounding environment, visual sensors with different perspectives are installed on the intelligent vehicle to collect video data. Elements in the videos collected by the visual sensors from different perspectives are then labeled to provide accurate data for downstream tasks such as target behavior prediction, target trajectory prediction, and vehicle path planning. Under current technology, existing video labeling schemes require users to label videos from each perspective separately, leading to repetitive work and low efficiency. Furthermore, the labeling results from different perspectives need to be manually synchronized by the user, further increasing the user's workload and the probability of errors, making it difficult to guarantee the consistency and accuracy of the labeling results from multiple perspectives. Summary of the Invention
[0004] This disclosure provides a video annotation method, apparatus, storage medium, electronic device, and program product, which enables simultaneous annotation of videos acquired synchronously from multiple different perspectives using a timeline, thereby improving the efficiency, accuracy, and consistency of annotating videos from multiple perspectives.
[0005] According to one aspect of the present disclosure, a video annotation method is provided, comprising: during the synchronous playback of multiple videos of the same scene simultaneously captured from different perspectives, in response to receiving an annotation instruction triggered by a target timeline corresponding to a target element contained in multiple videos through a visual interactive interface, determining video segments in the multiple videos that correspond to the first frame information and the second frame information based on the first frame information and the second frame information corresponding to the target timeline, wherein the annotation instruction includes attribute information of the target element; and annotating the target element in each frame of the video segment with attribute information to obtain the annotated video segment.
[0006] According to another aspect of the present disclosure, a video annotation apparatus is provided, comprising: a visual interactive interface module for synchronously playing videos of the same scene captured simultaneously from multiple different perspectives and interacting with a user; a video segment determination module for, during the synchronous playback of videos of the same scene captured simultaneously from multiple different perspectives, responding to an annotation instruction triggered by a target timeline corresponding to a target element contained in multiple videos received through the visual interactive interface, determining a video segment corresponding to the first frame information and the second frame information in the multiple videos based on the first frame information and the second frame information corresponding to the target timeline, wherein the annotation instruction includes attribute information of the target element; and an annotation module for annotating the attribute information of the target element in each frame of the video segment to obtain the annotation information of the target element in the video segment.
[0007] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program, the computer program instructions of which, when executed, implement the above-described video annotation method.
[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: a memory for storing a computer program product; and a processor for executing the computer program product stored in the memory. The processor is configured to: during the synchronous playback of videos of the same scene simultaneously captured from multiple different perspectives, in response to receiving a labeling instruction triggered by a target timeline corresponding to a target element contained in multiple videos through a visual interactive interface, determine video segments in the multiple videos corresponding to the first frame information and the second frame information based on first frame information and second frame information corresponding to the target timeline, wherein the labeling instruction includes attribute information of the target element; and label the target element in each frame of the video segment with attribute information to obtain a labeled video segment.
[0009] According to another aspect of the present disclosure, a computer program product is provided, including computer program instructions that, when executed by a processor, implement the above-described video annotation method.
[0010] Based on the embodiments of this disclosure, during the synchronous playback of videos of the same scene simultaneously captured from multiple different perspectives, if a labeling instruction triggered by a target timeline corresponding to a target element contained in multiple videos is received through a visual interactive interface, the video segment corresponding to the first and second frame information in the multiple videos can be determined based on the first frame information and the second frame information corresponding to the target timeline. Then, the target element attribute information of each frame in the video segment is labeled, resulting in a labeled video segment. The technical solution provided by this disclosure utilizes a timeline to simultaneously label videos simultaneously captured from multiple different perspectives, which helps improve the efficiency, accuracy, and consistency of labeling videos from multiple perspectives. Furthermore, based on the first and second frame information corresponding to the target timeline, batch labeling of each frame in the video segment determined by the first and second frame information is achieved, further improving the video labeling efficiency. Attached Figure Description
[0011] Figure 1 is a structural diagram of an electronic device provided in an exemplary embodiment of the present disclosure;
[0012] Figure 2 is a flowchart illustrating a video annotation method provided in an exemplary embodiment of this disclosure;
[0013] Figure 3 is a schematic diagram of video clip annotation provided in an exemplary embodiment of this disclosure;
[0014] Figure 4 is a schematic diagram of an interface for synchronously playing multiple videos provided in an exemplary embodiment of this disclosure;
[0015] Figure 5 is a schematic diagram of an interface for synchronously playing multiple videos provided in another exemplary embodiment of this disclosure;
[0016] Figure 6 is a schematic diagram of the process of configuring the timeline in a video annotation method provided by an exemplary embodiment of this disclosure;
[0017] Figure 7 is a flowchart illustrating a video annotation method provided in another exemplary embodiment of this disclosure;
[0018] Figure 8 is a schematic diagram of the process for verifying video annotation information provided in an exemplary embodiment of this disclosure;
[0019] Figure 9 is a schematic diagram of the structure of a video annotation device provided in an exemplary embodiment of the present disclosure;
[0020] Figure 10 is a schematic diagram of the structure of a video annotation device provided in another exemplary embodiment of this disclosure. Detailed Implementation
[0021] Example embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. It is obvious that the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure, and it should be understood that this disclosure is not limited to the example embodiments described herein.
[0022] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of this disclosure.
[0023] Application Overview
[0024] In the field of autonomous driving, it is crucial to annotate elements in video or image sequences acquired by visual sensors from different perspectives, as this can provide accurate data for downstream tasks such as target behavior prediction, target trajectory prediction, and vehicle path planning.
[0025] Existing video annotation schemes require users to annotate video or image sequences captured from each viewpoint separately, resulting in repetitive work and low efficiency. Furthermore, it is difficult to ensure the consistency of annotation results for video or image sequences from different viewpoints. In addition, existing video annotation schemes make it difficult to achieve synchronous playback of videos from different viewpoints. Therefore, the annotation process lacks comparison of video or image sequences from different viewpoints, which further increases the probability of errors when users annotate video or image sequences from different viewpoints, making it difficult to guarantee the consistency and accuracy of annotation results for video or image sequences from multiple viewpoints.
[0026] Exemplary device
[0027] Figure 1 illustrates an electronic device to which the video annotation method of the present disclosure can be applied.
[0028] As shown in Figure 1, the electronic device includes at least one processor 11 and a memory 12.
[0029] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 14 to perform desired functions.
[0030] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the video annotation methods and / or other desired functions of the various embodiments of this disclosure described above. Referring to FIG2, the processor may be configured to: during the synchronous playback of videos of the same scene simultaneously acquired from multiple different perspectives, in response to receiving annotation instructions triggered by a target timeline corresponding to a target element contained in multiple videos through a visual interactive interface, determine video segments corresponding to the first frame information and the second frame information in the multiple videos based on the first frame information and the second frame information corresponding to the target timeline, wherein the annotation instructions include attribute information of the target element; and annotate the attribute information of the target element in each frame of the video segment to obtain the annotated video segment.
[0031] In one example, the electronic device may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).
[0032] The input device 13 may also include, for example, a keyboard, a mouse, etc.
[0033] The output device 14 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.
[0034] Of course, for simplicity, Figure 1 only shows a portion of the components in the electronic device that are relevant to this disclosure, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.
[0035] It should be noted that the electronic devices in this disclosed technical solution can be various electronic devices, including but not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers.
[0036] In some optional implementations, the specific process by which the processor 11 synchronously plays videos of the same scene captured from multiple different perspectives can be configured as follows: playing videos of the same scene captured from multiple different perspectives simultaneously in multiple video playback areas of the visual interactive interface in a grid mode; or playing videos of the same scene captured from multiple different perspectives simultaneously in multiple video playback areas of the visual interactive interface in a window mode. The multiple video playback areas include a main playback area and secondary playback areas. In the main playback area, videos from different perspectives can be switched and played based on user-triggered video switching commands. In the secondary playback areas, other videos besides those played in the main playback area are played.
[0037] In some optional implementations, before the processor 11 synchronously plays videos of the same scene captured from multiple different perspectives, it can also be configured to: receive a timeline configuration instruction triggered by a visual interactive interface; determine the timeline to be displayed in the annotation area of the visual interactive interface based on the timeline configuration instruction, wherein at least one timeline is used to implement the operation of annotating target elements in multiple videos; and display the timeline in the annotation area of the visual interactive interface.
[0038] In some optional implementations, the specific process by which the processor 11 receives annotation instructions triggered by the target timeline corresponding to the target element contained in multiple videos through a visual interactive interface can be configured as follows: in response to receiving an annotation operation triggered by the target timeline corresponding to the target element, displaying an annotation information setting window in the annotation area of the visual interactive interface; receiving attribute information input by the user through the annotation information setting window; and in response to receiving a confirmation instruction indicating that the attribute information setting is complete, confirming that the annotation instruction has been received.
[0039] In some optional implementations, after obtaining the labeled video segments, the processor 11 can also be configured to: in response to receiving a confirmation instruction via a visual interactive interface that the target elements of multiple videos have been labeled, verify the labeling information corresponding to the target elements of the multiple videos; in response to the verification result indicating a verification failure, generate a prompt message, which carries the labeling information that failed the verification; and output the prompt message so that the user can modify the labeling information that failed the verification based on the prompt message.
[0040] In some optional implementations, after outputting the prompt information, the processor 11 can also be configured to: obtain the video segment corresponding to the verification failure annotation information carried in the prompt information; replay the video segment corresponding to the verification failure annotation information; and, in response to receiving an annotation information modification operation sent through the annotation area of the visual interactive interface, perform a modification operation on the verification failure annotation information based on the annotation information modification operation to obtain the annotation information of the video segment corresponding to the verification failure annotation information, wherein the modification operation includes modifying at least one of the first frame information, the second frame information, or the attribute information.
[0041] In some optional implementations, after outputting the prompt information, the processor 11 can also be configured to: receive a video playback operation triggered by the video playback area of the visual interactive interface; play back multiple annotated videos based on the video playback operation; and, in response to receiving an annotation information modification operation sent through the annotation area of the visual interactive interface, perform a modification operation on the annotation information that failed the verification based on the annotation information modification operation to obtain the annotation information of the video segment corresponding to the annotation information that failed the verification, wherein the modification operation includes modifying at least one of the first frame information, the second frame information, or the attribute information.
[0042] Exemplary methods
[0043] Figure 2 is a flowchart illustrating a video annotation method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to the electronic device or video annotation apparatus of Figure 1, and as shown in Figure 2, includes the following steps:
[0044] Step 201: During the synchronous playback of videos of the same scene captured from multiple different perspectives, in response to receiving a labeling instruction triggered by the target timeline corresponding to the target element in multiple videos through the visual interactive interface, based on the first frame information and the second frame information corresponding to the target timeline, the video segment corresponding to the first frame information and the second frame information is determined in multiple videos, and the labeling instruction includes the attribute information of the target element.
[0045] The viewpoint is used to indicate the direction from which video or images are captured from the scene. Different viewpoints in the embodiments of this disclosure may include a front viewpoint, a left front viewpoint, a right front viewpoint, a rear viewpoint, a left rear viewpoint, and a right rear viewpoint.
[0046] In this embodiment, multiple image acquisition devices installed on the vehicle can simultaneously acquire video from different perspectives of the same scene. For example, a front-view camera, a left front-view camera, a right front-view camera, a rear-view camera, a left rear-view camera, and a right rear-view camera can simultaneously acquire video data of the vehicle's surrounding environment. This disclosure does not limit the number, installation location, or viewing angle range of the image acquisition devices. This disclosure also does not limit the type of image acquisition device; it can be a monocular camera, a binocular camera, a depth camera, etc.
[0047] Synchronous acquisition refers to the complete synchronization of the images captured by multiple image acquisition devices. This time synchronization can be achieved through network-connected devices. For example, multiple image acquisition devices can exchange synchronization information with a master clock within a specific period. By utilizing the timestamp information in the sending and receiving times, multiple image acquisition devices can synchronize the video capture of the same scene.
[0048] In practice, when multiple videos captured from multiple perspectives are out of sync, interpolation or other methods can be used to align the times of each video.
[0049] Synchronous playback refers to a playback method in which multiple videos start playing simultaneously and can be paused or resumed at the same time through a toggle button or other interactive methods.
[0050] In this embodiment of the disclosure, the target element may include, but is not limited to, any one or more of the following: left-turn signal lights, right-turn signal lights, straight-ahead signal lights, traffic signs, lane lines, vehicles, pedestrians, and other static or dynamic elements that need to be labeled. The attribute information of the target element is used to indicate its shape, state, or direction of movement. Different types of target elements have different attribute information. For example, if the target element is a left-turn signal light, its attribute information may include the color of the left-turn signal light, such as red or green; if the target element is a vehicle, its attribute information may include the door open / closed status, the status of its lights, or its relative positional relationship with the vehicle.
[0051] The timeline is a timeline used to annotate target elements in a video according to video segmentation. The length of the timeline is the same as the duration of the multiple videos to be annotated. Each timeline is used to annotate one target element. The timeline can be adjusted by the user according to actual annotation needs and can be segmented based on changes in the target element's attributes. Different timelines are displayed sequentially in the annotation area of the visual interactive interface according to the configured order.
[0052] The annotation instruction is used to instruct the annotation of target elements in the video. The annotation instruction includes the attribute information of the target element, as well as the first frame information and the second frame information corresponding to the target timeline to be annotated.
[0053] In this implementation, since each timeline is a timeline corresponding to the video, the first frame information and the second frame information corresponding to the timeline can be used to represent the frame number of the start frame and the frame number of the end frame of the video segment, respectively.
[0054] In some optional implementations, the annotation instructions may further include: identification information of the target element, used to uniquely identify the target element. For example, the identification information can be the name of the target element. The type of the target element is already configured in the timeline configuration information when setting the timeline; during annotation, it is only necessary to annotate the target element's shape, state, or direction of motion in different image frames.
[0055] In an optional example, step 201 can be executed by the processor calling the corresponding instructions stored in the memory. The operation of synchronously playing videos of the same scene acquired from multiple different perspectives can also be executed by the processor and displayed through a visual interactive interface. The operation of determining the video segment corresponding to the first frame information and the second frame information corresponding to the target timeline in multiple videos, and the operation of labeling instructions including the attribute information of the target element, can be implemented by the video segment determination module executed by the processor.
[0056] Step 202: Label the target elements of each frame in the video clip with attribute information to obtain the labeled video clip.
[0057] Referring to Figure 3, which illustrates the process of labeling the straight lights in each video segment using the timeline corresponding to the target element "straight lights," the labeling information setting window in the visual interactive interface (the window indicated by number 31 in Figure 3) allows for labeling the straight lights in the video segments indicated by the starting frame 377 and the ending frame 566. For example, by setting color and time information, attribute values can be labeled for the straight lights in each frame between the starting frame 377 and the ending frame 566, such as labeling them as "red," thus completing the labeling of the straight lights in the video segment (frames 377-566).
[0058] In this implementation, video segments are segmented from multiple videos by using the first and second frame information corresponding to the timeline. The video segment is a video segment with the image frame identified by the first frame information as the start frame and the image frame identified by the second frame information as the end frame. Thus, by annotating the video segment, the effect of annotating each frame of the video segment can be achieved.
[0059] In an optional example, step 202 can be executed by the processor calling the corresponding instructions stored in memory, or it can be implemented by the annotation module executed by the processor.
[0060] The video annotation method provided in this embodiment, during the synchronous playback of videos of the same scene captured simultaneously from multiple different perspectives, if an annotation instruction triggered by a target timeline corresponding to a target element contained in multiple videos is received through a visual interactive interface, the video segment corresponding to the first and second frame information of the target timeline can be determined in multiple videos. Then, the target element attribute information of each frame in the video segment is annotated, resulting in an annotated video segment. Therefore, the technical solution provided in this disclosure utilizes a timeline to simultaneously annotate videos captured synchronously from multiple different perspectives, improving the annotation efficiency, accuracy, and consistency of video annotation across multiple perspectives. Furthermore, based on the first and second frame information corresponding to the target timeline, batch annotation of each frame in the video segment is achieved, further improving the video annotation efficiency.
[0061] In some optional implementations, in order to play videos of the same scene captured from multiple different perspectives simultaneously, video decoding can be performed on any video to be labeled to obtain the image sequence corresponding to any video to be labeled; the image sequence corresponding to any video to be labeled is rendered on the interface canvas of the corresponding visualization interface according to the original frame rate of the video; the rendering progress of the image sequence in different interface canvases is controlled by a playback component, thus realizing the synchronous playback of each video on the visualization interface.
[0062] When playing videos of the same scene simultaneously captured from multiple different perspectives through a visual interactive interface, multiple videos of the same scene captured from multiple different perspectives can be played simultaneously in multiple video playback areas of the visual interactive interface in a grid pattern, as shown in Figure 4. Playing videos from various perspectives in a grid pattern helps improve the viewing experience of users watching multiple videos at the same time. Furthermore, by comparing the synchronous playback of videos from various perspectives, the video with the better display effect on the target element can be determined, which helps improve the accuracy of video annotation.
[0063] As shown in Figure 5, multiple videos captured from different perspectives can be played simultaneously in multiple video playback areas of the visual interactive interface in window mode. These multiple video playback areas include a main playback area (indicated by label 51) and a secondary playback area (indicated by label 52). The main playback area allows switching between videos from different perspectives based on user-triggered video switching commands, while the secondary playback area plays other videos besides those played in the main playback area. Playing videos from various perspectives in window mode allows the main playback area to play the video captured from the best perspective, improving the user's viewing experience. Playing videos captured from other perspectives in the secondary playback area helps compensate for poor display of target elements in certain frames of the video played in the main playback area.
[0064] In this embodiment of the disclosure, grid mode and window mode can be switched based on a mode switching command triggered by the user in order to improve the accuracy of video annotation.
[0065] Figure 6 is a flowchart illustrating the configuration of the timeline in a video annotation method provided by an exemplary embodiment of this disclosure. This embodiment uses the configuration of the timeline as an example for illustrative explanation, as shown in Figure 6, including the following steps:
[0066] Step 601: Receive the timeline configuration command triggered through the visual interactive interface.
[0067] The timeline configuration command is used to instruct users to set the corresponding timeline for each target element that needs to be labeled. Before labeling a video, users can determine the target elements that may need to be labeled based on their labeling requirements. For example, in the field of autonomous driving, traffic lights, parked vehicles, and vehicles entering from ramps may need to be identified and labeled. Therefore, the target elements to be labeled can be determined to include traffic lights, parked vehicles, and ramps. Then, configuration commands can be triggered through the visual interactive interface to configure the timeline corresponding to traffic lights, parked vehicles, or ramps.
[0068] In some optional implementations, the video annotation tool (software) may have multiple preset timelines. Users can select the timeline that might be needed for the current annotation task, thus triggering a timeline configuration command. For example, the video annotation tool (software) may have pre-set timelines associated with elements such as pedestrians, vehicles, traffic lights (including straight-ahead lights), or ramps. When starting an annotation task, the required timeline can be selected from multiple preset timelines based on the task being annotated.
[0069] In some alternative implementations, users can create a corresponding timeline by triggering timeline configuration commands for the target element to be annotated through the timeline configuration menu or panel of the video annotation tool (software).
[0070] In practice, the elements that need to be labeled in the video can be determined according to the requirements of this video annotation. For example, if the current requirement is to annotate the driving scene of the vehicle, the elements to be labeled include the vehicle and lane lines, then the timeline information of the vehicle and lane lines needs to be configured.
[0071] In an optional example, step 601 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the first receiving module executed by the processor.
[0072] Step 602: Based on the timeline configuration instructions, determine the timeline to be displayed in the annotation area of the visual interactive interface. The timeline is used to implement the operation of annotating target elements in multiple videos.
[0073] The timeline displayed in the annotation area of the visual interactive interface can be one or more.
[0074] In this embodiment, the timeline displayed in the annotation area of the visual interactive interface is the timeline configured by the timeline configuration command. For example, for traffic lights, if six timelines are configured, namely "straight ahead light", "inferred straight ahead state", "left turn light", "inferred left turn state", "right turn light", and "inferred right turn state", then these six configured timelines can be displayed in the annotation area accordingly.
[0075] If only the three target elements—"straight ahead light," "left turn light," and "right turn light"—are labeled, the attribute information of these three target elements may not be directly determined from the video, requiring the labelers to infer it from certain elements in the video. Therefore, this disclosure additionally sets up timelines corresponding to the inference states for the three target elements, such as timelines for "inferring left turn light state," "inferring straight ahead state," and "right turn light," which helps to label the target elements better and more comprehensively.
[0076] As can be seen from the above description, in the technical solution disclosed herein, the configuration of the timeline is for better annotation of the target element. A target element can correspond to one timeline or two timelines. If it corresponds to two timelines, the basis for setting the attribute information of the target element for the two timelines can be different.
[0077] For example, annotations triggered by the "straight-ahead light" timeline generate actual straight-ahead light annotation information obtained from observing the target element "straight-ahead light," while annotations triggered by the "inferring straight-ahead state" timeline generate inferred straight-ahead light annotation information obtained from inferring the target element "straight-ahead light." Compared to the former, the annotation information triggered by the "straight-ahead light" timeline is more accurate. However, the annotations triggered by the "inferring straight-ahead state" timeline can assist in determining the straight-ahead light state when it is not possible to observe it accurately, which helps to improve the comprehensiveness and robustness of the target element annotations.
[0078] In an optional example, step 602 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the timeline determination module executed by the processor.
[0079] Step 603: Display the timeline in the annotation area of the visual interactive interface.
[0080] In an optional example, step 603 can be executed by the processor calling the corresponding instruction stored in memory, or it can be executed by the processor and displayed through the first display module.
[0081] The timeline configuration method provided in this embodiment, by configuring corresponding timelines for each target element to be annotated in multiple videos based on video annotation requirements, facilitates the annotation of target elements in videos through video segmentation based on timelines, thereby improving annotation efficiency. Furthermore, flexibly setting the corresponding timelines according to annotation requirements helps to more accurately control the content display in the annotation area of the visual interactive interface, satisfying the annotation needs of the videos while avoiding the display of unnecessary content.
[0082] Figure 7 is a flowchart illustrating a video annotation method provided in another exemplary embodiment of this disclosure; as shown in Figure 7, this embodiment includes the following steps:
[0083] Step 701: In response to receiving a annotation operation triggered by the target timeline corresponding to the target element, display the annotation information setting window in the annotation area of the visualization interface.
[0084] In this embodiment, the annotation operation is a user-triggered operation triggered by the timeline to segment and annotate the video. Specifically, the user can trigger the annotation operation by clicking on the timeline or by using the annotation button on the visual interactive interface.
[0085] The annotation information setting window is the window indicated by label 31 in Figure 3. This window provides users with attribute information for setting target elements and the first and second frame information of video clips.
[0086] In an optional example, step 701 can be executed by the processor calling the corresponding instruction stored in memory, or it can be executed by the processor and displayed through the second display module.
[0087] Step 702: Receive attribute information input by the user through the annotation information setting window.
[0088] For example, referring to Figure 3, the annotation information setting window in the visual interactive interface (the window indicated by label 31 in Figure 3) can be used to annotate the straight lights in the video segment indicated by the starting frame 377 (as the first frame information) and the ending frame 566 (as the second frame information). For example, by setting color information and time information, attribute values can be annotated for the straight lights in each frame between the starting frame 377 and the ending frame 566, such as annotating them as "red", thereby completing the annotation of the straight lights in the video segment (frames 377-566).
[0089] In some implementations, the corresponding "time" information can be automatically filled into the annotation information setting window based on the video playback progress when the user triggers the annotation operation. Specifically, a video segment can be determined based on the current playback frame of multiple images to be annotated when the annotation operation is received and the historical playback frames of multiple images to be annotated when the annotation operation was previously triggered. The previous adjacent frame of the current playback frame is the end frame of the video segment, and the historical playback frame is the start frame of the video segment.
[0090] In some implementations, the time information in the attribute information can be automatically segmented based on the attribute change state of the target element. For example, when the straight light changes from green to yellow, the time axis can be segmented at the point when it turns yellow. This allows for video segmentation of the frame image in the video where the green light turns yellow. The frame image where the green light turns yellow belongs to the next video segment, while the frame image before the frame image where the green light turns yellow belongs to the previous video segment.
[0091] In an optional example, step 702 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the second receiving module executed by the processor.
[0092] Step 703: In response to receiving the confirmation instruction that the attribute information setting is complete, confirm that the annotation instruction has been received.
[0093] In some implementations, after inputting attribute information through the annotation information setting window, a confirmation instruction can be obtained by triggering a button in the visual interactive interface. Therefore, based on the confirmation instruction, it can be determined that an annotation instruction has been received to annotate the target elements in the video segment (such as the video segment corresponding to frames 377-566 above).
[0094] In an optional example, step 703 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by a confirmation module executed by the processor.
[0095] Step 704: Based on the first frame information and the second frame information corresponding to the target timeline, determine the video segments corresponding to the first frame information and the second frame information in multiple videos, and the annotation instructions include the attribute information of the target elements.
[0096] In an optional example, step 704 can be executed by the processor calling the corresponding instructions stored in memory, or it can be implemented by the video segment determination module executed by the processor.
[0097] Step 705: Label the target elements of each frame in the video clip with attribute information to obtain the labeled video clip.
[0098] In an optional example, step 705 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the annotation module executed by the processor.
[0099] In some implementations, the implementation of steps 704-705 can be found in the description of steps 201-202 in the embodiment shown in Figure 2, and will not be described in detail here.
[0100] The video annotation method provided in this embodiment displays an annotation information setting window in the annotation area of the visual interactive interface by triggering annotation operations on the target timeline corresponding to the target element. In response to receiving attribute information input by the user through the annotation information setting window and a confirmation command indicating completion of the attribute information setting, the annotation command is confirmed, thereby enabling simultaneous annotation of videos captured synchronously from multiple different perspectives. This embodiment can automatically generate the start frame (first frame information) and end frame (second frame information) of a video segment based on user-triggered annotation operations, avoiding the complexity of manually inputting the start and end frames of the video segment and reducing the probability of errors.
[0101] After annotating target elements in multiple videos, the annotation information corresponding to the target elements in multiple videos can be verified. Figure 8 is a flowchart illustrating the verification of video annotation information provided in an exemplary embodiment of this disclosure. As shown in Figure 8, it includes the following steps:
[0102] Step 801: In response to receiving a confirmation instruction through the visual interactive interface that the target elements of multiple videos have been labeled, the labeling information corresponding to the target elements in the multiple videos is verified.
[0103] When annotating videos, multiple target elements are typically annotated. After annotating each target element, the electronic device receives a confirmation command triggered by the user. Upon receiving this confirmation, the electronic device simultaneously receives annotation information from the video annotation tool (software). This annotation information includes the annotation details for each target element across multiple viewpoints. For example, if the video annotation operation annotates straight-ahead lights, left-turn lights, and right-turn lights across multiple viewpoints, the annotation information would include the attribute information of these lights in each frame of the video from those viewpoints. Since target element annotation is implemented through video segmentation, the attribute information of target elements may be identical across multiple consecutive frames. Therefore, the annotation information of target elements can also be stored and recorded as video segments.
[0104] In specific verification, the completeness of the annotation information for each target element can be checked separately. For example, it can be checked whether the target element "straight ahead light" has attribute information annotated in every frame of the video from multiple perspectives. If the video segment corresponding to frames 566-788 of the video does not have attribute information for the target element "straight ahead light", then the annotation information of this target element has a problem of missing annotation, indicating that the verification fails. In some other implementations, the annotation information of any two or more target elements can also be cross-checked to check whether there is a conflict between the annotation information of different target elements. For example, for the same video segment, the attribute value of the target element "left turn light" is red, while the attribute value of the target element "inferred left turn light status" is green. Since the "inferred left turn light status" is the left turn light status inferred by the user based on certain elements in the video, when the attribute values of the two are different in the same video segment, the annotation information of the two conflict.
[0105] For example, based on the video from the left front view, it is determined that the straight-ahead light is red in the video segment from frame 377 to 566. However, in the video from the front view, a bus completely blocks the left-turn light. But based on the bus turning left in the video, the color information in the attribute information of "inferred left-turn light status" can be determined to be green. However, within the same time period (the video segment from frame 377 to 566), the attribute value of "left-turn light" and the attribute value of "inferred left-turn light status" should be the same. If they are different, it can be inferred that there is a conflict.
[0106] In some optional implementations, the annotation information corresponding to each annotated video segment can be traversed, and it can be determined whether there are any video segments without annotation information for any target element. For example, for "straight light", the attribute information of "straight light" is not set in video segment 3 (frames 233-376), which indicates that there is a problem of missing annotation information for "straight light". By traversing the annotation information of relevant elements, it can be determined whether there is any omission of annotation information.
[0107] Furthermore, if the verification result indicates that the verification is successful, the video annotation information can be saved as a standard format file, such as JSON or XML; if the verification result indicates that the verification fails, a prompt message can be generated through step 802.
[0108] In an optional example, step 801 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the verification module executed by the processor.
[0109] Step 802: In response to the verification result indicating verification failure, a prompt message is generated, which carries the annotation information of verification failure.
[0110] In this embodiment of the disclosure, if there are omissions or conflicts in the annotation information, a prompt message can be generated.
[0111] In some embodiments, the prompt message can be a text-based prompt, such as popping up a window and displaying the text message "Inference Straight-Ahead State [405-566] Frame Missing Marker"; in other embodiments, the prompt message can be an audio-based prompt, such as playing the audio message "Inference Straight-Ahead State [405-566] Frame Missing Marker"; in still other embodiments, both text-based and audio-based prompt messages can be output simultaneously. In this embodiment of the present disclosure, the prompt message can be any other information that can indicate the failure of the annotation information verification, and the form of the prompt message is not specifically limited.
[0112] Among them, the annotation information of failed verification is used to indicate missing or conflicting annotation information.
[0113] In an optional example, step 802 can be executed by the processor calling the corresponding instructions stored in memory, or it can be implemented by the generation module executed by the processor.
[0114] Step 803: Output a prompt message so that the user can modify the annotation information that failed the verification based on the prompt message.
[0115] In an optional example, step 803 can be executed by the processor calling the corresponding instruction stored in memory, or it can be displayed by the output module executed by the processor.
[0116] Step 804: Obtain the video segment corresponding to the verification failure annotation information carried in the prompt message.
[0117] In some implementations, the verification failure annotation in the prompt message carries the first frame information and the second frame information of the video segment. By analyzing the prompt message, the corresponding video segment can be obtained.
[0118] In an optional example, step 804 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the acquisition module executed by the processor.
[0119] Step 805: Play back the video segments corresponding to the annotation information that failed the verification.
[0120] In some optional implementations, a hyperlink to the video segment corresponding to the failed verification information can be embedded in the prompt message, so that users can click on the prompt message to replay the video segment corresponding to the failed verification information.
[0121] In some implementations, when replaying the video segments corresponding to the annotation information that failed verification, video segments from multiple video perspectives can be played back simultaneously.
[0122] In an optional example, step 805 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the first playback module executed by the processor.
[0123] Step 806: In response to receiving a labeling information modification operation sent through the labeling area of the visual interactive interface, based on the labeling information modification operation, perform a modification operation on the labeling information that failed the verification, and obtain the labeling information of the video segment corresponding to the labeling information that failed the verification. The modification operation includes modifying at least one of the first frame information, the second frame information, and the attribute information.
[0124] The annotation information modification operation is an action triggered by the user through the timeline to modify the annotation information. Specifically, users can trigger the annotation information modification operation by clicking on the timeline or by using the modification button on the visual interactive interface.
[0125] In this embodiment, when making modifications, the first frame information and the second frame information of the video segment corresponding to the annotation information can be modified. For example, if the video segment "Inference Straight-Ahead State [405-566] Frame" is missing an annotation, and the attribute value of "Inference Straight-Ahead State" of video segment [405-521] Frame is determined to be green based on the playback video, and the attribute value of "Inference Straight-Ahead State" of video segment [522-566] Frame is a flashing green light, then the video segment of [405-566] Frame can be divided into two video segments (the video segment of [405-521] Frame and the video segment of [522-566] Frame), and the corresponding attribute information can be set for the two video segments respectively.
[0126] In an optional example, step 806 can be executed by the processor calling the corresponding instruction stored in memory, or it can be implemented by the first modification module executed by the processor.
[0127] The video annotation information verification and modification method provided in this embodiment performs comprehensive verification when all target elements in multiple videos have been annotated. It can detect and prompt users about missing, incorrectly annotated, or conflicting data. It can also replay video segments that fail verification to assist users in making accurate modifications. This helps ensure that erroneous annotation information does not circulate, significantly improving the quality and reliability of annotation information.
[0128] In another optional implementation, after outputting the prompt information in step 803, the system can also receive a video playback operation triggered by the user through the video playback area of the visual interactive interface; and play back multiple annotated videos based on the video playback operation; and in response to receiving an annotation information modification operation sent through the annotation area of the visual interactive interface, modify the annotation information that failed the verification based on the annotation information modification operation, such as modifying at least one of the first frame information, the second frame information, and the attribute information.
[0129] In an optional example, this implementation can be executed by the processor calling the corresponding instructions stored in memory, or by the third receiving module, the second playback module, or the second modification module executed by the processor.
[0130] In this implementation, if the error message indicates a large number of failed annotations (e.g., more than 10), multiple annotated videos can be replayed directly. During playback, if an annotation modification request is received, the failed annotations are modified accordingly. This approach helps to uniformly modify the annotations of annotated videos when there are many failed annotations, improving the reliability of annotation modification.
[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.
[0132] Exemplary device
[0133] Figure 9 is a schematic diagram of the structure of a video annotation apparatus provided in an exemplary embodiment of this disclosure; as shown in Figure 9, the apparatus includes:
[0134] The visual interactive interface module 91 is used to synchronously play videos of the same scene captured from multiple different perspectives, and to interact with the user.
[0135] The video segment determination module 92 is used to determine the video segment corresponding to the first frame information and the second frame information in multiple videos of the same scene that are simultaneously captured from multiple different perspectives during the synchronous playback process. The annotation instruction includes the attribute information of the target element.
[0136] Annotation module 93 is used to annotate the attribute information of the target elements in each frame of the video clip, thereby obtaining the annotation information of the target elements in the video clip.
[0137] Figure 10 is a schematic diagram of the structure of a video annotation apparatus provided in another exemplary embodiment of this disclosure. As shown in Figure 10, based on the embodiment shown in Figure 9, in some embodiments, the visual interactive interface module 91 may include:
[0138] The first playback submodule 911 is used to simultaneously play videos of the same scene captured from multiple different perspectives in multiple video playback areas of the visual interactive interface according to a grid pattern; or,
[0139] The second playback submodule 912 is used to simultaneously play multiple videos captured from different perspectives in multiple video playback areas of the visual interactive interface in window mode. The multiple video playback areas include a main playback area and a secondary playback area. The main playback area can switch between videos from different perspectives based on the video switching command triggered by the user. The secondary playback area plays other videos besides the videos played in the main playback area.
[0140] In some embodiments, the video annotation device may further include:
[0141] The first receiving module 94 is used to receive timeline configuration instructions triggered through a visual interactive interface;
[0142] The timeline determination module 95 is used to determine at least one timeline displayed in the annotation area of the visual interactive interface based on the timeline configuration instructions. The at least one timeline is used to implement the operation of annotating target elements in multiple videos.
[0143] The first display module 96 is used to display at least one timeline in the annotation area of the visual interactive interface.
[0144] In some embodiments, the video annotation device may further include:
[0145] The second display module 97 is used to display the annotation information setting window in the annotation area of the visual interactive interface in response to receiving the annotation operation triggered for the target time axis corresponding to the target element;
[0146] The second receiving module 98 is used to receive attribute information input by the user through the annotation information setting window;
[0147] The confirmation module 99 is used to respond to the confirmation command received when the attribute information setting is completed, and to confirm that the annotation command has been received.
[0148] In some embodiments, the video annotation device may further include:
[0149] Verification module 11 is used to verify the annotation information corresponding to the target elements of multiple videos in response to a confirmation instruction received through a visual interactive interface indicating that the target elements of multiple videos have been annotated.
[0150] The generation module 12 is used to generate a prompt message in response to the verification result indicating that the verification failed. The prompt message carries the annotation information of the verification failure.
[0151] Output module 13 is used to output prompt information so that users can modify the annotation information of the verification failure based on the prompt information.
[0152] In some embodiments, the video annotation device may further include:
[0153] The acquisition module 14 is used to acquire the video segment corresponding to the verification failure annotation information carried in the prompt message;
[0154] The first playback module 15 is used to play back the video segments corresponding to the annotation information that failed the verification.
[0155] The first modification module 16 is used to respond to receiving a modification operation of the annotation information sent through the annotation area of the visual interactive interface, and to perform a modification operation on the annotation information that failed the verification based on the modification operation, so as to obtain the annotation information of the video segment corresponding to the annotation information that failed the verification. The modification operation includes modifying at least one of the first frame information, the second frame information and the attribute information.
[0156] In some embodiments, the apparatus includes:
[0157] The third receiving module 17 is used to receive video playback operations triggered through the video playback area of the visual interactive interface;
[0158] The second playback module 18 is used to play back multiple annotated videos based on video playback operations;
[0159] The second modification module 19 is used to respond to receiving a modification operation of the annotation information sent through the annotation area of the visual interactive interface, and to perform a modification operation on the annotation information that failed the verification based on the modification operation, so as to obtain the annotation information of the video segment corresponding to the annotation information that failed the verification. The modification operation includes modifying at least one of the first frame information, the second frame information and the attribute information.
[0160] The exemplary embodiments of this device correspond to the exemplary method section described above, and the relevant content can be referenced and cited interchangeably. The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section described above, and will not be repeated here.
[0161] Exemplary computer program products and computer-readable storage media
[0162] In addition to the methods and devices described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the video annotations of the various embodiments of this disclosure described in the "Exemplary Methods" section above.
[0163] Computer program products can be written in any combination of one or more programming languages to perform the operations of embodiments of this disclosure. The programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0164] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the video annotations of the various embodiments of this disclosure described in the "Exemplary Methods" section above.
[0165] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0166] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.
[0167] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.
Claims
1. An electronic device comprising a memory for storing a computer program product and a processor for executing the computer program product stored in the memory, the processor being configured to: During the synchronous playback of videos of the same scene captured from multiple different perspectives, in response to receiving a labeling instruction triggered by a target timeline corresponding to a target element in multiple videos through a visual interactive interface, based on the first frame information and the second frame information corresponding to the target timeline, the video segment corresponding to the first frame information and the second frame information is determined in the multiple videos, and the labeling instruction includes the attribute information of the target element. The attribute information is labeled for the target element in each frame of the video segment to obtain the labeled video segment.
2. The electronic device according to claim 1, wherein the processor is configured to synchronously play videos of the same scene simultaneously captured from multiple different perspectives, wherein: According to the grid pattern, videos of the same scene, captured simultaneously from multiple different perspectives, are played synchronously in multiple video playback areas of the visual interactive interface; or... synchronously playing, in a window mode, videos of a same scene synchronously captured from multiple different perspectives in multiple video playing areas of the visual interactive interface, wherein, The multiple video playback areas include a main playback area and a secondary playback area. The main playback area can switch between videos from different perspectives based on user-triggered video switching commands. The secondary playback area plays other videos besides those played in the main playback area. 3.The electronic device of claim 1, wherein, Before synchronously playing videos of the same scene captured from multiple different perspectives, the processor is further configured to: Receive timeline configuration instructions triggered through the visual interactive interface; Based on the timeline configuration instructions, the timeline displayed in the annotation area of the visual interactive interface is determined, and the at least one timeline is used to implement the operation of annotating target elements in the plurality of videos; The timeline is displayed in the labeled area of the visual interactive interface.
4. The electronic device of claim 1, wherein, The processor receives annotation instructions triggered by the target timeline corresponding to the target element in multiple videos through a visual interactive interface, and is configured as follows: In response to receiving a annotation operation triggered by the target timeline corresponding to the target element, an annotation information setting window is displayed in the annotation area of the visual interactive interface; Receive attribute information input by the user through the annotation information setting window; In response to receiving a confirmation instruction that the attribute information setting is complete, the annotation instruction is confirmed to have been received.
5. The electronic device according to any one of claims 1-4, wherein after obtaining the labeled video segment, the processor is further configured to: In response to receiving a confirmation command through a visual interactive interface that the target elements of the multiple videos have been labeled, the labeling information corresponding to the target elements of the multiple videos is verified. In response to the verification result indicating verification failure, a prompt message is generated, which carries the annotation information of verification failure; The prompt message is output so that the user can modify the annotation information of the verification failure based on the prompt message.
6. The electronic device according to claim 5, wherein after outputting the prompt information, the processor is further configured to: Obtain the video segment corresponding to the verification failure annotation information carried in the prompt message; Play back the video segments corresponding to the annotation information that failed the verification; In response to receiving a labeling information modification operation sent through the labeling area of the visual interactive interface, based on the labeling information modification operation, a modification operation is performed on the labeling information that failed the verification, to obtain the labeling information of the video segment corresponding to the labeling information that failed the verification. The modification operation includes modifying at least one of the first frame information, the second frame information, or the attribute information.
7. The electronic device according to claim 5, wherein after outputting the prompt information, the processor is further configured to: Receive video playback operations triggered through the video playback area of the visual interactive interface; Based on the aforementioned video playback operation, multiple annotated videos are played back; In response to receiving a labeling information modification operation sent through the labeling area of the visual interactive interface, based on the labeling information modification operation, a modification operation is performed on the labeling information that failed the verification, to obtain the labeling information of the video segment corresponding to the labeling information that failed the verification. The modification operation includes modifying at least one of the first frame information, the second frame information, or the attribute information.
8. A video annotation method, comprising: During the synchronous playback of videos of the same scene captured from multiple different perspectives, in response to receiving a labeling instruction triggered by a target timeline corresponding to a target element in multiple videos through a visual interactive interface, based on the first frame information and the second frame information corresponding to the target timeline, the video segment corresponding to the first frame information and the second frame information is determined in the multiple videos, and the labeling instruction includes the attribute information of the target element. The attribute information is labeled for the target element in each frame of the video segment to obtain the labeled video segment.
9. The method according to claim 8, wherein the synchronous playback of videos of the same scene simultaneously acquired from multiple different perspectives includes: According to the grid pattern, the videos of the same scene, which were simultaneously captured from multiple different perspectives, are played synchronously in multiple video playback areas of the visual interactive interface. or, According to the window mode, the videos of the same scene captured from multiple different perspectives are played synchronously in multiple video playback areas of the visual interactive interface. The multiple video playback areas include a main playback area and a secondary playback area. The main playback area can switch between videos from different perspectives based on the video switching command triggered by the user. The secondary playback area plays other videos besides the videos played in the main playback area.
10. The method of claim 8, wherein, Before synchronously playing videos of the same scene captured from multiple different perspectives, the process also includes: Receive timeline configuration instructions triggered through the visual interactive interface; Based on the timeline configuration instructions, the timeline displayed in the annotation area of the visual interactive interface is determined, and the at least one timeline is used to implement the operation of annotating target elements in the plurality of videos; The timeline is displayed in the labeled area of the visual interactive interface.
11. The method of claim 8, wherein, The step of receiving annotation instructions triggered by the target timeline corresponding to the target element contained in multiple videos through a visual interactive interface includes: In response to receiving a annotation operation triggered for the target timeline corresponding to the target element, an annotation information setting window is displayed in the annotation area of the visual interactive interface; Receive attribute information input by the user through the annotation information setting window; In response to receiving a confirmation instruction that the attribute information setting is complete, the annotation instruction is confirmed to have been received.
12. The method of any one of claims 8-11, wherein, After obtaining the labeled video clips, the process also includes: In response to receiving a confirmation command through a visual interactive interface that the target elements of the multiple videos have been labeled, the labeling information corresponding to the target elements of the multiple videos is verified. In response to the verification result indicating verification failure, a prompt message is generated, which carries the annotation information of verification failure; The prompt message is output so that the user can modify the annotation information of the verification failure based on the prompt message.
13. The method of claim 12, wherein, After outputting the prompt message, the method further includes: Obtain the video segment corresponding to the verification failure annotation information carried in the prompt message; Play back the video segments corresponding to the annotation information that failed the verification; In response to receiving a labeling information modification operation sent through the labeling area of the visual interactive interface, based on the labeling information modification operation, a modification operation is performed on the labeling information that failed the verification, to obtain the labeling information of the video segment corresponding to the labeling information that failed the verification. The modification operation includes modifying at least one of the first frame information, the second frame information, or the attribute information.
14. The method of claim 13, wherein, After outputting the prompt message, the method further includes: Receive video playback operations triggered through the video playback area of the visual interactive interface; Based on the aforementioned video playback operation, multiple annotated videos are played back; In response to receiving a labeling information modification operation sent through the labeling area of the visual interactive interface, based on the labeling information modification operation, a modification operation is performed on the labeling information that failed the verification, to obtain the labeling information of the video segment corresponding to the labeling information that failed the verification. The modification operation includes modifying at least one of the first frame information, the second frame information, or the attribute information.
15. A video annotation device, comprising: A visual interactive interface is used to synchronously play videos of the same scene captured from multiple different perspectives, and to interact with the user. The video segment determination module is used to determine the video segment corresponding to the first frame information and the second frame information in the multiple videos during the synchronous playback of videos of the same scene captured simultaneously from multiple different perspectives. The annotation instruction includes the attribute information of the target element. The annotation module is used to annotate the attribute information of the target element in each frame of the video segment, thereby obtaining the annotation information of the target element in the video segment.
16. A computer-readable storage medium storing computer program instructions that, when executed, implement the method described in any one of claims 8-14.
17. A computer program product comprising computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 8-14.
