Multi-view video interaction method and apparatus, file generation method and apparatus

By establishing a three-dimensional index table and an independent perspective storage and decoding mechanism, the latency and stuttering issues of multi-view video files during perspective switching are resolved, enabling efficient multi-view video interaction, which is suitable for scenarios such as multi-camera playback, VR panorama, and multi-channel security.

CN122679306BActive Publication Date: 2026-11-17SI CHUAN ZHONG SHENG MATRIX TECH DEV CO LTD +2
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202611135001.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-07-29
Publication Date
2026-11-17
Estimated Expiration
2046-07-29

AI Technical Summary

Technical Problem

Traditional multi-view temporal video files suffer from problems such as repeated addressing, sequential decoding, and redundant data loading when switching viewpoints, resulting in high latency and video stuttering, and they cannot achieve cross-view temporal synchronization.

Method used

By establishing a mapping relationship between absolute timestamps, viewpoint identifiers, and video frame data byte offsets, a three-dimensional index table is used for direct lookup and byte-level jumps. Combined with independent viewpoint storage and decoding mechanisms, a globally unified absolute time base and independent viewpoint clock control are ensured.

Benefits of technology

It effectively reduces the delay of perspective switching, improves the smoothness of multi-view video interaction, avoids screen tearing and timing misalignment, and is suitable for multi-camera playback, VR panorama and multi-channel security scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122679306B_ABST
    Figure CN122679306B_ABST
Patent Text Reader

Abstract

This application discloses a multi-view video interaction method and apparatus, and a file generation method and apparatus, relating to the field of data interaction technology. This application acquires a temporal dynamic file and parses a three-dimensional index table in the file header, establishing a mapping relationship between absolute timestamps, viewpoint identifiers, and frame byte offsets. Then, based on the first viewpoint identifier, it plays a video stream and listens for viewpoint switching commands. In response to the viewpoint switching command, it locks the target absolute timestamp corresponding to a globally unified absolute time base and looks up the target byte offset for the second viewpoint in the table. Then, it performs a byte-level jump to read the encoded data, sends it to a dedicated second-viewpoint decoder for independent decoding, compares frame timestamps to select matching frames, and renders and outputs them, achieving seamless cross-viewpoint temporal alignment. This application effectively improves the high latency and stuttering issues caused by re-addressing and sequential decoding in traditional multi-view video interaction, improving the response speed of multi-view switching and the smoothness of screen interaction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data interaction technology, specifically to a multi-view video interaction method and apparatus, and a file generation method and apparatus. Background Technology

[0002] A time-domain file is a sequential data file that uses time as its core dimension and associates multiple perspectives and orientations. Referring to the specific content of CN114820575A, "Image Verification Method, Apparatus, Computer Equipment and Storage Medium," for example, the three-dimensional target object mentioned in paragraphs

[0043] -

[0044] of the specification, a time-domain file can refer to a three-dimensional target object file containing time-dimensional information. A time-domain file can record the dynamic characteristics of the target object at different points in time (or time periods), and also associate information from multiple perspectives and orientations. In the time dimension, a time-domain file contains continuous or discrete time nodes, which can reflect the dynamic changes of the target object over time. In the visual dimension, time-domain data at the same point in time can correspond to different observation perspectives, recording the time-domain performance of the target object under different observation angles. In the orientation dimension, time-domain data at the same perspective and the same point in time can be further associated with different orientations.

[0003] Currently, interactive processing solutions for multi-view temporal video files have inherent flaws. The core issue is that high latency and screen stuttering are easily caused by re-addressing and sequential decoding during interactive switching of multi-view videos. First, traditional temporal video files rely solely on frame sequence numbers and relative time for single-level indexing. When performing viewpoint switching operations, it is necessary to sequentially traverse all frame data from the beginning of the video file to retrieve the target frame. The massive number of irrelevant frames leads to a large amount of disk input / output (IO) or network data reading overhead, significantly increasing addressing time and resulting in significant switching latency. Second, in traditional file storage structures, video frames from different camera positions are interleaved and mixed. After switching to the target viewpoint, the entire video stream for that viewpoint must be decoded sequentially from the beginning. It is impossible to directly locate the byte position of a single frame for independent decoding, resulting in repeated decoding of redundant preceding frames, high computational consumption, and prolonged screen buffering and stuttering. Third, in streaming transmission scenarios, traditional multi-view streaming media does not support fixed-point range requests based on byte offsets. Switching views can only sequentially retrieve the entire segmented data. In weak network environments, redundant data transmission further amplifies stuttering and black screen phenomena. Fourth, all perspectives share the same playback clock and global decoder. Switching perspectives will interrupt the original timing and destroy the decoding context. Each switch requires rebuilding the decoding process, which further exacerbates the switching delay and screen tearing issues.

[0004] In summary, traditional multi-view temporal video files generally suffer from technical defects such as repeated addressing, sequential decoding, and redundant data loading during each viewpoint switch. These defects not only lead to response delays and screen stuttering during viewpoint switching, but also easily cause screen tearing due to timing misalignment, severely affecting the smoothness of multi-view video interaction. Summary of the Invention

[0005] The purpose of this application is to provide a multi-view video interaction method and apparatus, and a file generation method and apparatus, to solve the problem of high latency and screen stuttering when switching perspectives in traditional multi-view time-domain files.

[0006] To achieve the above objectives, the first aspect of this application provides a multi-view video interaction method, applied to a playback end, comprising: The temporal dynamic file is obtained, and the three-dimensional index table in the file header of the temporal dynamic file is parsed. The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers and byte offsets of video frame data in the file. Play the corresponding video stream based on the first perspective identifier and receive perspective switching instructions in real time. The perspective switching instructions are used to indicate switching to the second perspective identifier. In response to the viewpoint switching command, the target absolute timestamp of the currently playing video is obtained, and the three-dimensional index table is queried based on the target absolute timestamp and the second viewpoint identifier to determine the target byte offset of the target video frame corresponding to the second viewpoint identifier and the target absolute timestamp. Based on the target byte offset, a byte-level jump is performed from the temporal dynamic file to read and decode the target video frame, and the rendering output is based on the target absolute timestamp.

[0007] A second aspect of this application provides a method for generating multi-view interactive video files, applied at a capture end, including: A temporal dynamic file synthesized from multiple video frame sequences is obtained. The data area of ​​the temporal dynamic file adopts a storage structure that is isolated by viewpoint. All video frame data corresponding to the same viewpoint identifier are stored continuously in the data area, while video frame data with different viewpoint identifiers are stored in blocks in the data area. An index area is constructed in the header of the time-domain dynamic file to generate a three-dimensional index table; The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers, and the byte offsets and frame length information of the corresponding video frames in the data area. When a viewpoint switching instruction is received, the three-dimensional index table allows the playback terminal to directly query and generate the target byte offset based on the current target absolute timestamp and the second viewpoint identifier to be switched, so as to support the playback terminal to perform byte-level jumps from the time-domain dynamic file and independently decode the target video frames.

[0008] A third aspect of this application provides a multi-view video interaction device for use in a playback terminal, the multi-view video interaction device comprising: The first acquisition module is used to acquire the temporal dynamic file and parse the three-dimensional index table in the file header of the temporal dynamic file. The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers and byte offsets of video frame data in the file. The receiving module is used to play the corresponding video stream based on the first viewpoint identifier and to receive viewpoint switching instructions in real time. The viewpoint switching instructions are used to indicate switching to the second viewpoint identifier. The query module is used to respond to the viewpoint switching command, obtain the target absolute timestamp of the currently playing video, and query the three-dimensional index table based on the target absolute timestamp and the second viewpoint identifier to determine the target byte offset of the target video frame corresponding to the second viewpoint identifier and the target absolute timestamp. The jump module is used to perform byte-level jumps from the temporal dynamic file based on the target byte offset, so as to read and decode the target video frame and render it based on the target absolute timestamp.

[0009] A fourth aspect of this application provides a multi-view video interactive file generation device, applied at a capture end, the multi-view video interactive file generation device comprising: The second acquisition module is used to acquire a temporal dynamic file synthesized from multiple video frame sequences. The data area of ​​the temporal dynamic file adopts a storage structure that is isolated by viewpoint. All video frame data corresponding to the same viewpoint identifier are stored continuously in the data area, and video frame data with different viewpoint identifiers are stored in blocks in the data area. The generation module is used to construct an index area in the file header of the time-domain dynamic file to generate a three-dimensional index table; The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers, and the byte offsets and frame length information of the corresponding video frames in the data area. When a viewpoint switching instruction is received, the three-dimensional index table allows the playback terminal to directly query and generate the target byte offset based on the current target absolute timestamp and the second viewpoint identifier to be switched, so as to support the playback terminal to perform byte-level jumps from the time-domain dynamic file and independently decode the target video frames.

[0010] The beneficial effects of this application are: This application establishes a direct mapping relationship between absolute timestamps, viewpoint identifiers, and byte offsets by parsing a three-dimensional index table within the header of a time-domain dynamic file. When playing a first-view video and receiving a viewpoint switching command, the playback terminal can directly obtain the target byte offset corresponding to the target video frame by looking up the table based on the current target absolute timestamp and the second-view identifier to be switched.

[0011] Based on this target byte offset, the playback device can directly perform byte-level jumps on the temporal dynamic file to read the target video frame. This mechanism allows the playback device to avoid traversing all video frames for retrieval when switching perspectives, and also avoids sequentially decoding redundant preceding images, thereby effectively reducing the time spent in the addressing and decoding process and lowering the latency when switching perspectives.

[0012] Meanwhile, after acquiring the target video frame, this application uses the target's absolute timestamp as the basis for rendering output throughout the entire process. By maintaining a globally consistent absolute time base, the synchronous alignment of cross-view shots in terms of timing is guaranteed, thereby avoiding screen tearing or jumps caused by timing misalignment during viewpoint switching.

[0013] In summary, the technical solution of this application effectively improves the high latency and screen stuttering problems caused by re-addressing and sequential decoding in traditional multi-view video interaction, and improves the response speed of multi-view switching and the smoothness of screen interaction.

[0014] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a multi-view video interaction method provided in an embodiment of this application. Figure 2 This is a schematic diagram of a dual independent playback clock principle provided in the embodiments of this application; Figure 3 This is a schematic diagram illustrating a viewpoint-level streaming decoding principle provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the spatiotemporal anchoring principle of AI metadata provided in the embodiments of this application; Figure 5 This is a flowchart illustrating a method for generating multi-view interactive video files provided in an embodiment of this application. Figure 6 This is a schematic diagram illustrating the principle of a full-frame circular buffer provided in the embodiments of this application; Figure 7This is a schematic diagram illustrating a time code frame alignment principle provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a multi-view video interaction device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of a multi-view video interactive file generation device provided in the embodiments of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified. Details are set forth in the following description for illustrative purposes. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessarily obscuring the description of this application. Therefore, this application is not intended to be limited to the embodiments shown, but rather to be consistent with the broadest scope of the principles and features disclosed herein.

[0018] Multi-view temporal dynamic files are temporal files containing footage from multiple synchronized camera positions. The file uses a unified absolute timestamp to concatenate video frames from various perspectives, completely recording the dynamic images of a target from different shooting positions at the same moment. This supports interactive needs such as simultaneous playback from multiple cameras and rapid switching between different perspectives. In this embodiment, the interaction refers to the user initiating a perspective switching command during playback, and the terminal completing the entire screen switching response process, including frame addressing, reading, decoding, and synchronous rendering. The overall solution of this embodiment includes two main execution entities: a capture end responsible for encapsulating and generating the file, and a playback end responsible for loading the file and responding to the user's switching operation. These two entities work together to complete the entire process from organizing the original multi-camera recordings to low-latency multi-view screen interaction. The methods for each of the two entities are described below.

[0019] Figure 1This is a flowchart illustrating a multi-view video interaction method provided in an embodiment of this application. Figure 1 As shown, this multi-view video interaction method is applied to the playback end and can include steps 101-104, which will be described in detail below.

[0020] Step 101: Obtain the time-domain dynamic file and parse the three-dimensional index table in the header of the time-domain dynamic file.

[0021] The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers, and byte offsets of video frame data in the file.

[0022] The temporal dynamic file is an integrated encapsulated file that uses time as the core dimension and associates multiple viewpoints and directions of video sequences. It is generated by the acquisition end after aligning multiple video frames, storing them in blocks, and pre-building the index. The file header is independently divided into an index area, which is stored separately from the data area storing the video frames. The three-dimensional index table is a proprietary index structure of this application embodiment. Each index record is synchronously bound to three sets of mapping fields: absolute timestamp, viewpoint identifier, and video frame byte offset. The absolute timestamp uses the globally unified UTC standard time and is not affected by the relative time axis drift of a single camera. The byte offset refers to the byte distance between the starting position of the single frame encoded data in the entire temporal file data area and the file header, and can be directly used for file pointer directional jumps.

[0023] The playback device reads locally stored temporal dynamic files or retrieves file header data blocks via network streaming requests (without downloading the complete video data). It reads the binary data from the independent index area within the file header, decodes it line by line according to preset index parsing rules, and constructs a three-dimensional index hash mapping table that can be quickly retrieved in memory, thus completing the association and binding of time, viewpoint, and file position. By pre-loading lightweight index information, a global addressing mapping can be established without traversing massive amounts of video frames, avoiding the large amount of disk I / O operations caused by traditional players scanning frame by frame, and reducing hardware overhead in the addressing stage from the source.

[0024] Step 102: Play the corresponding video stream based on the first-viewpoint identifier and receive viewpoint switching instructions in real time.

[0025] The viewpoint identifier is a unique numerical / character code assigned to each camera by the acquisition end to distinguish video streams from different camera positions. The first-person viewpoint refers to the viewpoint of the currently playing, continuously rendered video stream. Upon startup, the playback end loads the video stream corresponding to the user-selected camera position by default; the unique code for this camera position is the first-person viewpoint identifier. During playback, the system continuously reads and renders the consecutive video frames corresponding to this viewpoint identifier, maintaining the playback clock and decoding context of this viewpoint throughout, serving as the basic playback stream identifier before switching. The second-person viewpoint identifier is the target camera position code carried in the viewpoint switching command issued by the user through an interactive device (Virtual Reality (VR) controller, mouse, touchscreen, etc.); it is the viewpoint identifier corresponding to the new video stream the user wishes to switch to.

[0026] The viewpoint switching command indicates a switch to a second viewpoint identifier. When the playback device detects the viewpoint switching command, it extracts the second viewpoint identifier carried within the command, retrieves the 3D index table using the current global target absolute timestamp, locates the video frame for that camera position at the same time, and completes the cross-viewpoint jump. The viewpoint switching command is generated in real-time by the user interface and interactive peripherals, carrying the target second viewpoint identifier field. Based on the user's initial selection of the first viewpoint identifier, the playback device retrieves the byte offset sequence corresponding to consecutive timestamps under that viewpoint in the 3D index table, sequentially reads the corresponding video frames, and continuously renders and outputs them. During playback, the interaction command queue is monitored in real-time. Once a switching command carrying a second viewpoint identifier is captured, the sequential reading logic is immediately paused, and the fixed-point addressing branch process is entered. This maintains smooth and normal playback from the first viewpoint while simultaneously monitoring interaction needs in real-time, enabling parallel processing of playback and interaction commands without blocking the basic video rendering process.

[0027] Step 103: In response to the viewpoint switching command, obtain the target absolute timestamp of the currently playing target, and query the three-dimensional index table based on the target absolute timestamp and the second viewpoint identifier to determine the target byte offset of the target video frame corresponding to the second viewpoint identifier and the target absolute timestamp.

[0028] The target absolute timestamp is the globally unified standard time of the current frame, and the same time base is used before and after the switch. The target byte offset is the starting storage byte position of the video frame corresponding to the same absolute timestamp in the time domain file under the second perspective. The global target absolute timestamp bound to the current rendered frame is extracted, and "target absolute timestamp + second perspective identifier" is used as the joint search key. Hash matching is performed in the in-memory three-dimensional index table to directly hit the corresponding index record and read the target byte offset stored in the record, without having to sequentially traverse all frame indices for that perspective. The dual-dimensional joint index is used to locate the target frame storage location with one click, abandoning the traditional single-time or single-track linear traversal search mode, which greatly reduces the addressing time and eliminates the root cause of search waiting and lag when switching perspectives.

[0029] Step 104: Based on the target byte offset, perform a byte-level jump from the temporal dynamic file to read and decode the target video frame, and render the output based on the target absolute timestamp.

[0030] Byte-level jump refers to the player adjusting the file read / write pointer to directly jump to the target byte offset position, reading only the encoded segment corresponding to a single frame, without reading all preceding video frames before the jump. The rendering benchmark is unified to the global target absolute timestamp, ensuring the timing alignment of the scenes before and after the switch. The player calls the low-level file addressing interface, directly jumping the file read / write pointer to the target byte offset, reading only the encoded data of the corresponding length of the current single frame, and assigning an independent decoder for the corresponding viewpoint to complete frame decoding. After decoding, the timeline of the decoded frame is aligned to the global target absolute timestamp and synchronously output to the rendering buffer for display. In this way, fixed-point byte reading can be achieved, eliminating the need to sequentially decode redundant preceding frames of the target viewpoint, significantly reducing disk I / O read / write operations and decoding computational power consumption. Relying on the globally unified absolute timestamp as the rendering benchmark completely eliminates the problems of cross-viewpoint scene timing tearing and jumps caused by multi-camera crystal oscillator drift and recording time differences.

[0031] The playback process in this embodiment relies on a pre-built 3D index and temporal block file structure at the acquisition end. It employs four interconnected technical methods—lightweight file header index preloading, dual-dimensional joint fast addressing, byte-level fixed-point jump reading, and global absolute timestamp synchronous rendering—to completely resolve the core technical defects of existing multi-view video interaction switching caused by full-scale traversal retrieval and sequential decoding of preceding frames, resulting in excessive disk I / O, high latency, screen stuttering, and timing misalignment. The addressing stage eliminates frame-by-frame scanning overhead, the reading stage loads only single-frame data without redundant decoding, and the rendering stage uses a unified global time base to ensure screen synchronization. This significantly shortens the perspective switching response time in both local playback and network streaming scenarios, reduces device computing power and disk I / O resource consumption, and greatly improves the smoothness of video interaction in scenarios such as multi-camera playback, VR panoramas, and multi-channel security.

[0032] Traditional technologies employ a single global clock or independent relative time bases for each viewpoint. When switching views, either the overall timing is reset, resulting in image gaps, or the time bases of the two video streams are inconsistent, causing a timing offset of tens to hundreds of milliseconds after switching. Furthermore, switching destroys the original clock context and repeatedly loads frame data, amplifying addressing and decoding latency. For example, in medical aesthetic comparison scenarios (before and after surgery), the operator needs to pause one viewpoint at a certain frame and independently fast-forward another viewpoint to the corresponding frame for comparison. In global clock mode, the timing of all views is synchronized; pausing or adjusting the speed of any viewpoint will simultaneously affect the entire image, completely failing to support the comparative analysis requirements of independently adjustable viewpoints.

[0033] Therefore, in step 102, a globally unified absolute time base can be maintained while playing the corresponding video stream based on the first viewpoint identifier. When receiving viewpoint switching in real time and switching to the second viewpoint identifier, a two-layer timing control architecture of independent clocks for each viewpoint and a globally unified absolute time base is introduced to keep the globally unified absolute time base unchanged. The globally unified absolute time base uses the UTC standard absolute timestamp bound after aligning multiple video frames at the acquisition end as the unique timing scale for the entire domain. It does not depend on the local playback clock of any single viewpoint. All video frames from all viewpoints are uniformly mapped to this timeline, and are not affected by single-channel video speed-up, pause, or drag operations. It is the underlying time base for seamless cross-viewpoint alignment.

[0034] Based on a globally unified absolute time base, the target absolute timestamp corresponding to the instant of switching is obtained, and the video stream corresponding to the second-view identifier is driven to start rendering and output from the video frame of the target absolute timestamp, so as to achieve seamless temporal alignment during viewpoint switching. Seamless temporal alignment means that the screen before and after the switch strictly correspond to the same global absolute timestamp, without screen blackouts, frame skips, time offsets, or temporal tears, and visually without any discontinuities or stutters.

[0035] Figure 2 This is a schematic diagram illustrating the principle of dual independent playback clocks provided in this application embodiment. Dual independent playback clocks refer to the playback end allocating independent timing-driven clocks to each viewpoint without interference. Each clock has its own dedicated buffer, timing context, and rendering scheduling thread. All independent clocks are anchored to the same global absolute time base, enabling free adjustment of timing within a viewpoint and time unification across viewpoints.

[0036] like Figure 2 As shown in the upper and lower sections, the upper section represents the traditional global clock architecture. All perspectives share a single global clock; any pause or speed adjustment operation performed by any perspective will be synchronously transmitted to all other perspectives, resulting in synchronized timing across all scenes, making independent control of each perspective impossible. The lower section represents the independent clock scheme of this application's embodiment. Each perspective is assigned its own independent clock, which supports independent playback speed, independent timing progress, and independent decoding threads. Calibration is achieved solely through unified absolute time logic, without any forced binding relationship, making it suitable for complex interactive scenarios such as independent comparison of scenes before and after medical aesthetic procedures.

[0037] For example, independent playback clocks are allocated to the first-person and second-person perspectives, respectively. The first playback clock is an independent timing clock bound to the first-person perspective identifier, responsible only for driving the frame rendering progress of the current playback perspective and storing the playback progress, speed, pause status, and other independent timing context of that perspective. The second playback clock is an independent timing clock generated by switching and bound to the second-person perspective identifier. It is isolated from the first playback clock in terms of hardware / thread resources. The progress and status of the two clocks do not interfere with each other, but they share the same globally unified absolute time base. Active perspectives are the perspectives that are currently being rendered on the screen or are being preloaded and waiting to be switched. The system concurrently controls a maximum of 3 active perspectives on the screen at the same time (the current main perspective + 1 preloaded perspective on each side), limiting memory usage and ensuring controllable memory usage. Initial calibration is performed when the time-domain dynamic file is opened. The initial timestamp of all perspectives is uniformly set to the global time origin built into the file header, completing the multi-perspective timing initialization alignment. For comparison calibration, in the scenario of comparing two images before and after medical aesthetics, the system automatically calculates the difference in the time origin of the two captured images, completing the alignment of the two sets of global absolute time bases for the videos, ensuring accurate timing matching across file images. The drag-independent rule ensures that when dragging the progress bar of a specific viewpoint, only the current timestamp within that viewpoint's independent clock is updated; the timing and playback status of all other views remain completely unaffected. The preloading mechanism detects the sliding direction in advance when the user sends a viewpoint switching command, pre-starts the independent decoding thread for the target viewpoint, and pre-reads and caches 1-2 frames of video data to eliminate decoding delays during switching. The cache release rule automatically releases the frame buffer resources corresponding to a viewpoint after the user leaves that viewpoint for 500 milliseconds, reducing long-term memory usage. The concurrency mutual exclusion control rules ensure that the 3D index table is set to read-only access logic, eliminating read / write contention; each viewpoint's file input / output thread is independently allocated handles, preventing contention for file I / O resources; and all decoders are uniformly scheduled through a semaphore queue to avoid multi-threaded concurrent decoding crashes.

[0038] Throughout the execution of step 102, which involves playing the corresponding video stream based on the first-person perspective identifier, a globally unified absolute time reference is continuously maintained, which specifically includes three layers of timing control logic.

[0039] The first phase involves maintaining the first-person perspective playback timing. The playback device loads the first-person video stream and assigns a dedicated first playback clock, calibrating its timing progress using a globally unified absolute time base throughout. When performing arbitrary progress operations such as speeding up, pausing, dragging, or rewinding on the first-person perspective, only the internal timing context of the first playback clock is modified. The globally unified absolute time base value and timing scale remain unchanged throughout the entire process and will not shift with single-view operations. During playback, the device continuously monitors for perspective switching commands from external devices and the interface.

[0040] The second phase is the time base locking phase triggered by the viewpoint switch. Upon receiving the viewpoint switch command to the second viewpoint identifier in real time, the system does not reset or deviate from the globally unified absolute time base, but retains the target absolute timestamp corresponding to the moment of switch as a common alignment anchor. Simultaneously, a completely isolated second playback clock is allocated to the second viewpoint identifier. When the second playback clock is initialized, it is directly bound to the currently unchanged globally unified absolute time base. The two clock threads run independently, and progress operations do not affect each other.

[0041] Next is the cross-view synchronous rendering driving phase. Based on a globally unified absolute time base that remains unchanged throughout the entire process, the target absolute timestamp at the moment of switching is locked. The 3D index table is queried to obtain the target frame byte offset corresponding to this timestamp in the second view. After performing byte jump reading and decoding, the second playback clock is driven to continuously render and output from the video frame corresponding to this target absolute timestamp. The second view playback is continuously anchored to the same global time base throughout, while the first view clock can still maintain a timing cache in the background. The same time anchor point can be reused when switching back and forth, achieving seamless time domain alignment.

[0042] This application's embodiment utilizes dual independent playback clocks paired with a fixed, globally unified absolute time base. On one hand, the timing operations of each perspective are completely isolated, eliminating the need to rebuild timing and decoding resources during switching, reducing the significant disk I / O overhead caused by repeatedly reading video, and lowering the latency of perspective switching. On the other hand, the global time base remains locked throughout, sharing the same target absolute timestamp as the rendering starting point before and after switching, achieving seamless temporal alignment of multi-view images and completely resolving issues such as black screens, frame skipping, and timing tearing during switching. This entire timing mechanism works in conjunction with the aforementioned 3D indexing and byte-level jump reading scheme to reduce stuttering in multi-view interactions through timing synchronization and frame addressing, adapting to high-precision synchronization scenarios such as VR panoramas, sports replays, and multi-channel security.

[0043] Traditional multi-view streaming media generally adopts a whole-screen segmented streaming approach. The client must fully retrieve all the preceding bitstreams for the current view before it can switch to another view, making it impossible to send range requests based on byte offsets. Furthermore, multi-view bitstreams share a single global decoder. Switching views clears the decoding context and repeatedly preloads a massive number of preceding segments, resulting in extremely high bandwidth consumption and noticeable stuttering during switching. In addition, after retrieving the bitstream, only basic decoding is performed, without frame matching based on global absolute timestamps, leading to timing offsets and screen tearing issues during transitions.

[0044] In this embodiment, the temporal dynamic file is a streaming file. Unlike local complete offline files, streaming temporal dynamic files have a 3D index table at the beginning of the file header, and video frame data is stored in blocks and segments according to viewpoint on the streaming media server. This allows clients to first retrieve metadata and then request any byte segment on demand, enabling multi-view switching playback without downloading the complete file. This embodiment combines streaming fixed-point range retrieval, independent viewpoint decoding, and global timestamps after decoding to solve the above problems, adapting to streaming remote multi-view interactive scenarios such as VR live streaming, online multi-camera sports events, and cloud-based medical aesthetic imaging.

[0045] Step 101 may include establishing a data communication connection with the server and receiving the file header and 3D index table of the temporal dynamic file. As an example, a data communication connection with the streaming media server can be established, a metadata-specific request can be initiated, and the complete file header data of the temporal dynamic file can be received and cached first. The built-in 3D index table in the file header can be fully parsed and stored in an in-memory hash retrieval pool. For example, the client establishes an HTTP / HTTPS long connection with the streaming media server, supporting standard byte range request interactions, and distinguishing between metadata retrieval and video segment retrieval requests. Only lightweight file header metadata is retrieved, without requesting any video frame segments, completing the local caching of the 3D index mapping relationship. Subsequent viewpoint switching and addressing all rely on local index lookups, eliminating the need to repeatedly request index information from the server, significantly reducing the frequency of network requests during interaction. The lightweight file header is retrieved first, eliminating the need to download the entire large video file to complete index parsing. This allows for rapid player initialization in weak network conditions and low bandwidth environments on mobile devices, shortening startup loading time. The fixed-point range request mechanism abandons the traditional full-track segmented sequential download logic, and only transmits a small number of byte segments corresponding to the target frame, which greatly reduces network bandwidth consumption and significantly shortens the buffer time for perspective switching in weak network scenarios.

[0046] In step 104, streaming, viewpoint-independent decoding, and timestamp-matched rendering can be completed based on the target byte offset. Specifically, a range request command carrying the target byte offset is first sent to the server. This range request command is a network request carrying the target byte offset and a preset byte length for the segment. The server only returns the corresponding interval encoded data segment and does not transmit irrelevant preceding video streams, in order to obtain and cache the data segment of the preset length starting from the target byte offset returned by the server. The preset length data segment is a single frame / a small number of consecutive frame encoded data divided by the server according to a fixed byte threshold, serving as the smallest unit for a single network transmission, balancing transmission efficiency and the flexibility of fixed-point jumps. After obtaining the second-view target byte offset through a 3D index table, the client assembles a range request command carrying the target byte offset and the preset segment length and sends it to the streaming media server through the established data communication connection. The server recognizes the Range field and only extracts the video frame encoded segment of the preset length starting from the offset and returns it to the client. The client caches this segment locally and does not download all historical streams before this viewpoint offset.

[0047] Secondly, cached data fragments can be used as target video frame encoding data, which is then input into a viewpoint decoder independently allocated for the second-viewpoint identifier. The viewpoint decoder is a decoding instance created independently for a single viewpoint, caching the reference frame, prediction frame, and bitstream parsing parameters for that viewpoint, and is completely isolated from other decoders. The viewpoint decoder maintains its own independent decoding context. This independent decoding context stores the corresponding viewpoint bitstream parsing state, reference frame cache, entropy decoding parameters, and timing markers. Switching to other views will not overwrite or clear this context, and switching back and forth does not require reloading the reference frame.

[0048] Then, the viewpoint decoder is used to perform viewpoint-level streaming decoding on the encoded data of the target video frame to generate the target video frame. Viewpoint-level streaming decoding independently parses only a small number of segmented bitstreams pulled from the current viewpoint. The execution process of viewpoint-level streaming decoding is independent of the decoding process of other views, does not depend on other viewpoint bitstreams, and the decoding threads are isolated, without competing for CPU and memory resources. Switching views does not destroy or rebuild the decoding cache, eliminating the need to repeatedly decode historical reference frames, reducing CPU computing power overhead, and eliminating repeated loading and stuttering when switching between multiple views; the isolation of each decoding thread avoids screen tearing and crashes caused by concurrent decoding of multiple bitstreams.

[0049] Next, the timestamp of the video stream corresponding to the decoded second-view identifier is compared with the target absolute timestamp. Based on the comparison result, target video frames whose timestamps match the target absolute timestamp are selected, thereby filtering out redundant frames that are ahead or behind within the segment. Finally, the target video frame is used as the rendering start frame for rendering output, so as to achieve seamless temporal alignment based on absolute timestamps when switching perspectives. Streaming segments have a small number of redundant frames before and after; by accurately matching globally unified timestamps, misaligned frames are eliminated, ensuring seamless transitions and black screens at the bitstream decoding level.

[0050] Figure 3 This is a schematic diagram illustrating a viewpoint-level streaming decoding principle provided in an embodiment of this application. Figure 3 As shown, the diagram fully illustrates the end-to-end streaming interaction process, from server-side storage and network transmission to local index parsing in the player, targeted segmentation requests, cache hit detection, and weak network degradation fallback. The flow logic of each module in the diagram is explained in sequence below.

[0051] The data source of the top-level overall link is the streaming time-domain dynamic file stored on the server. The data is transmitted to the client player via the network. The player executes two core business links in parallel: the left file header index parsing link and the right target frame fragmentation request link. The two links are finally aggregated to the bottom weak network degradation processing module.

[0052] The left side shows the file header index parsing process (corresponding to the execution logic of step 101 on the playback end). After establishing a network connection, the player prioritizes reading the lightweight file header data, which consists of only 3128 sections. The file header is parsed to extract the offset of the 3D index table within the file. Then, based on the index offset, the complete 3D index table, approximately 150KB in size, is downloaded to local memory. After the index loading is complete, the player has multi-view interactive addressing capabilities. The player allocates a minimum usage cache locally, with the caching rules being to retain the three most recent active views, caching three frames of video data before and after each view, with a total cache size of approximately 1.8MB. This cache is used to reuse already retrieved and decoded frame data, reducing redundant network requests.

[0053] The right side shows the target frame segment request chain (corresponding to the streaming fetch and decoding logic in step 104 of the playback end). After receiving the perspective switching command, the player reads the local cached 3D index table, looks up the table to obtain the byte offset and frame data volume corresponding to the target frame, with a single frame video encoding data of approximately 1.8MB. A range request command carrying the target byte offset is sent to the server. The server only returns the local video segment data starting from that offset, not the entire bitstream of the complete track. After receiving the segment data, the player executes the preloading control logic: if the segment download times out and triggers a preset threshold, the preloading operation of adjacent perspectives is canceled, and interpolation rendering of the preceding and following frames in the cache is used to achieve the screen transition, avoiding long black screen buffering.

[0054] The middle branch is for cache hit detection. Before initiating a segment request, the player first searches the least used local cache. If the target frame is already cached, the cached encoded data is reused directly without sending a network request to the server. If the cache is not hit, a segment request is sent to begin the normal download process.

[0055] The bottom weak network degradation unified processing logic diagram sets a single segment download timeout threshold of 200 milliseconds. When the cache misses or the segment download takes more than 200 milliseconds, the weak network degradation mechanism is triggered in a unified manner. Combined with the cache interpolation transition strategy, the basic playback interaction smoothness is ensured in a weak network environment.

[0056] Figure 3 This document fully covers all technical features of step 101 (lightweight index preloading), step 104 (fixed-point byte range request), independent viewpoint decoding, local frame buffer reuse, and 200ms timeout for weak network degradation. It fully discloses key quantitative parameters such as file header, index table, single-frame data, and buffer capacity, clearly demonstrating the complete execution flow of "index first, frame retrieval on demand, buffer reuse, and weak network fallback" in streaming scenarios. Initial loading requires only approximately 150KB to begin interaction, without waiting for the full file to download. Each viewpoint switch transmits only approximately 200KB~600KB, saving over 95% of the transmission volume compared to downloading the entire file. A basically usable interactive experience can still be maintained in weak network environments.

[0057] Traditional multi-view video files only store pure image streams, with spatial annotation information such as facial features, medical devices, and treatment areas stored externally and not bound to the global absolute timestamp of the video frames. When switching between different viewpoints, it is necessary to re-annotate the points for each video stream individually, making it impossible to share a unified 3D spatial coordinate system. This results in a large amount of repetitive annotation work, and the spatial annotations become misaligned after switching viewpoints, failing to synchronously adapt to facial poses during projection. This leads to poor visualization in scenarios such as medical aesthetic diagnosis and treatment comparisons, and pre- and post-operative multi-view analysis. Furthermore, spatial data and video files are independent of each other, making it impossible to synchronously match the spatial metadata at the corresponding moment when retrieving images, resulting in low interaction efficiency. This application adds an AI metadata layer spatiotemporal anchoring mechanism, along with a corresponding AI metadata spatiotemporal anchoring principle diagram, to achieve integrated encapsulation of temporal and spatial information and automatic reprojection from multiple viewpoints.

[0058] In this embodiment, the temporal dynamic file may further include an AI metadata layer. The AI ​​metadata layer is an independently partitioned data storage layer within the temporal dynamic file, stored separately from video frame data. It specifically carries structured metadata such as spatial features, point coordinates, and diagnostic information extracted by AI algorithms, and shares the same global absolute timestamp system as the video frames. The AI ​​metadata layer stores spatiotemporal anchored metadata data packets, which are the smallest storage units of the AI ​​metadata layer. Each data packet is uniquely bound to a set of physical absolute timestamps and a complete set of physical three-dimensional coordinates. These two are anchored one-to-one, representing the position information of the face, treatment instruments, and surgical area in a unified three-dimensional space at that precise moment. The physical absolute timestamp is of the same origin as the global UTC standard timestamp of the video frame, serving as a unified temporal binding benchmark for video images and spatial metadata, ensuring accurate matching between images and three-dimensional points at the same moment. Physical space 3D coordinates are a set of 3D points in a unified world coordinate system obtained by the AI ​​engine (built-in AI feature extraction algorithm module) after extracting facial feature points, lifting treatment positions, instrument positions, and treatment areas from multiple (e.g., 15) synchronously acquired video frames. It is not affected by the shooting angle or camera position offset of a single camera and is a universal spatial reference for all perspectives.

[0059] In step 104, when rendering the output video frame, a spatiotemporal anchoring metadata data package matching the target absolute timestamp is obtained, and the physical space 3D coordinates are projected onto the 2D screen for overlay display based on the camera parameters of the current viewpoint. Specifically, after obtaining the target video frame matching the target absolute timestamp through byte jump reading and decoding, the player synchronously reads the temporal dynamic file AI metadata layer, using the target absolute timestamp at the current switching moment as the search key to accurately match and obtain the spatiotemporal anchoring metadata data package corresponding to the same moment. The data package reads a complete set of unified physical space 3D coordinates. The player reads the exclusive camera parameters corresponding to the current second-viewpoint identifier, including the shooting focal length, spatial position, rotation angle, and lens distortion calibration coefficient of the camera position. Only the camera parameters are switched, without modifying or recalculating the global physical space 3D coordinates throughout the process. Then, based on the camera parameters of the current viewpoint, the unified physical space 3D coordinates stored in the spatiotemporal anchoring metadata data package are transformed by perspective projection, mapping the 3D facial feature points, instrument points, and treatment area contours to the pixel coordinate system of the second-view 2D video screen, generating a 2D annotation layer adapted to the current camera position. The 2D annotation layer generated by reprojection is superimposed on the decoded target video frame and rendered synchronously. When switching between different perspectives (side view, front view, etc.), only the camera parameters of the corresponding perspective are replaced to reuse the same set of 3D coordinates, and the point reprojection is completed automatically without the need to re-identify the image features and regenerate the annotations.

[0060] Figure 4 This is a schematic diagram illustrating the spatiotemporal anchoring principle of AI metadata provided in an embodiment of this application. Figure 4 As shown, the overall process is divided into two main independent workflow areas: the acquisition stage and the playback stage. It fully demonstrates the entire implementation logic of the spatiotemporal anchoring metadata data, from generation and encapsulation to automatic reprojection during multi-view playback. The core design rule is marked at the bottom of the drawing: when switching perspectives, only camera parameters are changed; the 3D coordinates remain unchanged, and reprojection is completed automatically. Detailed explanations of each area are as follows.

[0061] The upper half of the area is the acquisition unit, which is the process of generating and encapsulating spatiotemporal anchoring metadata data packets. This area corresponds to the preprocessing steps at the acquisition end of the temporal dynamic file, used to generate metadata with unified 3D coordinates and bound absolute timestamps, completing the flow steps. The input source is 15 multi-view video frames acquired synchronously, with each frame carrying a globally unified physical absolute timestamp. In the AI ​​feature extraction stage, the 15 video frames are uniformly fed into the AI ​​engine, which automatically identifies and extracts key elements within the image: facial feature points, treatment operation positions, treatment area contours, and the spatial positions of medical aesthetic instruments. In the 3D coordinate transformation stage, the AI ​​engine uniformly converts the 2D image feature points from each viewpoint to a set of globally unified physical space 3D coordinates, eliminating the 2D point position deviations caused by the single camera shooting angle and forming a universal spatial reference for all views. Then, the metadata is packaged and written, binding the physical absolute timestamp corresponding to the current frame with the entire set of physical space 3D coordinates, encapsulating it into a spatiotemporal anchoring metadata data packet; all metadata data packets are uniformly written into the AI ​​metadata layer independently divided in the temporal dynamic file, and finally packaged to generate a complete temporal dynamic file for the playback end to load and use.

[0062] The lower half of the area represents the playback stage, specifically the multi-view automatic reprojection and overlay rendering process. This area corresponds to the execution logic of rendering and overlaying AI metadata in step 104 of the playback end. The diagram includes three parts: the left side view, the middle unified 3D physical space coordinate system, and the right front view. The middle reference layer is a globally unified 3D physical space coordinate system that stores the complete set of 3D coordinates generated by the acquisition end (facial feature points, injection path, instrument position, treatment area anchor points, etc.). This set of 3D coordinates is a fixed global reference and will not be modified when the viewpoint changes. The left and right dual-view images represent video rendering images from two different camera positions: the side view and the right front view. Each viewpoint is associated with exclusive camera parameters (focal length, camera position, rotation angle, lens distortion coefficient). The core logic of reprojection is that when the playback end switches to any viewpoint, only the camera parameters corresponding to the current viewpoint are replaced, while the unchanging global 3D physical space coordinates are reused throughout. Based on the new camera parameters, all points in the 3D coordinate system are perspective-converted and mapped to the current viewpoint's 2D image, automatically generating a labeling layer adapted to that camera position. Layer overlay output involves overlaying the face, instrument, and treatment area annotation layers generated by reprojection onto the corresponding viewpoint video frames for synchronous rendering, achieving precise matching of spatial annotations for synchronous display of multi-view images.

[0063] like Figure 3 As shown, when switching perspectives, only the camera parameters are changed, and the 3D coordinates remain unchanged while automatic reprojection occurs. This differs from traditional solutions where each perspective is individually identified and 2D annotations are generated. In this embodiment, AI 3D feature extraction is completed only once at the acquisition end. When switching perspectives at the playback end, AI inference does not need to be re-inferred; annotation adaptation can be completed simply by converting camera parameters. This significantly reduces the computational power consumption at the playback end while ensuring that the spatial position of the same diagnostic point corresponds completely under different perspectives, without any misalignment or offset issues.

[0064] Traditional multi-view video encapsulation files do not partition and isolate the video from multiple viewpoints, resulting in a chaotic mix of video frames from all camera positions. Furthermore, they lack a unified index pre-built in the file header, requiring the playback device to traverse all video frames to locate the target image after loading the file. This generates significant magnetic I / O overhead, leading to high latency during viewpoint switching and stuttering. Moreover, traditional indexes only record a single mapping between time and frame position, failing to simultaneously associate viewpoint identifiers and frame byte lengths, and thus cannot support point-to-point byte jumps or independent decoding logic on the playback device. This application addresses this by employing two core processes in its acquisition end: isolated storage for different viewpoints and pre-building a 3D index in the file header. These processes generate standardized temporal dynamic files adapted for low-latency interaction on the playback device.

[0065] Figure 5 This is a flowchart illustrating a method for generating multi-view interactive video files provided in an embodiment of this application. Figure 5 As shown, this multi-view video interactive file generation method is applied to the acquisition end and may include steps 501-502, which will be described in detail below.

[0066] Step 501: Obtain the temporal dynamic file synthesized from multiple video frame sequences.

[0067] The data area of ​​the temporal dynamic file adopts a storage structure that is segmented and isolated by viewpoint. This data area is a dedicated storage partition in the temporal dynamic file for storing multiple video encoded frames. Using this segmented and isolated storage structure, a unique viewpoint identifier is pre-assigned to each acquisition camera. All video frame encoded data corresponding to the same viewpoint identifier are continuously written to the same contiguous storage segment. Video frame data corresponding to different viewpoint identifiers are divided into independent storage blocks, with storage isolation boundaries reserved between the blocks to prevent overlap and mixing. Specifically, all video frame data corresponding to the same viewpoint identifier are stored contiguously in the data area, while video frame data with different viewpoint identifiers are stored in isolated blocks within the same data area.

[0068] After the acquisition end completes the pre-loop buffer frame extraction and global timecode frame-level alignment and trimming, it obtains a multi-channel video frame sequence with uniform length and matching frame timestamps. The acquisition end main control module performs storage partitioning planning for each video frame sequence according to preset viewpoint identifiers and pre-allocates space in the data area. Specifically, a continuous storage address range is allocated for each viewpoint identifier, and all video frame encoded data corresponding to the same viewpoint identifier are continuously written into the corresponding storage range in chronological order. Fixed isolation storage boundaries are set between video frame blocks corresponding to different viewpoint identifiers to achieve physical isolation of video data from different camera positions and prevent them from interleaving and mixing. Finally, the multi-channel video frame sequences are merged and encapsulated to generate a time-domain dynamic file basic carrier with a viewpoint-segmented isolated storage structure.

[0069] Frames from the same viewpoint are stored continuously, while different viewpoints are isolated in blocks. When the playback end reads a single stream of video from a single viewpoint, it can continuously read disk sectors, reducing the number of random disk I / O jumps and lowering hardware read / write overhead. Viewpoint data is physically partitioned and isolated, and the three-dimensional index table can be quickly grouped and retrieved by viewpoint identifier, improving the table lookup efficiency of the playback end. The multi-channel video storage arrangement is organized at the file level, eliminating the addressing and traversal overhead caused by the interleaving and mixing of multi-viewpoint frames in traditional files, and providing underlying storage structure support for byte-point jumps at the playback end.

[0070] Step 502: Construct an index area in the header of the time-domain dynamic file to generate a three-dimensional index table.

[0071] The file header is a lightweight, independent storage partition for the temporal dynamic file header. It does not store video image bitstreams but has a dedicated index area for storing 3D index table metadata. The file header size is much smaller than the data area, supporting priority and fast download and parsing in streaming scenarios. The index area is an independent binary storage area within the file header, specifically used to store and serialize the 3D index table. It is stored separately from the other basic metadata in the file header, making it easy for the playback end to read the index data separately.

[0072] The 3D index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers, and the byte offsets and frame length information of the corresponding video frames in the data area. Upon receiving a viewpoint switching command, the 3D index table allows the playback device to directly query and generate the target byte offset based on the current target's absolute timestamp and the identifier of the second viewpoint to be switched, enabling the playback device to perform byte-level jumps from the temporal dynamic file and independently decode the target video frame.

[0073] Specifically, the acquisition end divides a dedicated index area in the header of the temporal dynamic file, physically separated from the data area storing video frames. The index area only stores lightweight index metadata and does not write to the image bitstream. Each video frame's encoded data in the data area is traversed, extracting the frame's bound global absolute timestamp, the viewpoint identifier corresponding to its camera position, and reading the frame's starting byte offset and total byte length within the data area. These four parameters are bound into a complete index record. All index records are sorted in ascending order by absolute timestamp, uniformly serialized, and encoded before being stored in the file header index area, forming a three-dimensional index table that can be quickly hashed for retrieval. Finally, the three-dimensional index table has built-in joint retrieval logic, supporting a two-dimensional joint query from the playback end using "target absolute timestamp + second viewpoint identifier." A single table lookup can output the complete byte offset and frame length parameters corresponding to the target frame, eliminating the need for multiple rounds of traversal and matching.

[0074] The index table is stored in the lightweight file header. In streaming scenarios, the playback end only needs to download the file header index data, without having to fetch the complete video data to complete addressing initialization. A triplet-based joint mapping structure enables one-click binding of time, viewpoint, and file storage location. When switching viewpoints, the playback end does not need to traverse the entire frame; it can directly obtain the jump offset parameters by looking up the table. The index synchronously records the frame length, allowing the playback end to accurately read the corresponding length segment after a byte jump, avoiding unnecessary bitstream reading and supporting viewpoint-level independent streaming decoding.

[0075] Traditional multi-camera acquisition systems, in order to save memory, only begin recording video after a trigger signal arrives, failing to capture footage before the event occurs and missing crucial processes before accidents, cosmetic procedures, and experiments are triggered. Furthermore, each camera uses its own local relative timestamp, which, due to hardware crystal oscillator drift and device startup time differences, results in misaligned video frames, making frame-by-frame matching impossible. If directly packaged into a video file, playback will inevitably experience time tearing and alignment misalignment when switching perspectives, requiring significant additional real-time timing correction computation. Simultaneously, the lack of a standardized trimming process leads to redundant misaligned frames continuously occupying storage space, resulting in redundant file size. This application adds a complete acquisition-end preprocessing process before synthesizing the temporal dynamic file in step 501, consisting of two core stages: cyclic buffer frame extraction and timecode frame-level alignment trimming. This process standardizes multiple original video streams from the source, unifying the global timing benchmark, and providing compliant, time-matched original frame material for subsequent multi-view storage and 3D index construction. The complete process consists of two core stages: cyclic buffer frame extraction and timecode frame-level alignment trimming.

[0076] The first step is frame extraction from the circular buffer. The circular buffer (circular circular frame buffer) is a circular rolling storage area in the memory of the acquisition end. It has a preset fixed duration buffer window and follows the overwrite update rule of old frames being discarded and new frames being written. It continuously stores the latest acquired video frames, and historical frames that have exceeded the preset duration are automatically discarded from the head of the buffer. It always retains the most recent continuous frame, eliminating the need to store massive amounts of video on the ground for the entire time, which greatly reduces memory usage.

[0077] Specifically, multiple acquisition devices can be controlled to continuously write captured video frames into a circular buffer, maintaining a video frame sequence of preset duration within the buffer. Multiple acquisition devices can be controlled to synchronously and continuously acquire video frames, with each raw frame written to the local circular buffer at the acquisition end in real time. The buffer is updated cyclically according to a preset duration window, with new frames continuously written to the end of the buffer and older frames exceeding the preset duration automatically evicted from the beginning. The buffer always retains consecutive video frames close to the preset duration, achieving rolling storage without requiring long-term occupation of massive local disk space. The acquisition devices can be multiple synchronous shooting terminals (industrial cameras, synchronized mobile phones, etc.), each device bound to a unique viewpoint identifier, acquiring raw video footage in parallel.

[0078] The trigger signal is an external event trigger command (trial alarm, medical aesthetic operation initiation, manual trigger, collision detection signal, etc.). The command carries a precise global absolute timestamp, serving as the baseline boundary between the scenes before and after the event. In response to the trigger signal, pre-buffered video frames prior to the arrival time of the trigger signal are extracted from the circular buffer. The pre-buffered video frames are historical cached frames retained in the circular buffer before the arrival time of the trigger signal, completely recording the dynamic scene before the trigger event, used for accident retrospective and pre-operative state retention. The acquisition end continuously polls the external trigger signal interface. Once a trigger command carrying a precise global absolute timestamp of the trigger time is captured, the current circular buffer memory data is immediately locked, and the automatic replacement logic of old frames is paused to prevent the scene before the event from being overwritten and lost by new frames. Using the trigger time as the time sequence boundary point, all cached frames within a preset time range preceding the trigger time are extracted from the locked circular buffer as pre-buffered video frames, completely retaining the dynamic scene of the entire process before the trigger event.

[0079] Furthermore, the system continues to acquire post-recorded video frames after the trigger signal arrives. While extracting pre-buffered frames, a continuous acquisition command is simultaneously issued to control all acquisition devices to continuously acquire new video frames of a preset duration, forming a complete video frame sequence corresponding to the temporal dynamic file. Post-recorded video frames are those continuously acquired by all acquisition devices after the trigger signal is captured, completely recording the entire process after the event occurs. The complete video frame sequence is formed by stitching together pre-buffered and post-recorded video frames according to their global absolute timestamps, creating continuous raw video footage corresponding to a single acquisition device. After all multiple acquisition devices have completed frame extraction, the data is uniformly aggregated and transmitted to the industrial control computer for the next stage of frame-level alignment processing.

[0080] Figure 6 This is a schematic diagram illustrating the principle of a full-frame circular buffer provided in an embodiment of this application. Figure 6 As shown, the basic caching framework divides the overall cache into three logical areas: an old frame eviction area, a new frame writing area, and a 5-second total cache window. The cache as a whole adopts a circular frame storage structure, following a first-in-first-out (FIFO) overwrite update rule. Old frame eviction occurs when new frames continuously fill the 5-second window; historical frames exceeding 5 seconds at the front of the cache are automatically cleared and released to prevent unlimited memory occupation. New frame writing involves 15 capture phones continuously capturing video frames and writing them to the end of the cache in real time, continuously updating the cached image. The 5-second window is a circular buffer with a fixed maximum storage duration of 5 seconds, permanently retaining the most recently captured 5 seconds of video frames.

[0081] The trigger boundary logic sets the arrival of the trigger signal as the timing segmentation node, defining the trigger moment as the event baseline boundary point, thus dividing the footage into two segments: pre-buffered frames and post-recorded frames. For example, a 2-second, 60-frame pre-buffer means capturing 60 frames (2 seconds) of historical footage from the buffer, starting from the trigger moment, completely preserving the dynamic process before the trigger event occurs. A 3-second, 90-frame post-recorded frame means that after capturing the trigger signal, all acquisition devices continuously add and record 90 frames (3 seconds) of footage, recording the complete dynamics after the event occurs.

[0082] The frame extraction and splicing output involves splicing the pre-buffered 2-second (60 frames) and the subsequently recorded 3-second (90 frames) in chronological order to form a complete frame sequence for a single device. After all 15 mobile phones have completed frame extraction, a total of 2250 frames of raw video material are produced, which are then transmitted to the industrial control computer for timecode frame alignment processing.

[0083] Figure 6The underlying mechanism of the event footage is fully disclosed. By using a fixed 5-second loop window, 2-second pre-storage + 3-second post-recording quantization parameters, the entire event process can be captured without continuous storage throughout the day, which greatly reduces the memory and disk resource consumption of the acquisition device.

[0084] The second step is timecode frame alignment, which involves acquiring the timecode that marks the sequence of video frames for each channel. The timecode is a globally standardized UTC time sequence code, i.e., a global absolute timestamp, uniformly bound to each captured video frame by a dedicated synchronization and timing device. It is unaffected by local clock drift or power-on time differences from a single camera and serves as a unified timing reference for multiple video streams. During the acquisition phase, the synchronization and timing device has already bound a unified global timecode to each frame of each video stream, and each frame contains a unique UTC absolute timestamp timing marker, serving as a unified reference for the alignment of all tracks.

[0085] Frame-level alignment of multiple video frame sequences is performed based on timecodes. This alignment uses a globally unified timecode as the matching benchmark and horizontally adjusts the frame arrangement offset within 15 independent video tracks to ensure that each global absolute timestamp has a corresponding matching frame across all video tracks, achieving frame-by-frame temporal correspondence. The industrial control computer loads the independent video tracks corresponding to all acquisition devices and, using the frame's built-in global timecode as the matching basis, horizontally fine-tunes the temporal offset of frames within each track, completing the initial frame-by-frame temporal alignment of the 15 video tracks, ensuring that frames corresponding to the same absolute timestamp are horizontally aligned in the temporal dimension.

[0086] Furthermore, frames with misaligned timecodes in each video frame sequence are trimmed. These misaligned frames are isolated frames at the beginning and end of each video track due to device startup time differences and crystal oscillator deviations. These frames lack matching timecodes for other tracks and are considered temporally redundant data. For example, a frame traversal identification logic is used to search each video track for isolated frames at the beginning and end that lack cross-track timecode matching, and these temporally misaligned redundant frames are uniformly trimmed and deleted. The trimming rule is to retain only the intermediate frame segments that can match each other across all tracks. After trimming, the total duration and total number of frames for all video tracks are completely uniform, and any global absolute timestamp within a track has a one-to-one matching frame in all the remaining 15 video tracks. The regularized multi-track matching frame sequence is used as input material and sent to step 501 for split-view isolation storage and encapsulation to generate a temporal dynamic file base carrier. This ensures that the length of each aligned video frame sequence is consistent and the timecodes of corresponding frames match.

[0087] Figure 7 This is a schematic diagram illustrating a time code frame alignment principle provided in an embodiment of this application. For example... Figure 7 As shown, the alignment principle includes a complete four-stage process for 15 multi-view videos, from original decoding, frame-by-frame alignment, cropping redundant frames, and standardizing and synthesizing temporal dynamic files. The four stages are explained step by step from top to bottom.

[0088] The first stage involves global timecode encoding for multiple video streams. Using dedicated timecode synchronization equipment, a global UTC standard timecode is uniformly bound to each frame within the original video tracks output from the 15 acquisition phones, with each frame carrying a unique physical absolute timestamp. This resolves the issue of inconsistent timing benchmarks for different video streams caused by hardware crystal oscillator drift and startup time differences among multiple acquisition phones, providing a unified timing standard for subsequent frame alignment.

[0089] The second stage involves frame-level timing alignment of the 15 video tracks. The industrial control computer loads 15 independent video tracks and, relying on timing processing software, uses the built-in global timecode of each frame as the matching benchmark to fine-tune the timing offset of the frames within each track horizontally. This completes the initial timing alignment of the 15 video streams frame by frame, ensuring that the images corresponding to the same absolute timestamp correspond one-to-one horizontally in the timing dimension.

[0090] The third stage involves trimming redundant and misaligned frames at the beginning and end. The editing tool is used to traverse all 15 video tracks, identify isolated frames (misaligned frames) at the beginning and end of the tracks that do not have timecode matching across tracks, and uniformly delete these temporally redundant frames, retaining only the middle valid frame segments that can be matched between all tracks.

[0091] The fourth stage involves standardizing the frame sequence output and synthesizing a temporal dynamic file. After trimming, the total duration and number of frames of the 15 video tracks are completely unified, and any global absolute timestamp within a track can find a matching frame in the other 14 video tracks. Finally, through a self-developed underlying encapsulation algorithm, the normalized multi-channel synchronized video is combined into a dedicated Tang / T4D format temporal dynamic file. The file contains a built-in viewpoint isolation storage structure and a three-dimensional index table, which is provided to the playback terminal for loading to achieve low-latency multi-view interaction.

[0092] Figure 7 The acquisition end employs a standardized processing flow for multi-channel video timing, with four progressively advancing steps to eliminate inherent timing misalignments in multiple acquisition devices from the ground up. This ensures that all video frames in the output time-domain dynamic file share the same global absolute time reference. This provides timing-matched material for the acquisition end's step 501 (segmented viewpoint storage) and step 502 (construction of a 3D index table), and also lays the timing foundation for the entire set of innovative mechanisms on the playback end, including dual independent playback clocks, byte-level jumps, AI metadata spatiotemporal anchoring, and seamless cross-viewpoint time-domain alignment.

[0093] Figure 8 This is a schematic diagram of the structure of a multi-view video interaction device provided in an embodiment of this application. Figure 8 As shown, the multi-view video interaction device 800 is applied to the playback end and may include a first acquisition module 801, a receiving module 802, a query module 803, and a jump module 804.

[0094] The first acquisition module 801 is used to acquire the temporal dynamic file and parse the three-dimensional index table in the file header of the temporal dynamic file. The three-dimensional index table establishes a mapping relationship between the absolute timestamp, the viewpoint identifier and the byte offset of the video frame data in the file.

[0095] The receiving module 802 is used to play the corresponding video stream based on the first viewpoint identifier and to receive viewpoint switching instructions in real time. The viewpoint switching instructions are used to indicate switching to the second viewpoint identifier.

[0096] The query module 803 is used to respond to the viewpoint switching command, obtain the target absolute timestamp of the currently playing target, and query the three-dimensional index table based on the target absolute timestamp and the second viewpoint identifier to determine the target byte offset of the target video frame corresponding to the second viewpoint identifier and the target absolute timestamp.

[0097] The jump module 804 is used to perform byte-level jumps from the temporal dynamic file based on the target byte offset to read and decode the target video frame and render it based on the target absolute timestamp.

[0098] Figure 9 This is a schematic diagram of the structure of a multi-view video interactive file generation device provided in an embodiment of this application. Figure 9 As shown, the multi-view video interactive file generation device 900 is applied to the acquisition end and may include a second acquisition module 901 and a generation module 902.

[0099] The second acquisition module 901 is used to acquire a temporal dynamic file synthesized from multiple video frame sequences. The data area of ​​the temporal dynamic file adopts a storage structure that is isolated by viewpoint. All video frame data corresponding to the same viewpoint identifier are stored continuously in the data area, while video frame data with different viewpoint identifiers are stored in blocks in the data area.

[0100] The generation module 902 is used to build an index area in the file header of the time-domain dynamic file to generate a three-dimensional index table.

[0101] The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers, and the byte offsets and frame length information of the corresponding video frames in the data area. When a viewpoint switching instruction is received, the three-dimensional index table allows the playback terminal to directly query and generate the target byte offset based on the current target absolute timestamp and the second viewpoint identifier to be switched, so as to support the playback terminal to perform byte-level jumps from the temporal dynamic file and independently decode the target video frame.

[0102] The first acquisition module 801, the receiving module 802, the query module 803, and the jump module 804 can be used to execute steps 101-104 in the embodiments of the above-mentioned multi-view video interaction method. The second acquisition module 901 and the generation module 902 can be used to execute steps 701-702 in the embodiments of the above-mentioned multi-view video interaction file generation method. For the specific implementation of these modules and more details, please refer to the corresponding method section. They will not be described in detail here.

[0103] This application also provides a multi-view video acquisition device. This multi-view video acquisition device can be a portable acquisition terminal integrating a gimbal, multiple cameras, synchronous timing, and storage / computing capabilities. For example, it can be a multi-camera aerial / ground image acquisition device that can simultaneously output multiple synchronous video streams and locally encapsulate them into the tang / t4d format time-domain dynamic files of this application, completing the entire preprocessing and file encapsulation process without the need for an external industrial control computer. This multi-view acquisition device may include a gimbal assembly, at least one camera module, a memory, and a processor.

[0104] The gimbal assembly is the device's support and attitude adjustment mechanism. It can consist of a support platform, multi-axis drive motors, and an attitude encoder, and is used to support and drive at least one camera module for attitude adjustment. The support platform provides a unified installation benchmark for multiple modules, ensuring that the spatial positions of all camera modules are fixed. The multi-axis drive motors support three-axis linkage attitude adjustment, adapting to multi-angle synchronous shooting. The attitude encoder collects the gimbal's rotation and pitch angles in real time, which can be simultaneously written to the AI ​​metadata layer for 3D coordinate reprojection correction.

[0105] The camera module is a multi-channel synchronous image acquisition unit, equipped with an industrial-grade image sensor and a fixed-focus optical lens. Multiple modules share a synchronous timing signal, ensuring complete alignment of frame acquisition times and outputting a synchronous raw video frame sequence. For example, a single device can be equipped with 15 camera modules, matching the 15-channel synchronous acquisition embodiment of this application. At least one camera module can be mounted on the platform of the pan-tilt unit and may include an optical lens, image sensor, etc., for acquiring video frame sequences. As an example, a synchronous timing module may also be included, electrically connected to all camera modules, for uniformly marking each acquired video frame with a global standard timecode.

[0106] The memory is communicatively connected to the gimbal assembly, camera module, synchronization module, and memory, and is used to store computer programs. The processor, when executing the computer program, implements any of the multi-view video interactive file generation methods in the embodiments of this application.

[0107] Based on the aforementioned multi-view video acquisition equipment, in one example, the processor can control the gimbal component to drive the motor and adjust the platform to the target shooting posture, while the posture encoder transmits posture parameters in real time. The synchronization timing module sends out a synchronization exposure signal, driving all camera modules on the platform to synchronously acquire video frames. The acquired raw video frames are stored in the circular buffer of the memory in real time, maintaining a 5-second rolling buffer window. In response to an external trigger signal, a 2-second pre-buffered frame is extracted from the circular buffer, and a 3-second post-frame is continuously recorded, splicing them together to generate a single complete frame sequence. Relying on the global timecode marked by the synchronization timing module, frame-level alignment is performed on 15 video frames, and misaligned redundant frames are trimmed, outputting a multi-frame sequence with perfectly matched timing. Finally, the regularized multi-frame sequence is written to the memory data area in isolation according to the viewpoint. An independent index area is built in the file header, generating a three-dimensional index table containing absolute timestamps, viewpoint identifiers, byte offsets, and frame lengths, encapsulating and generating a complete temporal dynamic file with an AI metadata layer and storing it locally.

[0108] This multi-view video acquisition device integrates gimbal multi-channel shooting, hardware synchronization timing, and local computing and storage functions. It eliminates the need for an additional industrial control computer or external synchronization clock, independently completing the entire process from shooting to standardized file packaging, much like an all-in-one acquisition device. The device is lightweight and portable, suitable for outdoor aerial photography, medical aesthetics sites, and mobile acquisition scenarios at sporting events. Hardware level synchronization triggers multi-channel camera module acquisition, coupled with a globally unified timecode, eliminating multi-camera frame timing misalignment at the hardware level. This saves significant computational power for timing correction in post-production software, resulting in a unified timing reference for the packaged time-domain dynamic files and seamless screen tearing during playback. The gimbal attitude encoder outputs camera attitude parameters in real time, and the processor can simultaneously write these parameters into an AI spatiotemporal anchoring metadata package. When reprojecting 3D coordinates at the playback end, the gimbal calibration parameters can be combined to further improve annotation accuracy, adapting to the needs of multi-view AI spatial visualization analysis. The entire hardware architecture can be directly applied to aerial multi-view gimbals and ground-based multi-camera image acquisition equipment, uniformly outputting standardized tang / t4d temporal dynamic files. Combined with the playback terminal interaction method of this application, it enables low-latency, time-series-off-shift multi-view switching, enhancing the professional playback and analysis capabilities of image equipment.

[0109] This application also provides a computer-readable storage medium storing a program that can be loaded by a processor and executed by any of the multi-view video interaction methods or multi-view video interaction file generation methods in this application.

[0110] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0111] The above examples illustrate this application only to aid understanding and are not intended to limit its scope. Those skilled in the art to which this application pertains can make various simple deductions, modifications, or substitutions based on the ideas presented.

Claims

1. A multi-view video interaction method, characterized by, Applied to the playback end, including: Establish data communication with the streaming media server, initiate a metadata-specific request, prioritize receiving and caching the complete header data of the time-domain dynamic file, parse the three-dimensional index table built into the header and store it in the memory hash retrieval pool, and parse the three-dimensional index table in the header of the time-domain dynamic file. The time-domain dynamic file is a streaming file, and the three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers and byte offsets of video frame data in the file. Play the corresponding video stream based on the first perspective identifier and receive perspective switching instructions in real time. The perspective switching instructions are used to indicate switching to the second perspective identifier. The first perspective refers to the perspective corresponding to the video stream that is currently playing and the screen is continuously rendered and output. In response to the viewpoint switching command, the target absolute timestamp of the currently playing video is obtained, and the three-dimensional index table is queried based on the target absolute timestamp and the second viewpoint identifier to determine the target byte offset of the target video frame corresponding to the second viewpoint identifier and the target absolute timestamp. Based on the target byte offset, a byte-level jump is performed from the temporal dynamic file to read and decode the target video frame, and the rendering output is based on the target absolute timestamp.

2. The multi-view video interaction method of claim 1, wherein, Play the corresponding video stream based on the first-person perspective identifier and receive perspective switching commands in real time, including: During the playback of the corresponding video stream based on the first-person perspective identifier, a globally consistent absolute time reference is maintained; When receiving the viewpoint switch and switching to the second viewpoint identifier in real time, the globally unified absolute time base remains unchanged; Based on the globally unified absolute time reference, the target absolute timestamp corresponding to the moment of switching is obtained, and the video stream corresponding to the second viewpoint identifier is driven to start rendering and output from the video frame of the target absolute timestamp, so as to achieve seamless temporal alignment during viewpoint switching.

3. The multi-view video interaction method according to claim 1, characterized in that, The step of performing a byte-level jump from the temporal dynamic file based on the target byte offset to read and decode the target video frame includes: Send a range request instruction carrying the target byte offset to the server to obtain and cache the data fragments of a preset length starting from the target byte offset returned by the server; The cached data fragments are encoded as target video frame data and input into a viewpoint decoder independently assigned to the second viewpoint identifier; The target video frame is generated by performing viewpoint-level streaming decoding on the encoded data of the target video frame using the viewpoint decoder. The viewpoint decoder maintains an independent decoding context, and the execution process of the viewpoint-level streaming decoding is independent of the decoding process of other viewpoints.

4. The multi-view video interaction method according to claim 1, characterized in that, The rendering output based on the target absolute timestamp includes: The timestamp of the video stream corresponding to the decoded second perspective identifier is compared with the target absolute timestamp; Based on the comparison results, target video frames whose timestamps match the target absolute timestamp are selected. The target video frame is used as the starting frame for rendering output to achieve seamless temporal alignment based on absolute timestamps when the viewpoint changes.

5. The multi-view video interaction method according to claim 1, characterized in that, The time-domain dynamic file also includes an AI metadata layer, which stores spatiotemporal anchored metadata data packets. These spatiotemporal anchored metadata data packets are bound to physical absolute timestamps and physical three-dimensional coordinates. The rendering output based on the target absolute timestamp includes: When rendering and outputting the video frame, a spatiotemporal anchoring metadata data packet matching the absolute timestamp of the target is obtained, and the three-dimensional coordinates of the physical space are projected onto the two-dimensional screen for superimposed display based on the camera parameters of the current viewpoint.

6. A method for generating multi-view interactive video files, characterized in that, Applications at the acquisition end include: A temporal dynamic file synthesized from multiple video frame sequences is obtained. The data area of ​​the temporal dynamic file adopts a storage structure that is isolated by viewpoint. All video frame data corresponding to the same viewpoint identifier are stored continuously in the data area, while video frame data with different viewpoint identifiers are stored in blocks in the data area. An index area is constructed in the header of the time-domain dynamic file to generate a three-dimensional index table; The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers, and the byte offsets and frame length information of the corresponding video frames in the data area. When a viewpoint switching instruction is received, the three-dimensional index table allows the playback terminal to directly query and generate the target byte offset based on the current target absolute timestamp and the second viewpoint identifier to be switched, so as to support the playback terminal to perform byte-level jumps from the time-domain dynamic file and independently decode the target video frames. Before obtaining the temporal dynamic file synthesized from multiple video frame sequences, the method further includes: Multiple acquisition devices are controlled to continuously write the acquired video frames into a circular buffer, and a video frame sequence of a preset duration is maintained in the circular buffer; In response to a trigger signal, pre-buffered video frames before the arrival time of the trigger signal are extracted from the circular buffer, and post-recorded video frames after the arrival time of the trigger signal are continuously acquired to form a complete video frame sequence corresponding to the time-domain dynamic file. Obtain the timecodes that mark each video frame sequence, perform frame-level alignment on the multi-channel video frame sequences based on the timecodes, and trim off frames in each video frame sequence whose timecodes are not aligned, so that the length of each aligned video frame sequence is consistent and the timecodes of corresponding frames match.

7. A multi-view video interactive device, characterized in that, The multi-view video interaction device, applied to the playback end, includes: The first acquisition module is used to establish data communication with the streaming media server, initiate metadata-specific requests, prioritize receiving and caching the complete header data of the time-domain dynamic file, parse the three-dimensional index table built into the header and store it in the memory hash retrieval pool, and parse the three-dimensional index table in the header of the time-domain dynamic file. The time-domain dynamic file is a streaming file, and the three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers and byte offsets of video frame data in the file. The receiving module is used to play the corresponding video stream based on the first perspective identifier and receive perspective switching instructions in real time. The perspective switching instructions are used to indicate switching to the second perspective identifier. The first perspective refers to the perspective corresponding to the video stream that is currently being played and whose picture is continuously rendered and output. The query module is used to respond to the viewpoint switching command, obtain the target absolute timestamp of the currently playing video, and query the three-dimensional index table based on the target absolute timestamp and the second viewpoint identifier to determine the target byte offset of the target video frame corresponding to the second viewpoint identifier and the target absolute timestamp. The jump module is used to perform byte-level jumps from the temporal dynamic file based on the target byte offset, so as to read and decode the target video frame and render it based on the target absolute timestamp.

8. A multi-view video interactive file generation device, characterized in that, The multi-view video interactive file generation device, applied to the acquisition end, includes: The second acquisition module is used to acquire a temporal dynamic file synthesized from multiple video frame sequences. The data area of ​​the temporal dynamic file adopts a storage structure that is isolated by viewpoint. All video frame data corresponding to the same viewpoint identifier are stored continuously in the data area, and video frame data with different viewpoint identifiers are stored in blocks in the data area. The generation module is used to construct an index area in the file header of the time-domain dynamic file to generate a three-dimensional index table; The three-dimensional index table establishes a mapping relationship between absolute timestamps, viewpoint identifiers, and the byte offsets and frame length information of the corresponding video frames in the data area. When a viewpoint switching instruction is received, the three-dimensional index table allows the playback terminal to directly query and generate the target byte offset based on the current target absolute timestamp and the second viewpoint identifier to be switched, so as to support the playback terminal to perform byte-level jumps from the time-domain dynamic file and independently decode the target video frames. The second acquisition module is further configured to control multiple acquisition devices to continuously write acquired video frames into a circular buffer, and maintain a video frame sequence of a preset duration in the circular buffer; in response to a trigger signal, extract pre-buffered video frames before the arrival time of the trigger signal from the circular buffer, and continue to acquire post-recorded video frames after the arrival time of the trigger signal to form a complete video frame sequence corresponding to the time-domain dynamic file; acquire the timecode marked for each video frame sequence, perform frame-level alignment of the multiple video frame sequences based on the timecode, and trim frames with misaligned timecodes in each video frame sequence so that the length of each aligned video frame sequence is consistent and the timecodes of the corresponding frames match.

9. A multi-view video acquisition device, characterized in that, include: The gimbal assembly includes a support platform, a multi-axis drive motor, and an attitude encoder, used to support and drive at least one camera module for attitude adjustment; At least one of the camera modules is disposed on the support platform of the gimbal assembly for acquiring video frame sequences; Memory, used to store computer programs; A processor is configured to implement the multi-view video interactive file generation method as described in claim 6 when executing the computer program.

Citation Information

Patent Citations

  • Image verification method and device, computer equipment and storage medium

    CN114820575A

  • Dynamic three-dimensional reconstruction method, device, equipment, medium and system

    CN109840940A

  • Free view angle switching method and interactive free view angle playing system

    CN115174942A