Method for encapsulating, playing and managing surgical media resources and computer readable storage medium

Abstract

The encapsulation method of the surgical media resource comprises S10 to S60. S10: acquiring multiple video streams in a target surgical process. Each video stream comprises at least one video segment. S20: determining a video synchronization offset of each video segment relative to a zero point of a same master clock time axis. S30: encapsulating the video synchronization offset of each video segment and a video file address into a segment index file. S40: for each video stream, encapsulating all segment index files corresponding to the video stream and video stream index information of the video stream into a video track index file. S50: encapsulating all video track index files and global media description information into a structure file. S60: encapsulating the structure file and video files of each video segment into a surgical media resource file. The method can independently save multiple videos and facilitate synchronous playback. In addition, a surgical media resource playback method, a management system and a computer readable storage medium are also provided.

Description

Technical Field

[0001] This invention relates to the field of surgical media resource management technology, and in particular to a method for packaging, playing, managing, and computer-readable storage media for surgical media resources. Background Technology

[0002] Current surgical training videos are mostly created by combining multiple video streams into a single frame, which cannot meet the personalized viewing needs of different trainees, such as highlighting a particular video stream. Storing each video stream separately makes it difficult to achieve synchronization during playback, affecting the training effect. Therefore, there is an urgent need for a technical solution that can independently store multiple video streams and facilitate synchronized playback. Summary of the Invention

[0003] The purpose of this invention is to provide a method for encapsulating surgical media resources, which can independently save multiple video streams and facilitate synchronized playback.

[0004] Another objective of this invention is to provide a method for playing surgical media resources, which enables personalized synchronous playback of multiple video streams.

[0005] Another object of the present invention is to provide a management system for surgical media resources to implement the above-described encapsulation method and / or playback method.

[0006] Another object of the present invention is to provide a computer-readable storage medium for implementing the above-described encapsulation method and / or playback method.

[0007] This invention provides a method for encapsulating surgical media resources, comprising steps S10 to S60. S10: Acquire multiple video streams from the target surgical procedure. Each video stream includes at least one video segment that does not overlap in timing. S20: Determine the video synchronization offset of each video segment relative to the zero point of the same master clock time axis. S30: Encapsulate the video synchronization offset and video file address of each video segment into a segment index file. S40: For each video stream, encapsulate all segment index files corresponding to that video stream and the video stream index information of that video stream into a video track index file. S50: Encapsulate all video track index files and global media description information into a structure file. S60: Encapsulate the structure file and the video files of each video segment into a surgical media resource file.

[0008] This surgical media resource encapsulation method stores each video segment of each video stream independently, recording the video synchronization offset of each video segment based on the master clock timeline. Through a three-level indexing system—segment index file, video track index file, and structure file—it achieves independent storage and logical binding of multiple video streams. During playback, it can automatically align according to the video synchronization offset of each video segment to achieve synchronized playback. Furthermore, the generated surgical media resource file retains only valid video segments, reducing disk space usage.

[0009] In another illustrative embodiment of the method for encapsulating surgical media resources, S20 specifically includes: acquiring the start time of each video segment, and calculating the video synchronization offset of each video segment relative to the zero point of the master clock time axis based on the start time of each video segment. This allows for accurate determination of the video synchronization offset.

[0010] In another illustrative embodiment of the surgical media resource encapsulation method, S10 further includes: acquiring the audio stream in the target surgical procedure. S20 further includes: determining the audio synchronization offset of the audio stream relative to the zero point of the master clock time axis. S40 further includes: encapsulating the audio file address of the audio stream and the audio synchronization offset into an audio track index file. In S50, the audio track index file is also encapsulated into a structure file. In S60, the audio file of the audio stream is also encapsulated into a surgical media resource file. This facilitates further integration of audio resources and enables synchronized playback of audio and video.

[0011] In another illustrative embodiment of the surgical media resource encapsulation method, S10 further includes: acquiring the subtitle stream corresponding to the target surgical process. S20 further includes: determining the subtitle synchronization offset of the subtitle stream relative to the zero point of the master clock time axis. S40 further includes: encapsulating the subtitle file address of the subtitle stream and the subtitle synchronization offset into a subtitle track index file. In S50, the subtitle track index file is also encapsulated into a structure file. In S60, the subtitle file of the subtitle stream is also encapsulated into a surgical media resource file. This facilitates further integration of subtitle resources and enables synchronized playback of subtitles and video.

[0012] In another illustrative embodiment of the encapsulation method for surgical media resources, the video stream index information of the video stream includes any one or more of the following: unique video stream number, video stream name, video stream type, and resolution.

[0013] In another illustrative embodiment of the surgical media resource encapsulation method, the global media description information includes any one or more of the following: resource unique number, surgical unique number, resource name, sharer information, source information, start time, and end time.

[0014] This invention also provides a method for playing surgical media resources, which is used to play surgical media resource files generated by the above-mentioned surgical media resource encapsulation method. The playback method includes steps S80 and S90. S80: Parsing the surgical media resource file. S90: Generating a playback interface and presenting multiple playback windows on the interface. The layout of the playback windows can be adjusted by the user. The user-selected video stream is played in each playback window, and the video segments are time-axis aligned according to the video synchronization offset during playback. This method for playing surgical media resources, because each video segment of the video stream in the played surgical media resource file is stored independently, records the video synchronization offset for each video segment based on the master clock timeline, and achieves independent storage and logical binding of multiple video streams through a three-level index system of segment index file - video track index file - structure file, not only can it automatically align according to the video synchronization offset of each video segment to achieve synchronous playback, but it can also achieve independent playback of each video stream in a single window. Users can flexibly adjust the layout of the playback windows and the video streams to be played as needed, which is beneficial for meeting users' personalized viewing needs for surgical media resources. In addition, since the surgical media resource files being played only retain valid video segments, this not only helps reduce disk usage but also improves parsing speed.

[0015] In another illustrative embodiment of the method for playing surgical media resources, S90 further includes: presenting a playback content selection window on the playback interface. The playback content selection window has checkboxes corresponding to each playback window, allowing the user to select the video stream to be played in each playback window. In S90, playback is performed according to the video streams selected by the user in each playback window. This facilitates the user's selection of the video stream to be played in each playback window.

[0016] In another illustrative embodiment of the method for playing surgical media resources, S90 further includes: presenting a layout selection window on the playback interface, the layout selection window displaying multiple playback window layout modes for the user to choose from, and setting the layout of the playback window on the playback interface according to the playback window layout mode selected by the user. This facilitates the user in adjusting the layout of the playback window.

[0017] In another illustrative embodiment of the method for playing surgical media resources, S90 further includes: displaying a playback control bar on the playback interface, the playback control bar being used to simultaneously control the playback progress of all playback windows, and during any dragging or time jump operation, for a video segment containing the current playback time T, calculating the decoding time of the video segment in real time based on the video synchronization offset of the video segment, so that the decoding time is equal to the difference between T and the video synchronization offset, and starting decoding from that video segment, thereby maintaining synchronized playback of multiple window screens. This facilitates the user in adjusting the playback progress.

[0018] This invention also provides a management system for surgical media resources, comprising a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the aforementioned method for encapsulating surgical media resources. This allows for the independent storage of multiple video streams and facilitates synchronized playback, while also reducing disk space usage.

[0019] This invention also provides a management system for surgical media resources, comprising a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the aforementioned method for playing surgical media resources. This enables personalized synchronous playback of multiple video streams and also improves resolution speed.

[0020] This invention also provides a management system for surgical media resources, comprising a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the aforementioned method for encapsulating and playing surgical media resources. The encapsulation method can independently store multiple video streams and facilitates synchronized playback, while also reducing disk space usage. The playback method enables personalized synchronized playback of multiple video streams and further improves resolution speed.

[0021] The present invention also provides a computer-readable storage medium comprising a computer program that, when executed by a processor, implements the above-described method for encapsulating surgical media resources. This allows for the independent storage of multiple video streams and facilitates synchronized playback, while also reducing disk space usage.

[0022] The present invention also provides a computer-readable storage medium comprising a computer program that, when executed by a processor, implements the aforementioned method for playing surgical media resources. This enables personalized synchronous playback of multiple video streams and also improves resolution speed.

[0023] The present invention also provides a computer-readable storage medium comprising a computer program, which, when executed by a processor, implements the aforementioned method for encapsulating surgical media resources and the aforementioned method for playing surgical media resources. The encapsulation method can independently store multiple video streams and facilitates synchronized playback, while also reducing disk space usage. The playback method enables personalized synchronized playback of multiple video streams and further improves parsing speed. Attached Figure Description

[0024] The following figures are for illustrative purposes only and do not limit the scope of the invention.

[0025] Figure 1 A flowchart illustrating one implementation of a method for encapsulating surgical media resources.

[0026] Figure 2 for Figure 1The diagram shows the composition structure of the surgical media resource file generated by the encapsulation method.

[0027] Figure 3 for Figure 2 The diagram shows the structural composition of the file.

[0028] Figure 4 This is a flowchart illustrating one implementation method for playing surgical media resources.

[0029] Figure 5 for Figure 4 The diagram shows the timeline alignment after parsing the surgical media resource file in the playback method shown.

[0030] Figure 6 for Figure 4 A schematic diagram of the playback interface in the playback method shown.

[0031] Label Explanation

[0032] 10. Playback Interface

[0033] 11 Playback Window

[0034] 12 Playback content selection window

[0035] 13 Layout Selection Window

[0036] 14 Playback Control Bar

[0037] 15 subtitles

[0038] 16 Information Display Window Detailed Implementation

[0039] To provide a clearer understanding of the technical features, objectives, and effects of the invention, specific embodiments of the invention are now described with reference to the accompanying drawings. In the drawings, the same reference numerals indicate components with the same or similar structures but the same function.

[0040] In this document, “illustrative” means “serving as an example, illustration or description”, and any illustration or implementation described herein as “illustrative” should not be construed as a more preferred or advantageous technical solution.

[0041] In this patent application, nouns and pronouns relating to people are not limited to specific genders.

[0042] To keep the drawings simple, each drawing only schematically shows the parts related to the present invention, and they do not represent the actual structure of the product.

[0043] Figure 1 This is a flowchart illustrating one implementation of a method for encapsulating surgical media resources. For example... Figure 1As shown, the encapsulation method for surgical media resources includes S10 to S60.

[0044] S10: Acquire multiple video streams during the target surgical procedure. Each video stream includes at least one video segment that does not overlap temporally, and adjacent video segments within each video stream have, for example, time intervals. The video streams may originate from screen recordings of medical devices used during the surgery, or from panoramic surgical videos or close-up videos of surgical procedures captured by a camera, but are not limited to these. The medical devices include, but are not limited to: computed tomography (CT) scanners, magnetic resonance imaging (MRI) scanners, ultrasound diagnostic instruments, digital subtraction angiography (DSA) scanners, etc. In the illustrative embodiment, the number of video streams and the number of video segments in each video stream can be determined according to the actual situation, and are not limited here.

[0045] S20: Determine the video synchronization offset of each video segment relative to the zero point of the same master clock time axis. Specifically, this involves obtaining the start time of each video segment and calculating the video synchronization offset of each video segment relative to the zero point of the same master clock time axis based on its start time, thereby accurately obtaining the video synchronization offset. The start time of all video segments and the zero point of the master clock time axis are determined, for example, relative to the same time zone time axis.

[0046] S30: Encapsulate the video synchronization offset and video file address of each video segment into a segment index file.

[0047] S40: For each video stream, encapsulate all segment index files corresponding to that video stream and the video stream index information of that video stream into a video track index file. The video stream index information of a video stream may include, but is not limited to, any one or more of the following: unique video stream ID, video stream name, video stream type, and resolution.

[0048] S50: Encapsulate all video track index files and global media description information into a structured file. Global media description information may include, but is not limited to, any one or more of the following: resource unique ID, surgery unique ID, resource name, sharer information, source information, recording start time, and recording end time.

[0049] S60: Encapsulate the structure file and the video files of each video segment into a surgical media resource file. Figure 2 for Figure 1 The diagram shows the composition structure of the surgical media resource file generated by the encapsulation method. Figure 3 for Figure 2 The diagram shows the structural composition of the file.

[0050] The surgical media resource encapsulation method of this illustrative embodiment stores each video segment of each video stream independently, and records the video synchronization offset of each video segment based on the master clock timeline. Through a three-level indexing system—segment index file, video track index file, and structure file—multiple video streams are independently saved and logically bound. During playback, they can automatically align according to the video synchronization offset of each video segment to achieve synchronized playback. Furthermore, the generated surgical media resource file only retains valid video segments, which helps reduce disk space usage.

[0051] In the illustrative embodiment, S10 further includes: acquiring the audio stream during the target surgical procedure. S20 further includes: determining the audio synchronization offset of the audio stream relative to the zero point of the master clock time axis. S40 further includes: encapsulating the audio file address of the audio stream and the audio synchronization offset into an audio track index file. In S50, the audio track index file is also encapsulated into a structure file. In S60, the audio file of the audio stream is also encapsulated into a surgical media resource file. This facilitates further integration of audio resources and enables synchronized playback of audio and video. In this illustrative embodiment, only one audio stream is involved, but it is not limited to this. In other illustrative embodiments, the number of audio streams can be determined according to the actual situation and is not limited here.

[0052] In the illustrative embodiment, S10 further includes: acquiring the subtitle stream corresponding to the target surgical procedure. S20 further includes: determining the subtitle synchronization offset of the subtitle stream relative to the zero point of the master clock time axis. S40 further includes: encapsulating the subtitle file address of the subtitle stream and the subtitle synchronization offset into a subtitle track index file. In S50, the subtitle track index file is also encapsulated into a structure file. In S60, the subtitle file of the subtitle stream is also encapsulated into a surgical media resource file. This facilitates further integration of subtitle resources and enables synchronized playback of subtitles and video. In this illustrative embodiment, only one subtitle stream is involved, but it is not limited to this. In other illustrative embodiments, the number of subtitle streams can be determined according to the actual situation and is not limited here.

[0053] In an illustrative embodiment, in S30, for example, the duration of each video segment is encapsulated together into a segment index file. This facilitates rapid positioning and second-level jumps during playback, reduces disk reads, and improves playback efficiency.

[0054] In an illustrative implementation, in step S50, for example, all video track index files are first aggregated into a video track index set, and then this video track index set is encapsulated into a structure file. This helps reduce the number of file handles, centrally manage multiple indexes, and improve parsing efficiency and encapsulation speed.

[0055] Figure 4This is a flowchart illustrating one embodiment of a method for playing surgical media resources. This playback method is used to play surgical media resource files generated by the aforementioned surgical media resource encapsulation method. The playback method includes steps S80 and S90.

[0056] S80: Parse surgical media resource files. Figure 5 for Figure 4 The diagram shows the timeline alignment after parsing the surgical media resource file in the playback method shown.

[0057] S90: Generate playback interface 10 (see...) Figure 6 The system displays multiple playback windows 11 on the playback interface 10. The layout of the playback windows 11 can be flexibly adjusted by the user as needed. The layout of the playback windows 11 includes the number, position, and size of the playback windows 11. The user-selected video stream is played in each playback window 11, and the video segments are time-aligned according to the video synchronization offset during playback. Similarly, the audio stream and subtitle stream are played synchronously according to the audio synchronization offset and subtitle synchronization offset.

[0058] This illustrative implementation of the surgical media resource playback method, because each video segment of the video stream of the played surgical media resource file is stored independently, records the video synchronization offset of each video segment based on the master clock timeline, and achieves independent storage and logical binding of multiple video streams through a three-level index of segment index file - video track index file - structure file, can not only automatically align according to the video synchronization offset of each video segment to achieve synchronous playback, but also achieve independent playback of each video stream in a single window. Users can flexibly adjust the layout of the playback window 11 and the video stream to be played as needed, which is conducive to meeting users' personalized viewing needs for surgical media resources. In addition, since the played surgical media resource file only retains the valid video segments, it not only helps to reduce disk usage, but also helps to improve parsing speed.

[0059] like Figure 6 As shown in the illustrative embodiment, S90 further includes presenting a playback content selection window 12 on the playback interface 10. The playback content selection window 12 has selection boxes corresponding to each playback window 11, such as, but not limited to, drop-down selection boxes, for the user to select the video stream to be played in each playback window 11. It can be understood that the options in the selection boxes are the unique video stream number or video stream name corresponding to each video stream obtained from the parsing. In S90, the video streams to be played in each playback window 11 selected by the user in the playback content selection window 12 are played. This facilitates the user's selection of the video stream to be played in each playback window 11.

[0060] like Figure 6As shown, in the illustrative embodiment, S90 further includes: presenting a layout selection window 13 on the playback interface 10, the layout selection window 13 displaying multiple playback window layout modes for the user to select, and setting the layout of the playback window 11 on the playback interface 10 according to the playback window layout mode selected by the user. This facilitates the user in adjusting the layout of the playback window.

[0061] like Figure 6 As shown in the illustrative embodiment, S90 further includes: displaying a playback control bar 14 on the playback interface 10. The playback control bar 14 is used to simultaneously control the playback progress of all playback windows 11. During any dragging or time jump operation, for a video segment containing the current playback time T, the decoding time of the video segment is calculated in real time based on the video synchronization offset of the video segment, so that the decoding time is equal to the difference between T and the video synchronization offset, and decoding begins from that video segment, thereby maintaining synchronized playback of multiple window screens. This facilitates the user in adjusting the playback progress.

[0062] like Figure 6 As shown, in the illustrative embodiment, S90 further includes: displaying a subtitle bar 15 on the playback interface 10, the subtitle bar 15 being used for synchronized playback of the subtitle stream.

[0063] like Figure 6 As shown, in the illustrative embodiment, S90 further includes: presenting an information display window 16 on the playback interface 10. The information display window 16 is used to display relevant information about the surgical media resources, such as, but not limited to, the unique resource number, the unique surgical number, the resource name, the sharer information, the source information, the recording start time, and the recording end time.

[0064] This invention also provides a management system for surgical media resources. In one illustrative embodiment, the management system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the aforementioned method for encapsulating surgical media resources. This allows for the independent storage of multiple video streams and facilitates synchronized playback, while also reducing disk space usage.

[0065] In another illustrative embodiment, the management system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the aforementioned method for playing surgical media resources. This enables personalized synchronous playback of multiple video streams and also improves resolution speed.

[0066] In another illustrative embodiment, the management system includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the above-described method for encapsulating surgical media resources and the above-described method for playing surgical media resources. The encapsulation method can independently store multiple video streams and facilitates synchronized playback, while also reducing disk space usage. The playback method enables personalized synchronized playback of multiple video streams and further improves parsing speed.

[0067] The present invention also provides a computer-readable storage medium, in one illustrative embodiment, comprising a computer program that, when executed by a processor, implements the above-described method for encapsulating surgical media resources. This allows for the independent storage of multiple video streams and facilitates synchronized playback, while also reducing disk space usage.

[0068] In another illustrative embodiment, the computer-readable storage medium includes a computer program that, when executed by a processor, implements the aforementioned method for playing surgical media resources. This enables personalized synchronous playback of multiple video streams and also improves resolution speed.

[0069] In another illustrative embodiment, the computer-readable storage medium includes a computer program that, when executed by a processor, implements the above-described method for encapsulating surgical media resources and the above-described method for playing surgical media resources. This encapsulation method can independently store multiple video streams and facilitates synchronized playback, while also reducing disk space usage. This playback method enables personalized synchronized playback of multiple video streams and further improves parsing speed.

[0070] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0071] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent implementation schemes or modifications made without departing from the spirit of the present invention, such as combinations, divisions or repetitions of features, should be included within the scope of protection of the present invention.

Claims

1. A method for encapsulating surgical media resources, characterized in that, include: S10: Acquire multiple video streams during the target surgical process, each video stream including at least one video segment that does not overlap in time. S20: Determine the video synchronization offset of each video segment relative to the zero point of the same master clock time axis; S30: Encapsulate the video synchronization offset and video file address of each video segment into a segment index file; S40: For each video stream, encapsulate all the segment index files corresponding to the video stream and the video stream index information of the video stream into a video track index file; S50: Encapsulate all the video track index files and global media description information into a structure file; as well as S60: Encapsulate the structure file and the video files of each video segment into a surgical media resource file.

2. The method for encapsulating surgical media resources as described in claim 1, characterized in that, S20 specifically includes: obtaining the start time of each video segment, and calculating the video synchronization offset of each video segment relative to the zero point of the master clock time axis based on the start time of each video segment.

3. The method for encapsulating surgical media resources as described in claim 1, characterized in that, S10 further includes: acquiring the audio stream in the target surgical process; S20 further includes: determining the audio synchronization offset of the audio stream relative to the zero point of the master clock time axis; S40 further includes: encapsulating the audio file address of the audio stream and the audio synchronization offset into an audio track index file; S50 encapsulates the audio track index file into the structure file; S60 encapsulates the audio file of the audio stream into the surgical media resource file; and / or S10 further includes: acquiring the subtitle stream corresponding to the target surgical process; S20 further includes: determining the subtitle synchronization offset of the subtitle stream relative to the zero point of the master clock time axis; S40 further includes: encapsulating the subtitle file address of the subtitle stream and the subtitle synchronization offset into a subtitle track index file; S50 encapsulates the subtitle track index file into the structure file; and S60 encapsulates the subtitle file of the subtitle stream into the surgical media resource file.

4. The method for encapsulating surgical media resources as described in claim 1, characterized in that, The video stream index information includes any one or more of the following: unique video stream ID, video stream name, video stream type, and resolution; and / or The global media description information includes any one or more of the following: unique resource ID, unique surgery ID, resource name, sharer information, source information, start time, and end time.

5. A method for playing surgical media resources, used to play surgical media resource files generated by the encapsulation method of any one of claims 1 to 4, characterized in that, include: S80: Parse the surgical media resource file; as well as S90: Generate a playback interface (10) and present multiple playback windows (11) in the playback interface (10). The layout of the playback windows (11) can be adjusted by the user. Play the video stream selected by the user in each playback window (11). When playing, the video segment is time-axis aligned according to the video synchronization offset.

6. The method for playing surgical media resources as described in claim 5, characterized in that, The S90 further includes: presenting a playback content selection window (12) on the playback interface (10), wherein the playback content selection window (12) is provided with selection boxes corresponding to each of the playback windows (11) for the user to select the video stream to be played in each of the playback windows (11), and the S90 plays the video stream to be played in each of the playback windows (11) selected by the user in the playback content selection window (12).

7. The method for playing surgical media resources as described in claim 5, characterized in that, The S90 further includes: presenting a layout selection window (13) on the playback interface (10), the layout selection window (13) displaying a variety of playback window layout modes for the user to choose from, and setting the layout of the playback window (11) on the playback interface (10) according to the playback window layout mode selected by the user.

8. The method for playing surgical media resources as described in claim 5, characterized in that, The S90 further includes: presenting a playback control bar (14) on the playback interface (10), the playback control bar (14) being used to simultaneously control the playback progress of all the playback windows (11), and during any drag or time jump operation, for the video segment containing the current playback time T, calculating the decoding time of the video segment in real time based on the video synchronization offset of the video segment, so that the decoding time is equal to the difference between T and the video synchronization offset, and starting decoding from the video segment, thereby maintaining synchronized playback of the multi-window screen.

9. A management system for surgical media resources, characterized in that, The device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the encapsulation method of surgical media resources as described in any one of claims 1 to 4, and / or to implement the playback method of surgical media resources as described in any one of claims 5 to 8.

10. A computer-readable storage medium, characterized in that, It includes a computer program that, when executed by a processor, implements the encapsulation method for surgical media resources as described in any one of claims 1 to 4, and / or implements the playback method for surgical media resources as described in any one of claims 5 to 8.

Images