Method of displaying multimedia content related to location appearing in video

An electronic system detects landmarks in videos and links them with matching multimedia files, enabling seamless display and sharing of personal content related to video locations, addressing the lack of geographical information in existing platforms.

CN119094840APending Publication Date: 2024-12-06TOP VICTORY INVESTMENTS LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310860602.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-06-06
Filing Date
2023-07-13
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing video platforms lack the ability to provide geographical location information for landmarks within video frames, making it difficult for users to identify and share personal images or videos related to those landmarks during playback.

Method used

An electronic system that includes a display unit, processor, and storage unit, which generates a geographic location file by detecting landmarks in a video and associates matching multimedia files with the video, allowing for the display of thumbnails and bookmarks to switch between videos or images at relevant locations.

Benefits of technology

Enables users to identify and share personal memories related to landmarks in videos by automatically linking and displaying matching multimedia content, enhancing user experience and memory sharing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094840A_ABST
    Figure CN119094840A_ABST
Patent Text Reader

Abstract

A method of displaying multimedia content related to a location appearing in a video includes: generating, based on a main video, a geographic location file recording a timestamp and a corresponding location related to the main video; scanning multimedia files to find a file with metadata corresponding to the geographic position file from the multimedia files and taking the file as a file of a matching file; associating the matching file with the main video by generating an association file of the main video; playing the main video; displaying a thumbnail of the matching file; and displaying the matching file when the thumbnail is selected, so that the user can backtrack time and regain beautiful memory.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for displaying multimedia content, and more particularly to a method for displaying multimedia content related to a position appearing in a video when playing the video. Background Art

[0002] When watching a video, when a landmark appears in the video, the user may want to know whether he / she has visited the landmark before and whether there are any personal images or videos related to the landmark. Although the geographical location information of different scenes in the video is usually not stored in the video file, some online video platforms reveal the location (e.g., which city) where the video was recorded. However, information about the geographical location of various scenes or landmarks appearing in individual frames of the video is still lacking. Summary of the invention

[0003] Therefore, the object of the present invention is to provide a method for displaying multimedia content related to locations appearing in a video, so that personal images or videos containing landmarks that have been visited can be shown to relatives, friends or guests on the screen when a movie currently being watched shows the same landmarks.

[0004] Therefore, a method for displaying multimedia content related to a location appearing in a video according to the present invention is implemented by an electronic system. The electronic system includes a display unit, a processor and a storage unit. The storage unit stores a main video to be played on the display unit and a plurality of multimedia files. The method for displaying multimedia content related to a location appearing in a video comprises: the processor generates a geographic location file based on the main video, which records a plurality of time stamps and a plurality of locations related to the main video and corresponding to the plurality of time stamps respectively; the processor scans the multimedia files to find a file having metadata corresponding to one of the plurality of locations recorded in the geographic location file from the multimedia files, and regards the file as a matching file; the processor associates the matching file with the main video by generating an association file for the main video, the association file records a file path indicating the matching file and one of the plurality of time stamps corresponding to the geographic location file;

[0005] A bookmark entry with a timestamp corresponding to the location in the location file; and the processor plays the main video on the display unit, displays a thumbnail of the matching file on the display unit when the main video is at the timestamp, and displays the matching file on the display unit when the thumbnail is selected.

[0006] In some embodiments, when scanning the plurality of multimedia files to find the file, the metadata of the matching file records a location that matches the location recorded in the geolocation file.

[0007] In some embodiments, the location recorded in the metadata of the matching file indicates the location where the matching file is generated.

[0008] In some embodiments, the location recorded in the metadata of the matching file indicates a location of a landmark detected in the matching file.

[0009] In some embodiments, the distance between the location recorded in the metadata of the matching file and the location recorded in the geographic location file is within a specific distance.

[0010] In some embodiments, the main video includes different video parts. The metadata of the main video stores multiple pieces of geographic location information, each of which corresponds to a different video part and indicates a timestamp of the video part in the main video and a location when the video part was recorded; and when a geographic location file having multiple timestamps and multiple locations recorded is generated, the geographic location file records the multiple timestamps and the multiple locations indicated by the multiple pieces of geographic location information.

[0011] In some embodiments, before generating the geographic location file, the method further includes: the processor determines a plurality of landmarks appearing in the main video by performing a landmark detection process on the main video to determine the plurality of locations related to the main video.

[0012] In some embodiments, the main video includes a video frame sequence corresponding to different timestamps and consisting of multiple I frames, multiple P frames and multiple B frames; and performing a landmark detection process includes: extracting the I frame from the main video and using at least a portion of the extracted I frame as a key frame of the main video; and for each of the key frames, detecting the landmark in the key frame to obtain a detection result indicating the longitude and latitude coordinates representing the location of the landmark.

[0013] In some embodiments, when at least a portion of the captured I frame is used as a key frame, the processor selects a plurality of scene change frames from the I frame using a scene filter, and uses the selected scene change frames as the key frames.

[0014] BCPI-230250 Page 3 / 12

[0015] In some embodiments, when a geographic location file having a plurality of time stamps and a plurality of locations recorded therein is generated, the time stamp of the key frame and the location of the landmark detected in the key frame are recorded in the geographic location file.

[0016] In some embodiments, when the matching file is associated with the primary video, the file path indicated by the bookmark entry of the associated file record represents a uniform resource identifier (URI) of the matching file.

[0017] In some embodiments, when the matching file is associated with the main video, the associated file is a pairing file of the main video, is stored in the same directory as the main video, and has the same file name as the main video but a different file extension from the main video.

[0018] In some embodiments, the associated file is generated as a subtitle file of the main video, and the bookmark entry is recorded in the associated file in the form of a text string.

[0019] In some embodiments, the method further comprises: based on the associated file, displaying a bookmark indicator on the video progress bar of the main video, wherein the bookmark indicator corresponds to the bookmark entry recorded in the associated file and is located at a position of the video progress bar corresponding to the timestamp indicated by the bookmark entry.

[0020] In some embodiments, before displaying the thumbnail of the matching file, the method further includes: the processor accessing the matching file according to the file path indicated by the bookmark entry recorded in the associated file, and generating the thumbnail of the matching file.

[0021] The beneficial effects of the present invention are: even if the metadata of the main video does not initially store any geographic location information related to the location where the main video was recorded, the location of the landmark appearing in the main video can still be determined through the landmark detection process, and this location can be recorded in the metadata of the main video; personal images and personal videos with a common location can be combined together through the electronic system, thus saving the work of manually sorting the multimedia files according to location; when the main video shows a landmark, the user can switch from playing the main video to displaying or playing the matching file showing the same landmark, so that the user can go back in time and relive good memories; and the main video with the bookmark indicator embedded in the video progress bar can be shared with friends or relatives so that they can share memories. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features and effects of the present invention will be apparent from the following embodiments with reference to the accompanying drawings.

[0023] Rendering, where:

[0024] Figure 1 is a block diagram exemplarily illustrating an electronic system according to an embodiment of the present invention;

[0025] Figure 2 is a flow chart, exemplarily illustrating a method for displaying multimedia content associated with a position appearing in a video according to an embodiment of the present invention;

[0026] Figure 3 is a flow chart exemplarily illustrating the steps of the landmark detection process;

[0027] Figure 4 is a schematic diagram exemplarily illustrating an example of a key frame of a main video;

[0028] Figure 5 is a schematic diagram illustrating an example of four frames of personal images as matching files;

[0029] Figure 6 Four example thumbnails of matching files are shown together with the main video. DETAILED DESCRIPTION

[0030] Before the present invention is described in more detail, it should be noted that, where considered appropriate, reference numerals have been repeated among the drawings to indicate corresponding or analogous components, which may optionally have similar characteristics.

[0031] See also Figure 1, an electronic system 1 suitable for displaying multimedia content related to a position appearing in a video when playing a video according to an embodiment of the present invention is exemplarily described. The electronic system 1 includes a display unit 11, a processor 12, a storage unit 13 and an input unit 14. The display unit 11, the storage unit 13 and the input unit 14 are electrically connected to the processor 12. In some embodiments, the display unit 11 is implemented by a liquid crystal display (LCD), a light emitting diode (LED) display, an electronic visual display, a screen, a television, a computer monitor, a mobile display, a digital signage, a TV wall, etc.; the processor 12 is implemented by a central processing unit (CPU), a microprocessor, a mobile processor, a micro control unit (MCU), a digital signal processor (DSP), a field programmable gate array (FPGA), or any circuit that is planned or programmable in software and / or hardware to achieve the functions described in the present invention; the storage unit 13 is implemented by a flash memory, a hard disk drive (HDD), a solid state drive (SDD), an electrically erasable programmable read-only memory (EEPROM), or any other non-volatile memory device; and the input unit 14 is implemented by a physical key group or a touch panel that can be combined with the display unit 11 to form a touch screen. In some embodiments, the electronic device 1 is implemented by a personal computer, a laptop, a smart phone, a media server for home scenarios, a data server, a cloud server, or a combination thereof. In some embodiments, the storage unit 13 includes a BCPI-230250 installed in a home media server connected to the processor 12 via a local area network (LAN) cable or a wireless LAN.

[0032] A storage medium. In some embodiments, the storage unit 13 includes a personal storage space in a cloud server that is accessible to the processor 12 via the Internet. However, the implementation of the electronic system 1 and the components included therein are not limited to those disclosed herein and may be changed in other embodiments.

[0033] The storage unit 13 is configured to store a video to be played on the electronic system 1 (hereinafter referred to as a "main video"). In some embodiments, the main video is exemplified by a movie and includes a sequence of video frames corresponding to different time stamps. In some embodiments, the video frame is composed of an intra-coded picture (I frame) of a complete picture, a predicted picture (P frame) each storing the difference between the current frame and the previous frame, and a bidirectional predicted picture (B frame) each storing the difference between the current frame and the previous frame and the difference between the current frame and the next frame. The storage unit 13 is further configured to store multimedia files including personal images and personal videos of the user. In some embodiments, the personal images and personal videos may be personal photos or videos recorded when the user and / or his family members travel, visit landmarks, sightseeing spots, tourist attractions, etc. In some embodiments, the personal images or personal videos may only contain images of scenery or landmarks, and may not contain images of any person.

[0034] The processor 12 is configured to detect landmarks in the main video, detect landmarks in the multimedia files, scan the multimedia files to determine (if any) a matching file (which is one of the personal image and the personal video) containing a landmark that matches the landmark detected in the main video, and associate the matching file with the main video. The processor 12 is further configured to control the display unit 11 to display the matching file when the landmark is displayed in the main video during playback of the main video.

[0035] See also Figure 2 , exemplarily illustrating a method of displaying multimedia content related to a position appearing in a video when playing a video according to an embodiment of the present invention. The method is suitable for Figure 1 The electronic system 1 shown in the figure performs the method described herein, which includes the following steps S21-S24.

[0036] In step S21, the processor 12 generates a geographic location file related to the main video based on the main video to be played by the display unit 11, and stores the geographic location file in the storage unit 13. In some embodiments, when the main video is recorded using a smart phone or camera provided with a global navigation satellite system (such as a global positioning system, GPS), the metadata of the main video will store a geographic location information related to the location where the main video was recorded. It is worth noting that when the main video was recorded, BCPI-230250 Page 6 / 12

[0037] The landmark of the location may be captured as the background of the main video. The metadata of the main video may be stored in accordance with the Exchangeable Image File Format (Exif) standard. In addition, when the main video is recorded at different locations so that the main video includes different video parts corresponding to the different locations, the metadata may store multiple geographic location information, each of which corresponds to a different video part and indicates the timestamp of the video part in the main video and the location when the video part was recorded. In this case, the processor 12 extracts multiple geographic location information from the metadata of the main video by using a software program, such as ExifTool, and generates the geographic location file that records the timestamp and location indicated by the multiple geographic location information. In some embodiments, the geographic location file may be generated in GPS Exchange Format (GPX).

[0038] Alternatively, in some embodiments, when the main video is recorded using a video recorder or camera that is not provided with a global satellite navigation system, the metadata of the main video will not store any geographic location information related to the location where the main video was recorded. In some embodiments, the social media platform that maintains the main video can remove the geographic location information related to the main video originally stored in the metadata, so that the information related to the location where the main video was recorded will not be available. In this case, the processor 12 determines the landmarks that appear in the main video by performing a landmark detection process on the main video, and generates the geographic location file based on the results of the landmark detection process. Specifically, the landmark detection process includes the following steps. Figure 3 Steps S31-S32 are shown.

[0039] In step S31, the processor 12 extracts the I frame from the main video and uses the extracted I frame as the key frame of the main video. In some embodiments, the processor 12 executes a video processing software program, such as FFmpeg tool, to obtain the key frame. In some embodiments, if the I frame shows too much information redundancy, the processor 12 can further reduce the number of the extracted I frames by selecting scene change frames from the I frames using a scene filter, and use the selected scene change frames as the key frames. It is worth noting that the key frames correspond to their respective timestamps.

[0040] In step S32, for each key frame, the processor 12 detects a landmark in the key frame (if any) to obtain a detection result indicating the name of the landmark and the location of the landmark in a set of longitude and latitude coordinates. In some embodiments, the processor 12 detects the landmark in the key frame by using an image feature detection tool provided by a cloud computing service such as Google Cloud Vision application programming interface (API). Figure 4 , an example of a key frame of the main video presents a movie scene with the Taj Mahal as a landmark in the background. In this case, the processor 12 detects a landmark such as the Taj Mahal in the key frame and obtains the position of the Taj Mahal.

[0041] In this way, the processor 12 generates the geolocation file that records the timestamp of the key frame containing the detected landmark and the location of the detected landmark based on the detection result. In the geolocation file, the timestamps correspond to the locations respectively. Similarly, the geolocation file can be generated in GPX format. In some cases, the same landmark can appear in different key frames, and therefore multiple timestamps can correspond to the same location.

[0042] See also Figure 2, in step S22, the processor 12 scans all multimedia files stored in the storage unit 13 to find any file having metadata corresponding to the geographic location file associated with the main video from the multimedia files, and regards each file found as a matching file. Specifically, the metadata in a matching file records a location (hereinafter referred to as a "matching location") that matches a location recorded in the geographic location file. It is worth noting that the location recorded by the metadata indicates the location at which the matching file location was generated. Specifically, a matching file can be a personal image or a personal video, and the location recorded by the metadata can indicate the location at which the personal image (or the personal video) was captured (or recorded). The number of matching files can be single or multiple; that is, in some cases, multiple matching files can be determined in this step and each matching file corresponds to a location recorded in the geographic location file (that is, a corresponding matching location). Furthermore, determining two locations as matching does not necessarily mean that the latitude and longitude coordinate combinations of the two locations are exactly the same, and the processor 12 may determine a multimedia file as a matching file when it is determined that the metadata of the multimedia file records a location within a specific distance (e.g., 100 meters) from a location recorded in the geographic location file. In some cases, a single matching file may correspond to multiple matching locations, which are actually the same but correspond to different timestamps in the geographic location file.

[0043] In some cases, the metadata of some of the multimedia files may not record the location where the personal image or the personal video was captured or recorded. Therefore, before scanning all the multimedia files BCPI-230250 Page 8 / 12, for those multimedia files that do not have any geographic location information, the processor 12 performs the above-mentioned landmark detection process and records the location detected in the metadata of the multimedia files. It is worth noting that since a personal image does not include multiple (image) frames, step S31 related to key frame capture is omitted, and the processor 12 directly detects the landmark in the personal image to obtain the location of the landmark according to the description in step S32. In this case, the location recorded by the metadata of a personal image indicates the location of a landmark detected in the personal image, and the location recorded by the metadata of a personal video indicates the location of the landmark detected in the key frame of the personal video.

[0044] See also Figure 5, exemplarily illustrating four frames of personal images with Taj Mahal as a landmark in the background, and the four frames of personal images are determined by the processor 12 as matching files with metadata corresponding to the geographic location file related to the main video. That is, the metadata of the four frames of personal images records the location of Taj Mahal matching a location recorded in the geographic location file.

[0045] In step S23, the processor 12 associates the matching files with the main video by generating an association file for the main video. For each matching file, the association file records a bookmark entry indicating a file path to the matching file and a timestamp corresponding to a corresponding matching position in the geolocation file (e.g., a time point at which the corresponding matching position appears in the main video). It is noteworthy that the timestamp of a bookmark entry in the association file corresponds to a key frame in the main video related to the corresponding matching position, for example, a key frame containing the same landmark as the matching file or captured at the same position as the matching file. In the case where multiple matching files are found, the association file records multiple bookmark entries, each of which indicates a file path to the corresponding matching file and a timestamp corresponding to a corresponding matching position. In the case where the matching file corresponds to multiple timestamps (that is, the position matching the matching file appears multiple times in the geolocation file), multiple bookmark entries will be recorded, each corresponding to a different timestamp.

[0046] In some embodiments, the file path indicates the storage location of the matching file in a directory structure when the matching file is stored locally, such as in the same computer as the main video, such as in the same computer as the main video. In some embodiments, when the matching file is stored on a remote server and can be accessed via the Internet, the file path indicates the uniform resource identifier (URI) of the matching file. In some embodiments, the associated file is a paired file of the main video, which is stored in the same directory as the main video and has the same file name as the main video but has a different extension from the main video. For example, the associated file may be generated as a subtitle file of the main video and has a file extension of ".srt", and the bookmark entry is recorded in the associated file in the form of a text string. It is worth noting that the associated file is not limited to being stored locally with the main video. For example, in other embodiments, the associated file may be stored on a remote server, and the URI of the associated file may be recorded in the metadata of the main video. In this way, the associated file can be accessed through the Internet based on the URI recorded in the metadata recorded in the main video.

[0047] Therefore, after the main video generates the associated file, when the display unit 11 is playing the main video, the matching file can be displayed separately when the main video is played to the point where the matching position or landmark appears in the main video. Specifically, in step S24, the processor 12, in response to a user instruction for playing the main video received from the input unit 14, plays the main video on the display unit 11, and displays a thumbnail of the corresponding matching file on the display unit 11 whenever the playing time of the main video is the same as the time indicated by a timestamp of a bookmark entry recorded in the associated file while the main video is being played.

[0048] Specifically, in some embodiments, based on the associated file, the processor 12 displays a bookmark indicator corresponding to the bookmark entry on a video progress bar of the main video for each bookmark entry recorded in the associated file. The bookmark indicator is located at a position of the video progress bar corresponding to the timestamp indicated by the bookmark entry. In other words, the main video can be "embedded" with a bookmark indicator on the video progress bar. In some embodiments, each bookmark indicator can be implemented by a symbol (such as a triangle), which can be clicked or selected by user operation. In addition, for each bookmark entry recorded in the associated file, the processor 12 accesses the respective matching files according to the file path indicated by the bookmark entry, and generates a thumbnail of the matching file. It is worth noting that when multiple bookmark entries have the same timestamp, the corresponding multiple bookmark indicators can be merged into one bookmark entry and displayed on the video progress bar.

[0049] When a bookmark indicator is selected, the main video jumps to the time corresponding to the timestamp of the bookmark indicator BCPI-230250 Page 10 / 12 and pauses at that time, and the processor 12 displays the thumbnail of the corresponding matching file on the display unit 11. It is worth noting that at this time, the main video will present a landmark or scene at the corresponding matching position, and the corresponding matching position is the same landmark or scene as the position represented in the thumbnail of the matching file. Figure 6 , when the middle bookmark indicator is selected, the main video jumps and pauses to display a scene of the Taj Mahal in the background, and four thumbnails of personal images with the same landmark, the Taj Mahal, in the background are displayed below the main video (i.e., four matching files). It is worth noting that the user can navigate the bookmark indicator by selecting the arrow symbol to view other matching files corresponding to different locations in the main video.

[0050] In addition, when a thumbnail is selected by a user operation, the processor 12 displays the corresponding matching file on the display unit 11, that is, the personal image or the personal video having a position matching the position appearing in the main video will be displayed or played. In some embodiments, the matching file being displayed can be superimposed on the main video in a picture-in-picture (PIP) manner, or can be presented side by side with the main video in a split-screen manner. In response to receiving a user instruction to close the matching file, the processor 12 stops displaying or playing the matching file and continues to play the main video. In other words, the user can switch from the main video to displaying or showing the matching file by selecting a thumbnail, and switch back to playing the main video after viewing the matching file.

[0051] In some embodiments, when the main video is played to the time indicated by the timestamp of the bookmark entry recorded in the associated file (for example, when the current progress of the main video reaches a bookmark indicator on the video progress bar), the main video continues to play while a thumbnail of the corresponding matching file is displayed next to the main video for user selection. In this way, the user's viewing experience will not be interrupted, and the user is provided with the option of switching to viewing the matching file.

[0052] In some embodiments, when the processor 12 determines that multiple matching files are found so that the matching files record multiple bookmark entries, if the multiple bookmark entries indicate two or more different timestamps, the bookmark indicator may be displayed on the video progress bar, and if two or more of the bookmark entries indicate the same timestamp, multiple thumbnails (i.e., Figure 6 example shown).

[0053] It is worth noting that by transmitting the main video, the associated file and the matching file to the electronic device of other users, such as friends or relatives, the video embedded in the BCPI-230250 Page 11 / 12

[0054] The main video of the bookmark indicator on the video progress bar is shared with the other users. In some embodiments, when the URI of the associated file is recorded in the metadata of the main video and the URI of each matching file is recorded in the matching file, only the main video or the URI of the main video is transmitted to the electronic device of the other user, and the associated file and the matching file can be accessed through a network (e.g., the Internet) according to their respective URIs.

[0055] In some embodiments, the associated file can be edited by using a subtitle editing software application, such as SubRip, to manually add a bookmark entry to the associated file or delete a bookmark entry from the associated file. In this way, any multimedia file can be used as a matching file and associated with the main video. For example, the application of the method can be extended to educational videos (e.g., videos related to geography or history courses), in which case the bookmark entries can be edited to indicate other videos, images, or even documents in portable document format (PDF) as supplementary teaching materials, making online teaching more attractive and engaging for students.

[0056] In some embodiments, the idea of ​​the method can be further extended to content aggregation based on landmark detection. The processor 12 scans all multimedia files to find associated files with the same or similar locations as recorded in the metadata, and generates a playlist indicating the file paths of these associated files. In this way, the user can play all video clips and image slides with a matching landmark. It is worth noting that by changing the criteria for location matching, a higher level of content aggregation can be achieved. For example, when playing multimedia files related to the Eiffel Tower, it can be recommended to play multimedia files related to the Louvre because these two landmarks are located in the same city of Paris.

[0057] In short, by detecting landmarks in the main video and the multimedia file, a location associated with a scene appearing in the main video or the multimedia file can be determined. By comparing the locations of the main video and the multimedia file in order to find a matching file in the multimedia file, an associated file can be generated for the main video, in which a bookmark entry indicating a file path to the matching file and a timestamp corresponding to the corresponding matching location is recorded. Therefore, during the playback of the main video, when the matching location appears in the main video, a thumbnail of the matching file is shown to the user for selection. Further, when the thumbnail of the matching file is selected, the playback of the main video can be switched to display or play the matching file.

[0058] BCPI-230250 Page 12 / 12

[0059] In this way, the electronic system 1 and the method for displaying multimedia content related to a position appearing in a video according to the embodiments of the present invention have at least the following advantages.

[0060] Even if the metadata of the main video does not initially store any geographic location information related to the location at which the main video was recorded, the location of a landmark appearing in the main video can still be determined through the landmark detection process, and this location can be recorded in the metadata of the main video.

[0061] Personal images and personal videos with a common location can be grouped together through the electronic system 1, thus saving the work of manually sorting the multimedia files according to location.

[0062] When the main video shows a landmark, the user can switch from playing the main video to displaying or playing the matching file showing the same landmark, allowing the user to go back in time and relive good memories.

[0063] The main video with the bookmark indicator embedded on the video progress bar can be shared with friends or relatives so that they can share memories.

[0064] However, what is described above is only an embodiment of the present invention and should not be used to limit the scope of the present invention. All simple equivalent changes and modifications made according to the claims and description of the present invention are still within the scope of the present invention.

Claims

1. A method for displaying multimedia content associated with a position appearing in a video, implemented by an electronic system, the electronic system comprising a display unit, a processor and a storage unit, the storage unit storing a main video to be played on the display unit and a plurality of multimedia files, characterized in that: The method of displaying multimedia content associated with a location appearing in a video comprises: The processor generates a geographic location file based on the main video, which records a plurality of time stamps and a plurality of locations related to the main video and corresponding to the plurality of time stamps respectively; The processor scans the multimedia files to find a file having metadata corresponding to one of the plurality of locations recorded in the geographic location file from among the multimedia files, and regards the file as a matching file; The processor associates the matching file with the main video by generating an association file for the main video, the association file recording a bookmark entry indicating a file path to the matching file and one of the multiple timestamps corresponding to the location in the geolocation file; and The processor plays the main video on the display unit, displays a thumbnail of the matching file on the display unit when the main video is at the timestamp, and displays the matching file on the display unit when the thumbnail is selected.

2. The method for displaying multimedia content associated with a location appearing in a video according to claim 1, characterized in that When scanning the plurality of multimedia files to find the file, the metadata of the matching file records a location that matches the location recorded in the geographic location file.

3. The method for displaying multimedia content associated with a location appearing in a video according to claim 2, characterized in that: The location recorded in the metadata of the matching file indicates the location where the matching file is generated.

4. The method for displaying multimedia content associated with a location appearing in a video according to claim 2, characterized in that: The position recorded in the metadata of the matching file indicates a position of a landmark detected in the matching file.

5. The method for displaying multimedia content associated with a location appearing in a video according to claim 2, characterized in that: The distance between the location recorded in the metadata of the matching file and the location recorded in the geographic location file is within a specific distance.

6. The method for displaying multimedia content associated with a location appearing in a video according to claim 1, characterized in that: The main video includes different video parts, and the metadata of the main video stores multiple pieces of geographic location information, each of which corresponds to a respective different video part and indicates a timestamp of the video part in the main video and a location when the video part was recorded; and When a geographic location file recording a plurality of time stamps and a plurality of locations is generated, the geographic location file records the plurality of time stamps and the plurality of locations indicated by the plurality of pieces of geographic location information.

7. The method for displaying multimedia content associated with a location appearing in a video according to claim 1, characterized in that: Before generating the geolocation file, it also includes: The processor determines the plurality of locations associated with the main video by performing a landmark detection process on the main video to determine a plurality of landmarks appearing in the main video.

8. The method for displaying multimedia content associated with a location appearing in a video according to claim 7, characterized in that The main video includes a video frame sequence corresponding to different time stamps and consisting of a plurality of I frames, a plurality of P frames and a plurality of B frames; and The process of performing landmark detection includes: Extracting the I frame from the main video and using at least a portion of the extracted I frame as a key frame of the main video; and For each of the key frames, a landmark in the key frame is detected to obtain a detection result indicating a longitude and latitude coordinate representing a position of the landmark.

9. The method for displaying multimedia content associated with a location appearing in a video according to claim 8, characterized in that When at least a portion of the captured I frame is used as a key frame, the processor selects a plurality of scene change frames from the I frame using a scene filter, and uses the selected scene change frames as the key frames.

10. The method for displaying multimedia content associated with a location appearing in a video according to claim 8, characterized in that When a geographic location file having a plurality of time stamps and a plurality of locations recorded therein is generated, the time stamp of the key frame and the location of the landmark detected in the key frame are recorded in the geographic location file.

11. The method of displaying multimedia content associated with a location appearing in a video according to claim 1, characterized in that: When the matching file is associated with the main video, the file path indicated by the bookmark entry of the associated file record represents a uniform resource identifier URI of the matching file.

12. The method of displaying multimedia content associated with a location appearing in a video according to claim 1, characterized in that: When the matching file is associated with the main video, the associated file is a pairing file of the main video, which is stored in the same directory as the main video and has the same file name as the main video but a different file extension from the main video.

13. The method of displaying multimedia content associated with a location appearing in a video according to claim 12, characterized in that The associated file is generated as a subtitle file of the main video, and the bookmark entry is recorded in the associated file in the form of a text string.

14. The method of displaying multimedia content associated with a location appearing in a video according to claim 1, characterized in that: Also includes: Based on the associated file, a bookmark indicator is displayed on the video progress bar of the main video, the bookmark indicator corresponds to the bookmark entry recorded in the associated file and is located at a position of the video progress bar corresponding to the timestamp indicated by the bookmark entry.

15. The method of displaying multimedia content associated with a location appearing in a video according to claim 1, characterized in that: Before displaying the thumbnails of the matching files, it also includes: The processor accesses the matching file according to the file path indicated by the bookmark entry recorded in the associated file, and generates the thumbnail of the matching file.