Video playback method, device, electronic device and storage medium

Abstract

The video playback method, device, electronic device and storage medium provided by the embodiments of the present disclosure can reduce the preparation cycle and cost when presenting information using augmented reality display technology by obtaining a real-scene captured image, detecting and determining the display position of a target image in the real-scene captured image, obtaining a target video associated with the target image, and playing the target video at the display position in the real-scene captured image. On the other hand, it also provides users with more presentation channels for presenting video information, so that users can have a better interactive experience and visual experience.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the computer field, and in particular to a video playback method, device, electronic device, and storage medium. Background Art

[0002] Augmented Reality (AR) technology is a technology that cleverly integrates virtual information with the real world.

[0003] Using augmented reality to present information has become a possible way of presenting information. In the prior art, a 3D modeling image of a virtual character or virtual scene to be presented is pre-built so that when a user uses a terminal to shoot a real scene, he or she can simultaneously obtain a real scene shot image including the 3D modeling image.

[0004] However, this information presentation method requires the construction of a large number of three-dimensional modeling images in advance, which has a high construction cost and a long construction cycle, and is not conducive to the rapid presentation of information. Summary of the invention

[0005] In view of the above problems, the embodiments of the present disclosure provide a video playback method, device, electronic device and storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides a video playback method, including:

[0007] Obtaining real scene shooting images;

[0008] Detecting a target image in the real scene captured image;

[0009] Determining a display position of the target image in the real scene captured image;

[0010] A target video associated with the target image is acquired, and the target video is played at a display position of the target image in the real-scene captured image.

[0011] In a second aspect, an embodiment of the present disclosure provides a video playback device, comprising:

[0012] A processing module, used for obtaining a real-scene captured image, detecting a target image in the real-scene captured image, and determining a display position of the target image in the real-scene captured image;

[0013] The playing module is used to obtain a target video associated with the target image and play the target video at the display position of the target image in the real-scene shooting image.

[0014] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0015] The memory stores computer-executable instructions;

[0016] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the first aspect as well as various possible video playback methods related to the first aspect.

[0017] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video playback method described in the first aspect and various possible designs of the first aspect is implemented.

[0018] The video playback method, device, electronic device and storage medium provided by the embodiments of the present disclosure obtain a real-scene captured image, detect a target image in the real-scene captured image; determine the display position of the target image in the real-scene captured image; obtain a target video associated with the target image, and play the target video at the display position of the target image in the real-scene captured image; the video playback method provided by this embodiment can reduce the presentation cost and preparation cycle when presenting information using augmented reality display technology, and on the other hand, it also provides users with more presentation channels for presenting video information, so that users can get better interactive experience and visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 A schematic diagram of a network architecture on which the present disclosure is based;

[0021] Figure 2 A flowchart of a video playback method provided by an embodiment of the present disclosure;

[0022] Figure 3 A schematic diagram of a first interface of a video playback method provided by an embodiment of the present disclosure;

[0023] Figure 4 A signaling interaction diagram of a video playback method provided in an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of a second interface of a video playback method provided by an embodiment of the present disclosure;

[0025] Figure 6 A schematic diagram of a third interface of a video playback method provided by an embodiment of the present disclosure;

[0026] Figure 7 A flowchart of another video playback method provided by the present disclosure;

[0027] Figure 8 A schematic diagram of a fourth interface of a video playback method provided by an embodiment of the present disclosure;

[0028] Fig. 9 A structural block diagram of a video playback device provided in an embodiment of the present disclosure;

[0029] Fig.10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0031] Augmented Reality (AR) technology is a technology that cleverly integrates virtual information with the real world.

[0032] When displaying augmented reality, the terminal will first shoot the real scene of the real scene to obtain the current real scene shooting image. Then, the real scene shooting image is processed using augmented reality technology to superimpose the preset virtual information on the real scene shooting image, and the superimposed image is presented to the user.

[0033] Generally speaking, the virtual information superimposed on the real-life image is usually a pre-established 3D modeling image of a virtual character or virtual scene. However, the construction process of the 3D modeling image is relatively complicated, and the technical and labor costs required for construction are high. This will make the period required for presenting each piece of information longer and more costly when using augmented reality display technology to present information.

[0034] In response to such problems, the inventors have creatively discovered through research that the information presentation method is not limited to three-dimensional modeling images, but can also be replaced by a method with lower technical and labor costs. According to the method of the embodiment of the present disclosure, some existing video data is placed in a real-life shooting image and displayed through an augmented reality display. In this way, the preparation cycle and cost of presenting information using augmented reality display technology can be reduced. On the other hand, it also provides users with more presentation channels to continue video information, so that users can get a better interactive experience and visual experience.

[0035] refer to Figure 1 , Figure 1 A schematic diagram of a network architecture based on the present disclosure is shown in FIG. Figure 1 The network architecture shown may specifically include a terminal 1 and a server 2 .

[0036] Among them, terminal 1 can specifically be a user's mobile phone, smart home device, tablet computer, wearable device, etc., which can be used for capturing real scenes and displaying the captured real scenes. A video playback device can be integrated or installed in the terminal 1. The video playback device is hardware or software for executing the video playback method disclosed in the present invention. The video playback device can provide a display page of augmented reality display for terminal 1, and terminal 1 uses its screen or display component to display the display page of augmented reality display provided by the video playback device to the user.

[0037] The server 2 may specifically be a server or a server cluster arranged in the cloud, and the server or the server cluster may store video data and image data related to the video playback method provided by the present disclosure.

[0038] Specifically, when executing the video playback method provided by the present disclosure, the video playback device can also utilize the network component of the terminal 1 to interact with the server 2, obtain the image data and video data stored in the server 2, and perform corresponding processing and display.

[0039] Figure 1 The shown architecture can be applied to the field of information presentation. In other words, it can be used for information presentation in various scenarios.

[0040] For example, the video playback method provided by the present disclosure can be applied to game scenarios based on augmented reality display. For example, in some "treasure hunt" games based on augmented reality display technology, the video playback method provided by the present disclosure can be used to push and present "clue" videos in the "treasure hunt" process.

[0041] For example, the video playback method provided by the present disclosure can be applied to advertising scenarios based on augmented reality display. For example, for some commodities or products, the video playback method provided by the present disclosure can be used to present relevant videos of these commodities, thereby providing users with more information about the commodities and enhancing the user experience.

[0042] For another example, in some scenarios where "cloud interpretation" is performed on landmark buildings, museums, etc. based on augmented reality display technology, for images of "landmarks", "buildings", "museum collections", "historical pictures", etc. that require "cloud interpretation", the video playback method provided by the present invention is used to play videos associated with the "landmarks", "buildings", "museum collections", "historical pictures", etc., thereby providing better services to users.

[0043] In addition, in some daily life scenes where the shooting function is used, the video playback method provided by the present disclosure can also be used to play video information, thereby presenting more information about the scene to the user and enhancing the user's interactive experience.

[0044] For further example, in scenarios such as "scan code to pay" and "take pictures" that require turning on the terminal camera for real-scene shooting, the video playback method provided by the present invention can be executed together to complete the presentation of information in various daily life scenarios.

[0045] The video playback method provided by the present disclosure is further described below:

[0046] First, Figure 2 A flowchart of a video playback method provided by an embodiment of the present disclosure. Figure 2 , the video playback method provided by the embodiment of the present disclosure includes:

[0047] Step 101: obtaining a real scene shooting image, and detecting a target image in the real scene shooting image;

[0048] Step 102: determining a display position of the target image in the real scene captured image;

[0049] Step 103: Acquire a target video associated with the target image, and play the target video at the display position of the target image in the real-scene captured image.

[0050] It should be noted that the execution subject of the processing method provided in this embodiment is the aforementioned video playback device, which specifically refers to a client or display terminal that can be installed or integrated on a terminal in some embodiments of the present disclosure. The user can operate the video playback device through the terminal so that the video playback device can respond to the operation triggered by the user.

[0051] Figure 3 A schematic diagram of a first interface of a video playback method provided in an embodiment of the present disclosure.

[0052] First, if Figure 3 As shown, the video playback device will obtain a real-scene captured image, which can be an image obtained by the terminal calling its own shooting component to shoot the current environment, or it can be a real-time image of the real scene obtained by the video playback device through other means.

[0053] Subsequently, the video playback device will perform image recognition in the real-scene captured image to determine whether there is a target image that can be used for video playback in the real-scene captured image. It can be understood that the recognition of the target image in the real-scene captured image by the video playback device can be achieved through image recognition technology. In one embodiment, the target image can be a two-dimensional plane image, and the corresponding display position can be the position where the two-dimensional plane figure is located. In one embodiment, the target image can also be an image of a three-dimensional object, and the corresponding display position can be the projection position of the three-dimensional object on the two-dimensional plane, etc. The image recognition technology according to the embodiment of the present disclosure can be implemented based on two-dimensional image recognition technology, that is, by using image recognition technology, image recognition can be performed for a preset plane picture, a projection surface of a three-dimensional object, and a plane picture or a plane picture that has a certain deformation. In addition, when the target image includes an image of a three-dimensional object, according to the embodiment of the present disclosure, it can be implemented based on object recognition technology. The present disclosure does not limit the specific image recognition technology.

[0054] By performing image recognition on the real-scene captured image, the video playback device can detect the display position of the target image in the real-scene captured image. Figure 3 As shown, the target image may include Figure 3 The image in the image frame described in the above, correspondingly, the display position can be framed by the image frame 301, and the image frame 301 is used to represent the playback area of ​​the target video. The display position of the target image can be specifically determined by the image position of the target image, and the image position can include but is not limited to the image edge position and image vertex position of the target image.

[0055] Afterwards, the video playback device may determine a target video associated with the target image, and play the target video at the display position where the target image is located in the real-scene captured image.

[0056] In the above process, in order to play the video at the display position of the image, the target video needs to be rendered based on three-dimensional space according to the display position of the image, so that the video screen of the target video can fit the display position and be played.

[0057] In a specific implementation, the corresponding playback area in the real-scene captured image may be determined according to the display position, and then the target video may be pre-processed according to the playback area, and the target video may be played in the playback area.

[0058] It can be understood that the above-mentioned display position refers to the range occupied by the target image in the real-scene captured image, which may include the image edge position, or / and the image vertex position.

[0059] Therefore, when determining the playback area, a corresponding image frame can be divided in the real scene captured image according to the image edge position and / or the image vertex position of the target image to serve as the playback area (eg Figure 3 301 shown).

[0060] Then, according to the spatial features of the playback area in the real scene shooting image, the video data of the target video can be subjected to three-dimensional spatial rendering processing to obtain a video image in the playback area (such as Figure 3 301) in the example shown in FIG. Figure 3 302 shown).

[0061] Among them, the spatial features are used to represent the spatial position attributes of the playback area in the three-dimensional space of the real-scene shooting image, such as the spatial position coordinates of the vertices of the playback area, the spatial position coordinates of the edges of the playback area, the spatial angle information between the plane where the playback area is located and the shooting plane of the real-scene shooting image, etc.

[0062] In combination with the spatial characteristics of the playback area, the video data of the target video may be subjected to three-dimensional spatial rendering processing so that the rendered video screen of the target video can fit the playback area in three-dimensional space. Optionally, the three-dimensional spatial rendering processing may include pixel coordinate mapping processing, that is, mapping the two-dimensional pixel coordinates of each pixel in the video screen of the target video to the three-dimensional coordinates of the playback area by spatial mapping. Of course, the three-dimensional spatial rendering processing may also be implemented by other existing methods, which is not limited by the present application.

[0063] Finally, the effect of playing the target video at the display position in the real-scene shooting image is achieved.

[0064] In order to facilitate the rapid deployment of the association relationship between the target video and the target image, and also to facilitate the rapid acquisition of the target video associated with the target image, in an optional implementation, the association relationship between the target video and the target image is pre-established and stored in the aforementioned server.

[0065] That is to say, Figure 4A signaling interaction diagram of a video playback method provided by an embodiment of the present disclosure. The association relationship between the target image and the target video is pre-built in the server. The pre-building method of the association relationship can be found in Figure 4 :

[0066] First, the terminal can obtain images and videos through various channels, and then through the interface (such as Figure 5 The target image and target video to be associated are uploaded to the server as shown in the interface. The image and video can be obtained by shooting at the terminal, downloaded by the terminal through the network, or obtained from other terminals through near-field transmission, which is not limited in the present disclosure.

[0067] Subsequently, after receiving the target image and the target video to be associated, the server will associate and store the two to determine the association relationship between the two. Specifically, the server may pre-store a storage list of association relationships for storing the association relationships between the images and videos to be associated uploaded by different terminals. The specific storage method will be described in the following embodiment and is not limited here.

[0068] Afterwards, when the terminal starts the camera, the video playback method will be executed in the aforementioned manner, that is, the terminal will shoot the real scene, obtain the corresponding real scene shooting image, and then perform image recognition on the real scene shooting image to obtain the target image in the real scene shooting image; then, determine the display position of the target image and send the target image to the server.

[0069] At this time, the server will determine the target video corresponding to the target image according to the pre-established association relationship in the storage list, and send the target video to the terminal. Finally, after the terminal receives the target video, the terminal will play the target video at the display position of the target image in the real scene shooting image.

[0070] It should be noted that although Figure 4 It is shown that the terminal for uploading the target image and the target video to be associated is the same terminal as the terminal for playing the video using this solution. However, in other examples or actual uses, the terminal for uploading the target image and the target video to be associated may be a different terminal from the terminal for playing the video using this solution. That is, this application does not impose any restrictions on whether the terminal for uploading the target image and the target video to be associated is the same terminal as the terminal for playing the video using this solution, and those skilled in the art can determine it by themselves in combination with the actual scenario.

[0071] In particular, since in the technical solution provided in the present invention, the target video associated with the target image can be played directly without the need for three-dimensional virtual modeling, the information owner who needs to present information can quickly associate the target image and the target video through the following operations, so that more users can obtain the information that the information owner wants to present, and the cost and preparation difficulty are greatly reduced.

[0072] Specifically, Figure 5 Schematic diagram of the second interface of a video playback method provided by an embodiment of the present disclosure. Figure 5 As shown, the terminal stores a number of image and video materials; the user (information owner) can select the target image (such as Figure 5 2) and the target video (as shown in Figure 5 That is, first, the video playback device will respond to the upload operation triggered by the user (information owner) to determine the target image and target video to be associated. Then, the video playback device will upload the target image and target video to be associated to the server, so that the server can associate and store the target image and target video to be associated.

[0073] There are many ways of associating and storing. For example, identification IDs can be set for images and videos respectively, and the target image and target video to be associated can be stored by storing the two identification IDs in a corresponding manner. In other words, the specific implementation method of the terminal sending the target image to the server is that the terminal sends the identification ID of the target image to the server, so that the server can find the target image with the corresponding identification ID from a large number of pre-stored images according to the identification ID of the target image, and send the corresponding target video.

[0074] Of course, in other associated storage methods, the associated target image and target video can also be stored by symmetric key encryption and decryption. For example, when storing, the target image can be processed to obtain a unique key, and the target video associated with the target image can be encrypted and stored using the key; when used subsequently, the target image can be processed again to obtain the aforementioned unique key, and the unique key can be used to find a video that can be decrypted from several stored videos, and the video will be the target video.

[0075] After completing the above configuration of the target image and the target video, the user (information recipient) can watch the target video (video 3) associated with the target image (image 2) on the terminal through the method provided in the above embodiment.

[0076] When in use, after the terminal sends the detected target image to the server, it will receive the target video associated with the target image returned by the server. It is particularly noted that for the terminal, it can send the image data of the target image to the server, and can also analyze and process the image data of the target image to obtain the image identification ID of the target image and / or the image features of the target image, and send them to the server. Regardless of image data, image identification ID or image features, the server can determine the corresponding target video according to the pre-established association relationship, and send the corresponding target video to the terminal for display.

[0077] It should be noted that when receiving the target images and target videos to be associated uploaded by the user, the user can upload at least one group of target images and target videos to be associated at the same time. Among them, different groups of target images and target videos to be associated can be associated with different identifiers so that the server can store different groups of target images and target videos to be associated. In this way, the configuration efficiency of the target images and target videos to be associated can be greatly improved. By uploading multiple groups of associated target images and target videos, video information presentation for multiple target images can be achieved, which can further increase the user's interactive experience.

[0078] It is understandable that the user who uploads the target image and target video to be associated and the user who uses this solution to play the video may be the same or different. Taking the above-mentioned actual application scenario as an example, in the scenario where the video playback method provided by the present disclosure presents related videos for some commodities or products, the user who uploads the target image and target video to be associated may be a product promoter, or a promoter of the commodity video; and the user who uses this solution to play the video may be a product user, a commodity video receiver or viewer, etc.

[0079] also, Figure 6 A schematic diagram of a third interface of a video playback method provided by an embodiment of the present disclosure. In other optional embodiments, the target image may include an image of a three-dimensional object in a real-life captured image. By identifying the image of the three-dimensional object, a two-dimensional plane formed by projecting the surface of the three-dimensional object may be used as the display position of the target image, and a video associated with the target image may be displayed at the display position. Figure 6 As shown, Figure 6 Take the vase in the figure as an example of the target image, and the video associated with the vase will be played at the projection plane of the vase.

[0080] In addition, in one embodiment, during the continuous playback of the target video, the video playback device will continue to track the display position of the target image, so as to adjust the playback position of the target video accordingly according to the change of the display position of the target image, so as to play the target video at the real-time display position of the target image. Figure 6 As shown, the target image is Figure 6 When the shooting angle of the real scene image changes (for example, the shooting angle of the vase changes), the display position of the identified target image will also change accordingly. At this time, the video playback device will adjust the position and size of the image frame in real time and display the target video in the adjusted image frame (playback area). Figure 6 In the scheme shown, at the initial stage, the side surface (projection surface) of the target image vase (three-dimensional object) is photographed at a vertical angle. At this time, the display position of the target image vase can be identified, and the corresponding target video is displayed at the display position. Subsequently, the camera is rotated to adjust the shooting angle. At this time, it can be identified that the display position of the side surface (projection surface) of the target image vase (three-dimensional object) has been adjusted to a certain extent, and the corresponding target video is displayed at the adjusted display position.

[0081] In addition, in other optional implementations, in order to provide more information to the user, the user can trigger the target video, and the display interface of the video playback device will be switched to display the information associated with the target video. That is, the video playback device will respond to the user's triggering operation on the target video being played, determine the information associated with the triggered target video, and display the information.

[0082] The information associated with the target video may be web page information, other image information, program information of other applications, etc. For example, if the target image is an image of a product and the target video shows the product usage of a brand product, the associated information may be a web page introduction of the brand product, or a purchase link of the brand product in a certain online application store, etc.; for another example, if the target image is an image of a scenic spot and the target video is a promotional video of the scenic spot, the associated information may be more pictures of the scenic spot, or a web page introduction of the scenic spot on the official website, or ticket purchase information of the scenic spot, etc.

[0083] In addition, in other optional embodiments, the video playback device also supports breakpoint playback of the target video, that is, when the display position of a target image corresponding to a target video is lost in the real-life captured image (for example, the target video is played to 00':30" at this time), and then the display position of the target image is found within a preset time period, the video playback device can continue to play the target video from the last playback progress of the target video (i.e., 00':30") (continue playing from 00':30").

[0084] That is to say, when playing the target video, the video playing device will also obtain the playing progress of the target video; and play the target video according to the playing progress.

[0085] In an optional implementation, the video playback device may store the playback progress of the target video, such as the current playback time information. Each time the target video is played, the video playback device may first extract the playback progress such as the current playback time information of the target video, then determine the start time of this playback, and finally play the target video from the start time. In this way, the breakpoint playback function of the target video is realized, further improving the user's audio-visual experience.

[0086] On the basis of the above-mentioned embodiments, the number of target images obtained by identifying the real scene shooting image in the video playback method provided by the present disclosure can be multiple. That is, when the video playback device performs image recognition on the real scene shooting image, it can identify the target image in the image and play the target videos of the multiple target images simultaneously.

[0087] Figure 7 A flowchart of another video playback method provided by the present disclosure is shown in FIG. Figure 7 As shown, the method includes:

[0088] Step 201, obtaining a real scene shooting image;

[0089] Step 202: determine the display position of each target image in the real scene captured image;

[0090] Step 203: storing the target videos associated with the multiple target images in a preset video play list according to the acquisition order of the target videos associated with the multiple target images;

[0091] Step 204: According to the storage order of each target video in the video play list, determine at least one target video from the video play list as the target video to be played;

[0092] Step 205: Play the target video to be played at the display position of the target image associated with the target video to be played.

[0093] It should be noted that the execution subject of the processing method provided in this embodiment is the aforementioned video playback device, which specifically refers to a client or display terminal that can be installed or integrated on a terminal in some embodiments of the present disclosure. The user can operate the video playback device through the terminal so that the video playback device can respond to the operation triggered by the user.

[0094] Different from the above-mentioned embodiment, in this embodiment, the real scene shooting image will include multiple target images, and the video playback device will determine the display position of each target image in the real scene shooting image, and play the target video associated with each target image at its corresponding display position. The playback method for playing each target video at its corresponding display position can refer to the above-mentioned method, and this embodiment will not be described in detail.

[0095] In the above process, the number of target videos may be large, and if a large number of target videos are played at the same time, the terminal may be stuck. Therefore, in order to obtain a better video playback effect and to provide a better visual experience for the user, a video play list may be set in the video playback device to set the number of target videos to be played simultaneously in the real scene shooting image.

[0096] Specifically, Figure 8 FIG. 4 is a schematic diagram of a fourth interface of a video playback method provided by an embodiment of the present disclosure. Figure 8 As shown, first, the video playback device obtains a real-scene shooting image, then performs image recognition on multiple target images in the real-scene shooting image, and uses multiple image frames to frame the display positions of each target image in the real-scene shooting image in sequence, thereby obtaining a plurality of image frames (playback areas). Figure 8 As shown, the playback area corresponding to the target image A is 801 , the playback area corresponding to the target image B is 802 , and the playback area corresponding to the target image C is 803 .

[0097] Then, the video playback device will send each target image to the server respectively, so that the server can determine the target video corresponding to each target image according to the preset association relationship, and return each target video to the video playback device.

[0098] When the video playback device receives these target videos, it will store them in the video play list. When playing the target videos, the video playback device will select one or more target videos as the target videos to be played according to the storage order of the target videos stored in the video play list, and play each target video in the play area of ​​the target image associated with it, for example, target video 1 associated with target image A is played in the corresponding play area 801, target video 2 associated with target image B is played in the corresponding play area 802, and target video 3 associated with target image C is played in the corresponding play area 803.

[0099] It should be noted that the video playlist stores all target videos obtained from the server from the moment the real-scene shooting image is obtained. The video playlist can also determine the cleanup cycle of the list and clean up the target videos stored in the list according to the cleanup cycle.

[0100] In addition, the acquisition time of the target video is also stored in the video playlist. When each target video is stored in the video playlist, the target videos are stored in the video playlist in the reverse order of the acquisition time of each target video. In other words, the target video that is acquired closer to the current moment is stored at the top of the video playlist and is played first.

[0101] In other optional embodiments, if multiple videos include audio, playing the audio of multiple target videos at the same time will result in a poor visual experience and information acquisition experience for the user. To avoid this problem, in this embodiment, the audio and video can be played separately. For example, while playing the video images of multiple target videos to be played at the same time, only the audio data of the only target video being played is played.

[0102] Specifically, when playing the target video, each target video to be played can be decoded first to obtain the audio data and video data of each target video to be played; the video data of each target video to be played can be played; and the audio data of the target video to be played that was recently acquired and stored in the video playlist can be played.

[0103] Among them, for the target video corresponding to the currently played audio data, its audio data and video data will be synchronously processed using audio and video synchronization technology to ensure audio and video synchronization during playback. In this way, users can only receive the sound from the same target video at the same time, while watching the pictures of multiple target videos, and can browse more video information while ensuring that the audio-visual experience of the video information is not affected.

[0104] The video playback method provided in the embodiment of the present disclosure obtains a real-scene shot image; detects a target image in the real-scene shot image; determines the display position of the target image in the real-scene shot image; obtains a target video associated with the target image, and plays the target video at the display position of the target image in the real-scene shot image. The video playback method provided in this embodiment can reduce the preparation cycle and cost when presenting information using augmented reality display technology. On the other hand, it also provides users with more presentation channels for playing video information, so that users can get a better interactive experience and visual experience.

[0105] Corresponding to the video playing method of the above embodiment, Fig. 9 This is a structural block diagram of a video playback device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Fig. 9 The video playback device includes: an acquisition module 10, a processing module 20 and a playback module 30.

[0106] Acquisition module 10, the user obtains the real scene shooting image. In one embodiment, the acquisition module may include an image acquisition device configured by the video playback device itself, and acquires the real scene shooting image by capturing the real scene image in real time. In addition, the acquisition module can also acquire the real scene shooting image stored or shot by the server or the video playback device itself, which is not limited by the present disclosure.

[0107] The processing module 20 is used to detect the target image in the real scene shooting image and determine the display position of the target image in the real scene shooting image.

[0108] The playing module 30 is used to obtain a target video associated with the target image, and play the target video at the display position of the target image in the real scene shooting image.

[0109] In an optional embodiment, the video playback device also includes: a first interaction module; the first interaction module is used to respond to the user's triggering operation on the target video to be played, and determine the information associated with the triggered target video; so that the playback module 20 can display the information.

[0110] In an optional embodiment, the video playback device further includes: a second interaction module;

[0111] The second interaction module is used to receive at least one set of target images and target videos to be associated uploaded by a user, and upload the at least one set of target images and target videos to be associated to a server so that the server can associate and store the target images and target videos to be associated.

[0112] In an optional embodiment, when acquiring the target video associated with the target image, the playback module 30 is specifically configured to: send the target image to a server, and receive the target video associated with the target image returned by the server.

[0113] In an optional embodiment, the real-scene captured image includes multiple target images; when the playback module 30 obtains a target video associated with the target image and plays the target video at the display position of the target image in the real-scene captured image, it is specifically used to: store the target videos associated with the multiple target images in a preset video play list in the order in which the target videos associated with the multiple target images are acquired; determine at least one target video from the video play list as a target video to be played according to the storage order of each target video in the video play list; and play the target video to be played at the display position of the target image associated with the target video to be played.

[0114] In an optional embodiment, the video playlist also stores the acquisition time of the target video; when the playback module 30 stores the target videos associated with the multiple target images in a preset video playlist according to the acquisition order of the target videos associated with the multiple target images, it is specifically used to: store the target videos in the video playlist in reverse order of their acquisition time.

[0115] In an optional embodiment, when playing the target video to be played, the playback module 30 is specifically used to: decode each target video to be played to obtain audio data and video data of each target video to be played; play the video data of each target video to be played; and play the audio data of the target video to be played that was most recently acquired and stored in the video playlist.

[0116] In an optional embodiment, when the playback module 30 plays the target video at the display position of the target image in the real-scene shooting image, it is specifically used to: determine the corresponding playback area in the real-scene shooting image according to the display position; perform video preprocessing on the target video according to the playback area, and play the target video in the playback area.

[0117] In an optional embodiment, when the playback module 30 performs video preprocessing on the target video according to the playback area and plays the target video in the playback area, it is specifically used to: perform three-dimensional spatial rendering processing on the video data of the target video according to the spatial features of the playback area in the real-life captured image, so as to play the target video in the playback area.

[0118] In an optional embodiment, the display position includes: an image edge position, or / and an image vertex position.

[0119] In an optional embodiment, when playing the target video, the playing module 30 is specifically used to: obtain the playing progress of the target video; and play the target video according to the playing progress.

[0120] The video playback device provided in the embodiment of the present disclosure obtains a real-scene shot image; detects a target image in the real-scene shot image; determines a display position of the target image in the real-scene shot image; obtains a target video associated with the target image, and plays the target video at the display position of the target image in the real-scene shot image. The video playback device provided in the embodiment of the present disclosure can reduce the preparation cycle and cost when presenting information using augmented reality display technology. On the other hand, it also provides users with more continuous channels to continue video information, so that users can get better interactive experience and visual experience.

[0121] The electronic device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0122] refer to Fig.10 , which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure, and the electronic device 900 may be a terminal device or a media library. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (Portable Media Players, PMPs), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Fig.10 The electronic device shown is only one embodiment and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0123] like Fig.10As shown, the electronic device 900 may include a processor 901 for executing the video playback method (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The video playback method 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0124] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.10 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0125] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the methods shown in the flowcharts according to the embodiments of the present disclosure. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the video playback method 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0126] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0127] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0128] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0129] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or media library. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0132] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0133] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific embodiment of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] The following are some embodiments of the present disclosure.

[0135] In a first aspect, according to one or more embodiments of the present disclosure, a video playback method includes:

[0136] Obtaining real scene shooting images;

[0137] Detecting a target image in the real scene captured image;

[0138] Determining a display position of the target image in the real scene captured image;

[0139] A target video associated with the target image is acquired, and the target video is played at a display position of the target image in the real-scene captured image.

[0140] In an optional embodiment, the method further includes:

[0141] In response to a user's triggering operation on a target video to be played, determining triggered information associated with the target video;

[0142] The information is displayed.

[0143] In an optional embodiment, the method further includes:

[0144] receiving at least one set of target images and target videos to be associated uploaded by a user;

[0145] The at least one group of target images and target videos to be associated is uploaded to a server, so that the server associates and stores the target images and target videos to be associated.

[0146] In an optional embodiment, the step of acquiring a target video associated with the target image includes:

[0147] Sending the target image to a server;

[0148] The target video associated with the target image returned by the server is received.

[0149] In an optional embodiment, the real scene captured image includes a plurality of target images;

[0150] The step of obtaining a target video associated with the target image and playing the target video at a display position of the target image in the real-scene captured image includes:

[0151] According to the acquisition order of the target videos associated with the multiple target images, storing the target videos associated with the multiple target images in a preset video play list;

[0152] According to the storage order of each target video in the video play list, determining at least one target video from the video play list as a target video to be played;

[0153] The target video to be played is played at the display position of the target image associated with the target video to be played.

[0154] In an optional embodiment, the video playlist also stores the acquisition time of the target video;

[0155] The step of storing the target videos associated with the multiple target images in a preset video play list according to the acquisition order of the target videos associated with the multiple target images comprises:

[0156] The target videos are stored in the video playlist in reverse order of their acquisition time.

[0157] In an optional embodiment, the playing of the target video to be played includes:

[0158] Decoding each target video to be played to obtain audio data and video data of each target video to be played;

[0159] Playing the video data of each target video to be played; and

[0160] The audio data of the target video to be played that is recently acquired and stored in the video playlist is played.

[0161] In an optional embodiment, playing the target video at the display position of the target image in the real-scene captured image includes:

[0162] Determining a corresponding playback area in the real-scene captured image according to the display position;

[0163] The target video is pre-processed according to the playback area, and the target video is played in the playback area.

[0164] In an optional embodiment, the performing video preprocessing on the target video according to the playback area and playing the target video in the playback area includes:

[0165] According to the spatial features of the playback area in the real-scene shooting image, three-dimensional spatial rendering processing is performed on the video data of the target video to play the target video in the playback area.

[0166] In an optional embodiment, the display position includes: an image edge position, or / and an image vertex position.

[0167] In an optional embodiment, the playing of the target video includes:

[0168] Obtaining the playback progress of the target video;

[0169] The target video is played according to the playback progress.

[0170] In a second aspect, according to one or more embodiments of the present disclosure, a video playback device includes: an acquisition module, a processing module, and a playback module;

[0171] Acquisition module, where users acquire real-scene images;

[0172] A processing module, used for detecting a target image in the real scene captured image, and determining a display position of the target image in the real scene captured image;

[0173] The playing module is used to obtain a target video associated with the target image and play the target video at the display position of the target image in the real-scene shooting image.

[0174] In an optional embodiment, the video playback device further includes: a first interaction module;

[0175] The first interaction module is used to respond to a user's triggering operation on a target video to be played, and determine the triggered information associated with the target video, so that the playback module can display the information.

[0176] In an optional embodiment, the video playback device further includes: a second interaction module;

[0177] The second interaction module is used to receive at least one set of target images and target videos to be associated uploaded by a user, and upload the at least one set of target images and target videos to be associated to a server so that the server can associate and store the target images and target videos to be associated.

[0178] In an optional embodiment, when the playback module obtains the target video associated with the target image, it is specifically used to: send the target image to a server, and receive the target video associated with the target image returned by the server.

[0179] In an optional embodiment, the real-scene captured image includes multiple target images; when the playback module obtains the target video associated with the target image and plays the target video at the display position of the target image in the real-scene captured image, it is specifically used to: store the target videos associated with the multiple target images in a preset video play list in the order in which the target videos associated with the multiple target images are acquired; determine at least one target video from the video play list as the target video to be played according to the storage order of each target video in the video play list; and play the target video to be played at the display position of the target image associated with the target video to be played.

[0180] In an optional embodiment, the video playlist also stores the acquisition time of the target video; when the playback module stores the target videos associated with the multiple target images in a preset video playlist in the order in which the target videos are acquired, it is specifically used to: store each target video in the video playlist in reverse order of the acquisition time of each target video.

[0181] In an optional embodiment, when playing the target videos to be played, the playback module is specifically used to: decode each target video to be played to obtain audio data and video data of each target video to be played; play the video data of each target video to be played; and play the audio data of the target video to be played that was most recently acquired and stored in the video playlist.

[0182] In an optional embodiment, when the playback module plays the target video at the display position of the target image in the real-scene capture image, it is specifically used to: determine the corresponding playback area in the real-scene capture image according to the display position; perform video preprocessing on the target video according to the playback area, and play the target video in the playback area.

[0183] In an optional embodiment, when the playback module performs video preprocessing on the target video according to the playback area and plays the target video in the playback area, it is specifically used to: perform three-dimensional spatial rendering processing on the video data of the target video according to the spatial features of the playback area in the real-life captured image, so as to play the target video in the playback area.

[0184] In an optional embodiment, the display position includes: an image edge position, or / and an image vertex position.

[0185] In an optional embodiment, when playing the target video, the playback module is specifically used to: obtain the playback progress of the target video; and play the target video according to the playback progress.

[0186] In a third aspect, according to one or more embodiments of the present disclosure, an electronic device includes: at least one processor and a memory;

[0187] The memory stores computer-executable instructions;

[0188] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video playback method as described in any of the preceding items.

[0189] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions, and when a processor executes the computer execution instructions, the video playback method as described in any of the preceding items is implemented.

[0190] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0191] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0192] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely the embodiment forms of implementing the claims.

Claims

1. A video playback method, characterized in that: include: Obtaining real-scene shooting images; Detecting a target image in the real scene captured image; Determining a display position of the target image in the real scene captured image; A target video associated with the target image is acquired, and based on the spatial features at the display position in the real-scene captured image, the target video is played at the display position of the target image in the real-scene captured image, wherein the spatial features at the display position are used to perform three-dimensional spatial rendering processing on the video data of the target video.

2. The video playback method according to claim 1, characterized in that: Also includes: In response to a user's triggering operation on a target video to be played, determining triggered information associated with the target video; The information is displayed.

3. The video playback method according to claim 1, characterized in that: Also includes: receiving at least one set of target images and target videos to be associated uploaded by a user; The at least one group of target images and target videos to be associated is uploaded to a server, so that the server associates and stores the target images and target videos to be associated.

4. The video playback method according to claim 3, characterized in that: The acquiring of a target video associated with the target image comprises: Sending the target image to a server; The target video associated with the target image returned by the server is received.

5. The video playback method according to claim 1, characterized in that: The real scene captured image includes a plurality of target images; The step of obtaining a target video associated with the target image and playing the target video at a display position of the target image in the real-scene captured image includes: According to the acquisition order of the target videos associated with the multiple target images, storing the target videos associated with the multiple target images in a preset video play list; According to the storage order of each target video in the video play list, determining at least one target video from the video play list as a target video to be played; The target video to be played is played at the display position of the target image associated with the target video to be played.

6. The video playback method according to claim 5, characterized in that: The video playlist also stores the acquisition time of the target video; The step of storing the target videos associated with the multiple target images in a preset video play list according to the acquisition order of the target videos associated with the multiple target images comprises: The target videos are stored in the video playlist in reverse order of their acquisition time.

7. The video playback method according to claim 5, characterized in that: The playing of the target video to be played includes: Decoding each target video to be played to obtain audio data and video data of each target video to be played; Playing the video data of each target video to be played; and The audio data of the target video to be played that is recently acquired and stored in the video playlist is played.

8. The video playback method according to claim 1, characterized in that: The playing the target video at the display position of the target image in the real scene captured image based on the spatial features of the display position in the real scene captured image comprises: Determining a corresponding playback area in the real-scene captured image according to the display position; The target video is played in the playback area based on the spatial features of the playback area in the real scene shooting image.

9. The video playback method according to claim 8, characterized in that: The step of playing the target video in the playback area based on the spatial feature of the playback area in the real scene captured image includes: According to the spatial features of the playback area in the real-scene shooting image, three-dimensional spatial rendering processing is performed on the video data of the target video to play the target video in the playback area.

10. The video playback method according to claim 8, characterized in that: The display position includes: an image edge position, or / and an image vertex position.

11. The video playback method according to any one of claims 1 to 10, characterized in that: The playing of the target video includes: Obtaining the playback progress of the target video; The target video is played according to the playback progress.

12. A video playback device, characterized in that: include: An acquisition module is used to acquire real-scene shooting images; A processing module, used for detecting a target image in the real scene captured image, and determining a display position of the target image in the real scene captured image; A playback module is used to obtain a target video associated with the target image, and based on the spatial features at the display position in the real-scene shooting image, play the target video at the display position of the target image in the real-scene shooting image, wherein the spatial features at the display position are used to perform three-dimensional spatial rendering processing on the video data of the target video.

13. An electronic device, wherein: include: at least one processor; as well as Memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video playback method according to any one of claims 1 to 11.

14. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the video playback method according to any one of claims 1 to 11 is implemented.

Images