Panoramic video playing method and playing device

Abstract

The invention provides a playing method and a playing device of a panoramic video. The method comprises the steps that in response to a panoramic video downloading operation, equipment information is acquired, and the equipment information comprises a system version; if the system version is within the preset version range, storing the panoramic video in a sandbox folder, and extracting video feature information from the panoramic video; adjusting transcoding parameters based on the equipment information and the video feature information; and transcoding the panoramic video at least based on the adjusted transcoding parameter to play the transcoded panoramic video. Therefore, it is ensured that the high-resolution panoramic video can be smoothly played on equipment with a low system version, and the definition of played pictures is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure generally relates to the field of video playing, and in particular, to a playing method of a panoramic video, a playing device of the panoramic video, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] With the increasing demand for home security, monitoring camera devices have been widely used in daily monitoring. The common monitoring camera devices on the market currently include single-camera camera devices and double-camera camera devices. The double-camera camera devices are usually used to meet the needs of different monitoring angles. Specifically, the double-camera camera devices use two high-resolution lenses to respectively collect images and / or videos of different angles, and then seamlessly stitch the images and / or videos of different angles into panoramic images and / or panoramic videos, so as to meet the needs of user groups with super large angle monitoring needs. However, the panoramic videos generated by the double-camera camera devices cannot be normally played, which has become a problem to be solved in the current monitoring and care field. SUMMARY

[0003] In a first aspect of the present disclosure, a playing method of a panoramic video is provided, including: in response to a panoramic video downloading operation, obtaining device information, the device information including a system version; if the system version is within a preset version range, saving the panoramic video into a sandbox folder and extracting video feature information from the panoramic video; adjusting transcoding parameters based on the device information and the video feature information; and transcoding the panoramic video based on at least the adjusted transcoding parameters, to play the transcoded panoramic video.

[0004] In a second aspect of the present disclosure, a playing device of a panoramic video is provided, including a device information obtaining module configured to, in response to a panoramic video downloading operation, obtain device information, the device information including a system version; a feature information extracting module configured to, if the system version is within a preset version range, save the panoramic video into a sandbox folder and extract video feature information from the panoramic video; a transcoding parameter adjusting module configured to adjust transcoding parameters based on the device information and the video feature information; and a video transcoding module configured to transcode the panoramic video based on at least the adjusted transcoding parameters, to play the transcoded panoramic video.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer readable storage medium is provided. The computer readable storage medium has stored thereon a computer program, which is executable by a processor to implement the method of the first aspect.

[0007] It should be understood that all statements herein made regarding the exemplary embodiments of the present disclosure are intended to be illustrative of the key and essential characteristics of the embodiments of the present disclosure, and it is not intended that the scope of the present disclosure be limited to such disclosed embodiments. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0008] The above and other features, aspects and advantages of various embodiments of the present disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals denote like elements, and wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented is shown; Figure 2 A flowchart showing a playing method of a panoramic video according to some embodiments of the present disclosure is shown; Figure 3 A transcoding flowchart of a panoramic video according to some embodiments of the present disclosure is shown; Figure 4 A block diagram of a playing device of a panoramic video according to some embodiments of the present disclosure is shown; and Figure 5 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0010] Embodiments of the present disclosure will be described in more detail by referring to the drawings. Although certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein, but rather, the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of the present disclosure.

[0011] It should be noted that the titles of any sections / subsections provided herein are not limiting. Various embodiments are described throughout this document and any type of embodiment can be included under any section / subsection. Furthermore, embodiments described in any section / subsection can be combined with any other embodiment described in the same section / subsection and / or a different section / subsection in any manner.

[0012] In the description of embodiments of the disclosure, the term "comprising" and similar terms are to be interpreted as open-ended, i.e., "including but not limited to". The term "based on" is to be interpreted as "based, at least in part, on". The term "one embodiment" or "the embodiment" is to be interpreted as "at least one embodiment". The term "some embodiments" is to be interpreted as "at least some embodiments". Other explicit and implicit definitions can also be included below. The terms "first", "second", etc. can refer to different or the same objects. Other explicit and implicit definitions can also be included below.

[0013] Data of users, acquisition and / or use of data, etc. can be involved in embodiments of the disclosure. These aspects all comply with corresponding laws and regulations and relevant provisions. In embodiments of the disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are performed on the premise that the user is aware of and confirms. Accordingly, when implementing embodiments of the disclosure, the type of data or information that can be involved, the use range, the use scenario, etc. should be notified to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations. The specific notification and / or authorization manner can vary according to actual situations and application scenarios, and the scope of the disclosure is not limited in this aspect.

[0014] For the convenience of description, some concepts and terms are defined first.

[0015] Video transcoding refers to converting an already compressed video stream into another video stream to adapt to different network bandwidths, different electronic device (terminal) processing capabilities and different user needs. Transcoding is essentially a process of transcoding and then encoding.

[0016] React Native refers to an open-source framework based on React, which allows developers to write native (iOS / Android) mobile applications using JavaScript. With front-end development efficiency, a mobile application experience close to native is achieved.

[0017] Native Module refers to the core mechanism in the React Native framework that connects JavaScript and native platform (iOS / Android) code, which is a communication bridge between JavaScript and the native system, retaining the cross-platform advantages of JS and using all the capabilities of the native system.

[0018] FFmpeg refers to a set of open-source cross-platform audio and video processing frameworks that provide multimedia processing capabilities such as recording, format conversion, transcoding, streaming media transmission, etc.

[0019] As briefly mentioned above, generally, the resolution of the panoramic video captured by the dual camera is high, for example, the resolution of the panoramic video can reach 4608*1296, of course, it can also be other higher resolution. At present, the system version of the traditional device (for example, the monitoring camera device) installed is low, for example, the system version is lower than iOS 15.3 version, when the panoramic video is played on the device with the low version of the system, the situation of lag or even unable to play will occur, and the panoramic video cannot be shared to other devices or third-party software.

[0020] To this end, embodiments of the present disclosure propose a panoramic video playing method. According to various embodiments of the present disclosure, in response to a panoramic video downloading operation, device information is acquired, the device information including a system version. Further, if the system version is within a preset version range, the panoramic video is saved into a sandbox folder, and video feature information is extracted from the panoramic video; based on the device information and the video feature information, transcoding parameters are adjusted; and at least based on the adjusted transcoding parameters, the panoramic video is transcoded to play the transcoded panoramic video. In this way, the panoramic video can be normally and smoothly played in the electronic device with the lower system version.

[0021] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0022] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In the environment 100, generally, an electronic device 110 transcodes a panoramic video 120 based on device information 130 to obtain a transcoded panoramic video 140, and plays the transcoded panoramic video 140 in the device. The panoramic video 120 refers to a panoramic video captured by a dual camera device. For example, the resolution of the panoramic video can reach 4608*1296, of course, it can also be other higher or lower resolution video. The captured panoramic video 120 can be stored in a software platform or a hardware platform such as a cloud, an SD memory card, etc.

[0023] The electronic device 110 can be any device with computing capability. In some embodiments, at least a portion of the electronic device 110 can be a server, an edge node, a computing device in a cloud environment, etc. For example, the electronic device 110 can be a server controlled and coordinated by a cloud operating system in a cloud environment, providing data processing and the like operations through a virtual machine or the like. In some embodiments, the electronic device 110 can be a terminal device. The terminal device can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a subnotebook computer, a netbook computer, a tablet computer, a media player, a

[0024] The following further describes in detail an example process of transcoding a panoramic video based on device information to obtain a transcoded panoramic video according to some embodiments of the present disclosure, and playing the transcoded panoramic video. Figure 2 A flowchart 200 of a method of playing a panoramic video according to some embodiments of the present disclosure is shown. The process 200 can be performed by Figure 1 the electronic device 110 as shown.

[0025] As Figure 2 shown, in some embodiments, at block 210, in response to a panoramic video download operation, device information is obtained, the device information including a system version.

[0026] For example, in response to a user-issued download operation for a panoramic video, the electronic device 110 can download the panoramic video from a cloud or an SD memory card through an App (application) installed in the device, such as a camera App or other type of application. Alternatively, in response to a user-issued screen recording operation, the electronic device 110 can also record a screen during playing of a live picture, and generate a panoramic video at the end of the screen recording within a predetermined time period.

[0027] Further, when the panoramic video is downloaded successfully or generated by screen recording, the electronic device 110 can obtain the device information in any suitable manner, and the device information includes the system version. For example, the system version installed in the device (e.g., a playback device) can include various versions of the Android system, various versions of the iOS system, various versions of the Harmony system, and the like. For example, various versions of the iOS system can include the iOS 15.3 version, a version greater than the iOS 15.3 version such as the iOS 16 version, the iOS 18.3 version, and a version less than the iOS 15.3 version such as the iOS 14 version. It should be understood that the system version can be any version, and the present disclosure does not make a specific limitation in this regard.

[0028] In block 220, if the system version is within the preset version range, the panoramic video is saved to the sandbox folder, and the video feature information is extracted from the panoramic video.

[0029] In some embodiments, if the system version is within the preset version range, the electronic device 110 can save the panoramic video to the sandbox folder. For example, if the preset version is the iOS 15.3 version, the electronic device 110 determines from the device information that the current system version is within the range below the iOS 15.3 version, and then saves the panoramic video obtained by downloading and / or screen recording to the sandbox folder, for example, a sandbox folder named camera device ID. This facilitates subsequent transcoding of the panoramic video stored in the sandbox folder, and the transcoding process will be described in detail below. In other words, the panoramic video stored in the sandbox folder is all the panoramic video that is played with stuttering or not supported in the electronic device 110. It should be understood that the preset version range can be a range below the iOS 15.3 version, and can also be other preset version ranges in which the panoramic video is played with stuttering or not supported, and the present disclosure does not make a specific limitation in this regard.

[0030] Correspondingly, if the electronic device 110 determines from the device information that the system version is the Android system version or the version range is within the range above the iOS 15.3 version, the panoramic video obtained by downloading and / or screen recording can be stored in the system album of the electronic device 110, for example, a mobile phone album named camera device ID. The panoramic video stored in the system album does not need to be transcoded, which facilitates the user to find the panoramic video in the system album and play or share the panoramic video to other devices through the electronic device 110.

[0031] In some embodiments, the idle state of the device is determined based on the device information; and if the device is in the idle state, the transcoding process for the panoramic video is triggered. Illustratively, the electronic device 110 can also extract the device state at the current time from the device information, and if it is detected that the device is in the idle state at the current time, the transcoding process is automatically triggered.

[0032] Next, the overall transcoding process is introduced.

[0033] In some embodiments, the panoramic video stored in the sandbox folder is obtained, the key area and the non-key area of each frame image of the panoramic video are determined by performing semantic segmentation on the panoramic video; the key feature information is extracted from the key area, and the non-key feature information is extracted from the non-key area; and the key feature information and the non-key feature information are taken as the video feature information.

[0034] The electronic device 110 can obtain the panoramic video stored in the sandbox folder, and perform semantic segmentation on the panoramic video by using a semantic segmentation algorithm, for example, performing semantic segmentation on all frame images in the panoramic video to obtain a semantic segmentation result. The electronic device 110 can determine the key area and the non-key area of each frame image of the panoramic video based on the semantic segmentation result. As an example, the electronic device 110 is a monitoring camera device, and the panoramic video is obtained by shooting a monitoring scene. The panoramic video includes multiple monitoring pictures. The key area of each monitoring picture may, for example, include a region located in a specific range in the center of the picture, and the non-key area may, for example, include an edge region of the monitoring picture. As another example, the key area of each monitoring picture may, for example, include one or more regions where specific monitoring objects (such as people, doors, windows, and other objects) are located, and the non-key area may, for example, include a region where non-monitoring objects (such as the sky, walls, and the like) are located. It should be understood that the key area and the non-key area can be adaptively adjusted, and the present disclosure does not make specific limitations in this regard.

[0035] The electronic device 110 can extract the key feature information from the key area, and extract the non-key feature information from the non-key area, and take the key feature information and the non-key feature information as the video feature information. For example, the feature information of important care objects such as people, doors, and windows is extracted from the key area as the key feature information, and the feature information of non-monitoring objects such as the sky and walls is extracted from the non-key area as the non-key feature information.

[0036] The prior art uses a unified transcoding strategy to transcode the entire video image, resulting in the transcoded panoramic video still appearing to be stuttering, blurred, and the like when played. The present application combines semantic analysis of the panoramic video with transcoding processing, divides the images in the panoramic video into key regions and non-key regions, assigns different transcoding strategies to the key regions and non-key regions, and respectively transcodes them, improving the picture clarity of the key regions after transcoding, improving the transcoding efficiency, and effectively avoiding the phenomena of stuttering, blurred pictures, and the like when playing the panoramic video.

[0037] Next, the specific process of respectively transcoding the key regions and non-key regions using different transcoding strategies is introduced.

[0038] In block 230, the transcoding parameters are adjusted based on the device information and the video feature information. The electronic device 110 dynamically adjusts the transcoding parameters according to the device information and the video feature information, so that the transcoding parameters are adapted to the transcoding capability of the device and the video features of the panoramic video, ensuring that the transcoded panoramic video can be played smoothly and clearly in the device.

[0039] The electronic device 110 obtains device information, which includes hardware information of the device, such as CPU model, GPU type, memory size, and the like. A transcoding scoring model is established based on the device information, and the transcoding scoring model is trained using device information of various different types of devices to obtain a trained transcoding scoring model, so as to score the transcoding capability of the device based on the trained transcoding scoring model.

[0040] In some embodiments, the trained transcoding scoring model is used to process the device information, outputting a transcoding score of the device; if the transcoding score of the device is greater than a preset score, the device is determined to be a strong transcoding device; or if the transcoding score of the device is less than a preset score, the device is determined to be a weak transcoding device.

[0041] From the device information, it is determined that the system version of the device is within a preset version range, resulting in the device being unable to normally play high-resolution panoramic videos. The electronic device 110 can use the trained transcoding scoring model to process the device information whose system version is within the preset version range, and predict the transcoding score of the device. As an example, if the device is various models of mobile phones, such as mobile phone iPhone SE2, the predicted score is 60 points, and the predicted transcoding score of mobile phone iPhone11 is 80 points.

[0042] For example, the preset score can be adaptively adjusted. For example, the preset score can be 70 points. It should be understood that the preset score can also be any other score, such as 80 points, 90 points, and the like. The present disclosure does not make specific limitations in this regard. After scoring the transcoding capability of the device by using the trained transcoding scoring model, the device is divided into two types of “weak transcoding device” and “strong transcoding device” by using the preset score. For example, if the transcoding score of the device is greater than or equal to the preset score of 70 points, the device is determined to be a strong transcoding device. If the transcoding score of the device is less than the preset score of 70 points, the device is determined to be a weak transcoding device. The electronic device 110 can assign different transcoding strategies to the devices with different transcoding capabilities, i.e., the strong transcoding device and the weak transcoding device.

[0043] In some embodiments, if the device is a strong transcoding device, the resolution of the key region is unchanged, the code rate of the key region is increased, the code rate of the non-key region is unchanged, and the resolution of the non-key region is reduced.

[0044] The electronic device 110 obtains transcoding parameters, including resolution and code rate. The resolution refers to the number and density of pixels in an image or video. The code rate refers to the number of bits of video information transmitted per second. At the same resolution, the higher the code rate, the clearer the image quality, but the terminal transcoding capability needs to be matched. After obtaining the transcoding parameters, it is determined whether the device is a strong transcoding device. If the device is a strong transcoding device, the electronic device 110 assigns a transcoding strategy for the key region in the panoramic video, which guarantees that the resolution of the key region is unchanged and the code rate of the key region is increased. For the non-key region, a transcoding strategy is assigned, which guarantees that the code rate is unchanged and the resolution of the non-key region is reduced.

[0045] For example, if the device is a strong transcoding device (such as iPhone 11), the electronic device 110 assigns a transcoding strategy for the key region in the panoramic video, which keeps the resolution unchanged at 1920*1080 and reduces the code rate from 8 Mbps to 5 Mbps. The electronic device 110 assigns a transcoding strategy for the non-key region in the panoramic video, which reduces the resolution from 2688*1296 to 1280*720 and reduces the code rate from 6 Mbps to 2 Mbps or guarantees that the code rate is unchanged at 6 Mbps.

[0046] In some embodiments, if the device is a weak transcoding device, the resolution and code rate of the key region are reduced, and the resolution and code rate of the non-key region are reduced.

[0047] After obtaining the transcoding parameters, it is determined whether the device is a weak transcoding device. If the device is a weak transcoding device, a transcoding strategy of reducing resolution and code rate is allocated to the key area and non-key area in the panoramic video. As an example, if the device is a weak transcoding device (such as iPhone SE2), the electronic device 110 allocates a transcoding strategy of reducing the resolution to 1280*720 and reducing the code rate to 3 Mbps to the key area in the panoramic video; and the electronic device 110 allocates a transcoding strategy of reducing the non-key area to 960*540 and reducing the code rate to 1.5 Mbps to the non-key area in the panoramic video.

[0048] In the prior art, since the transcoding parameters (such as code rate and resolution) are usually fixed values, the hardware differences (such as different transcoding capabilities of different devices) are not considered, and the video transcoded by using the fixed transcoding parameters will still have problems such as freezing when playing. The present application uses a trained transcoding score model to predict the transcoding capability of the device to obtain a transcoding score, compares the transcoding score with a preset score to determine whether the device is a strong transcoding device or a weak transcoding device, and dynamically allocates different transcoding strategies to devices with different transcoding capabilities. At the same time, the transcoding requirements of the key area and the non-key area in the panoramic video are also considered, and different transcoding strategies are allocated to the key area and the non-key area. The present application dynamically adjusts the transcoding parameters in combination with the transcoding capability of the device, the transcoding requirements of the key area and the non-key area of the panoramic video, so that the adjusted transcoding parameters are adapted to the device, thereby improving the smoothness and clarity of the panoramic video when playing in a low version system.

[0049] Further, after determining the transcoding strategy, an adapted transcoding file is generated, and the transcoding file is stored in a sandbox folder. The electronic device 110 can establish a transcoding file index library according to the transcoding file, which is used to record information such as the associated source file path of the transcoding file, the device adaptation type, the generation time, and the access frequency. When it is detected that the same source file is viewed in different devices (such as iPhone SE2 and iPhone 11), the transcoding file corresponding to the device is automatically generated and associated with the index to avoid repeated transcoding. For transcoding files that are not accessed within a certain time, automatic compressed storage is performed (such as retaining the original resolution and code rate of the key area and reducing the resolution and code rate of the non-key area), thereby releasing the sandbox space, and if the transcoding file is accessed again, the resolution and code rate are quickly restored.

[0050] In block 140, the panoramic video is transcoded based on at least the adjusted transcoding parameters to play the transcoded panoramic video on the device.

[0051] In some embodiments, the electronic device 110 can search for a corresponding transcoding file from the transcoding file index library in the sandbox folder according to the file name of the panoramic video, if the transcoding file corresponding to the file name of the panoramic video exists in the index library, transcode the panoramic video by using the searched transcoding file, and obtain the transcoded panoramic video.

[0052] In order to avoid the problem of image distortion of the panoramic video after transcoding, in some embodiments, the electronic device 110 can search for a corresponding transcoding file from the sandbox folder according to the file name of the panoramic video; obtain the stitching parameters of the panoramic video, and write the stitching parameters and the adjusted transcoding parameters into the transcoding file to obtain a new transcoding file; and transcode the panoramic video by using the new transcoding file.

[0053] The electronic device 110 can obtain the stitching parameters of the panoramic video downloaded or obtained by screen recording, for example, the stitching parameters include the overlapping area coordinates of the dual camera, the projection matrix parameters, etc. The stitching parameters and the adjusted transcoding parameters are embedded into the transcoding file to obtain a new transcoding file, and the panoramic video is transcoded by using the new transcoding file, so that the panoramic video can still be played in the panoramic mode on the device with a lower system version, instead of being played in the flat mode.

[0054] The method for playing the panoramic video provided by the present application dynamically adjusts the transcoding parameters based on the transcoding capability of the device and the transcoding requirement of the image of the panoramic video, transcodes the panoramic video by using the dynamically adjusted transcoding parameters to obtain the transcoded panoramic video, and ensures that the high-resolution panoramic video can be smoothly played on the device with a lower system version, and improves the picture definition of the playing.

[0055] Figure 3 A transcoding flowchart 300 of a panoramic video according to some embodiments of the present disclosure is shown. The transcoding flowchart 300 can be executed by the electronic device 110 in the system Figure 1 The electronic device 110 can include an application (App), a system (System), a native module (Native Module), and a multimedia processing framework (FFmpeg).

[0056] As shown in FIG. 3, the transcoding flowchart 300 includes the following steps. Figure 3As shown, the electronic device 110 responds to the panoramic video download operation or the screen recording operation performed by the user terminal 310, and the App obtains device information from the system (System), including system version, device hardware information, user terminal 310 historical data, etc. The system (System) determines whether the system version of the device is within the preset version range according to the device information. If it is within the preset version range, for example, the system version is within the range less than iOS 15.3 version, the native module (NativeModule) saves the panoramic video obtained by downloading or screen recording to the sandbox folder. If it is not within the preset version range, for example, the system version is within the range greater than or equal to iOS 15.3 version, or the system version is Android version, the App sends a request to write to the album, the system returns the permission state, and the native module (NativeModule) saves the panoramic video to the system album.

[0057] The App of the device can receive the entering album operation sent by the user terminal 310. If the system version is within the range greater than or equal to iOS 15.3 version, or the system version is Android version, the native module (NativeModule) reads the system album. If the system version is within the range less than iOS 15.3 version, the native module (NativeModule) reads the sandbox folder, and returns the video list from the system album or the sandbox folder to the user terminal 310. The user terminal 310 sends a select video operation. If the system version is within the range greater than or equal to iOS 15.3 version, or the system version is Android version, the user terminal 310 performs the select video operation, which usually has the functions of regular video playing, sharing, and deleting. If the system version is within the range less than iOS 15.3 version, after the user terminal 310 performs the select video operation, the selected video can be named “videoname.mp4”, and the native module (NativeModule) first checks whether there is a file named “videoname_output.mp4” in the sandbox folder. If there is, “videoname_output.mp4” is taken as an object that can be directly played in the device, and also has the functions of regular video playing, sharing, and deleting. If there is no file named “videoname_output.mp4”, it is prompted that the selected video cannot be played, and the system needs to be upgraded to iOS 16 or above or the panoramic video needs to be transcoded.

[0058] The native module checks whether the transcoding file exists in the sandbox folder. If the transcoding file exists, the native module returns the path of the transcoding file of the App. The user terminal 310 displays a transcoding prompt and a transcoding button. After the system receives a transcoding confirmation operation sent by the user terminal 310, the transcoding process is triggered, or after the system returns the path of the transcoding file of the App, the transcoding process is automatically executed. Specifically, the transcoding process is executed by calling the instruction of the multimedia processing framework (FFmpeg). The transcoding process refers to the embodiments described above, that is, performing semantic segmentation on the selected panoramic video, determining the key area and non-key area of each frame image of the panoramic video, extracting key feature information from the key area, and extracting non-key feature information from the non-key area, taking the key feature information and the non-key feature information as video feature information, processing the device information by using the trained transcoding score model, and outputting the transcoding score of the device. If the transcoding score of the device is greater than the preset score, the device is determined as a strong transcoding device; or if the transcoding score of the device is less than the preset score, the device is determined as a weak transcoding device. If the device is a strong transcoding device, the resolution of the key area is unchanged, the code rate of the key area is increased, and the code rate of the non-key area is unchanged or decreased, and the resolution of the non-key area is decreased. If the device is a weak transcoding device, the resolution and code rate of the key area of the panoramic video are reduced, and the resolution and code rate of the non-key area of the panoramic video are reduced. After the multimedia processing framework (FFmpeg) completes the transcoding process, a file named “videoname_output.mp4” (the file name of the transcoded panoramic video) is generated and returned to the App. The user terminal 310 refreshes the playback state and plays the transcoded panoramic video. The user terminal 310 can perform functions such as sharing or deleting, updating, and the like on the transcoded panoramic video.

[0059] By transcoding the panoramic video, the high-resolution panoramic video can be smoothly played on a device with a lower system version (for example, a device with a system version less than the range of iOS 15.3 version), and the picture clarity of the playback is improved, effectively avoiding the problems of playback lag, blurred picture, and low clarity.

[0060] Figure 4 A schematic structural block diagram of an apparatus 400 for playing a panoramic video according to some embodiments of the present disclosure is shown. The apparatus 400 can be implemented as or included in the electronic device 110. Various modules / components in the apparatus 400 can be implemented by hardware, software, firmware, or any combination thereof.

[0061] As shown, the apparatus 400 includes a device information obtaining module 410 configured to obtain device information in response to a panoramic video downloading operation, the device information including a system version. The apparatus 400 further includes a feature information extracting module 420 configured to save the panoramic video into a sandbox folder and extract video feature information from the panoramic video if the system version is within a preset version range. The apparatus 400 further includes a transcoding parameter adjusting module 430 configured to adjust transcoding parameters based on the device information and the video feature information. The apparatus 400 further includes a video transcoding module 440 configured to transcode the panoramic video based at least on the adjusted transcoding parameters to play the transcoded panoramic video.

[0062] In some embodiments, the feature information extracting module 420 is further configured to obtain the panoramic video stored in the sandbox folder, determine key regions and non-key regions of each frame image of the panoramic video by performing semantic segmentation on the panoramic video; extract key feature information from the key regions and non-key feature information from the non-key regions; and take the key feature information and the non-key feature information as the video feature information.

[0063] In some embodiments, the transcoding parameter adjusting module 430 is further configured to process the device information by using a trained transcoding score model to output a transcoding score of the device; determine the device as a strong transcoding device if the transcoding score of the device is greater than a preset score; or determine the device as a weak transcoding device if the transcoding score of the device is less than the preset score.

[0064] In some embodiments, the transcoding parameter adjusting module 430 is further configured to, if the device is the strong transcoding device, keep the resolution of the key regions unchanged and increase the code rate of the key regions; and keep the code rate of the non-key regions unchanged or decrease the code rate of the non-key regions and decrease the resolution of the non-key regions.

[0065] In some embodiments, the transcoding parameter adjusting module 430 is further configured to, if the device is the weak transcoding device, decrease the resolution and the code rate of the key regions; and decrease the resolution and the code rate of the non-key regions.

[0066] In some embodiments, the video transcoding module 440 is further configured to search for a corresponding transcoding file from the sandbox folder according to a file name of the panoramic video; obtain stitching parameters of the panoramic video, and write the stitching parameters and the adjusted transcoding parameters into the transcoding file to obtain a new transcoding file; and transcode the panoramic video by using the new transcoding file.

[0067] In some embodiments, further including a transcoding triggering module configured to determine an idle state of the device based on the device information; and trigger transcoding processing for the panoramic video if the device is in the idle state.

[0068] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the disclosure can be implemented is shown. It should be understood that Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to implement Figure 1 the electronic device 110.

[0069] As Figure 5 shown, the electronic device 500 is in the form of a general electronic device. Components of the electronic device 500 can include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 can be a real or virtual processor and is capable of executing various processing in accordance with programs stored in the memory 520. In a multi-processing system, multiple processing units execute computer-executable instructions in parallel to improve the processing power of the electronic device 500.

[0070] The electronic device 500 typically includes a plurality of computer storage media. Such media can be any available media that is accessible by the electronic device 500 and includes both volatile and non-volatile media, removable and non-removable media. The memory 520 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 can be a removable or non-removable media and can include a machine readable medium, such as a flash drive, a magnetic disk, or any other medium that can be used to store information and / or data (e.g., training data for training) and that can be accessed by the electronic device 500.

[0071] The electronic device 500 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 5 , a disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk (e.g., a CD-ROM) can be provided. In these instances, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 520 can include a computer program product 525 having one or more program modules configured to carry out the various methods or actions of the various embodiments of the present disclosure.

[0072] The communication unit 540 enables communication through the communication medium with other electronic devices. Additionally, the functionality of the components of the electronic device 500 can be implemented in a single computing cluster or a plurality of computer machines that are capable of communicating over a communication connection. As such, the electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0073] The input device 550 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 560 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 can also communicate with one or more external devices (not shown), such as a storage device, a display device, etc., through the communication unit 540, as necessary, with one or more devices that enable a user to interact with the electronic device 500, or with any device (e.g., a network card, a modem, etc.) that enables the electronic device 500 to communicate with one or more other electronic devices. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0074] According to an example implementation of the present disclosure, a computer readable storage medium is provided having computer executable instructions stored thereon, where the computer executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided that is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the method described above.

[0075] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0076] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0077] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0078] The flow diagrams and the block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions (s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in some cases, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or combinations of hardware and software.

[0079] implementations. Numerous modifications and adaptations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The herein disclosed implementations are meant to be illustrative and not limiting, of the scope or spirit of the implementations. It will be apparent to those reasonably skilled in the art that varying and / or modified implementations can be apparent at ome time to some embodiments. As such, although the subject matter has been described in language specific to structural features, methodological acts, it is to be understood that the subject matter defined herein is not necessarily limited to these described examples. Any and all equivalents arising from a reasonable application of the essentials, spirit, and scope of the following claims should be covered thereby.

Claims

1. A method for playing panoramic video, characterized in that, include: In response to the panoramic video download operation, device information is obtained, including the system version; If the system version is within the preset version range, the panoramic video is saved to the sandbox folder, and video feature information is extracted from the panoramic video; Adjust transcoding parameters based on the device information and the video feature information; as well as The panoramic video is transcoded at least based on the adjusted transcoding parameters in order to play the transcoded panoramic video.

2. The method for playing panoramic video according to claim 1, characterized in that, The step of extracting video feature information from the panoramic video also includes: The panoramic video stored in the sandbox folder is obtained, and the key and non-key regions of each frame of the panoramic video are determined by performing semantic segmentation on the panoramic video. Extracting key feature information from the key regions and extracting non-key feature information from the non-key regions; and The key feature information and the non-key feature information are used as the video feature information.

3. The method for playing panoramic video according to claim 2, characterized in that, The step of adjusting transcoding parameters based on the device information and the video feature information includes: The device information is processed using a trained transcoding scoring model to output the device's transcoding score; If the transcoding score of the device is greater than a preset score, the device is identified as a strong transcoding device; or If the transcoding score of the device is less than the preset score, the device is identified as a weak transcoding device.

4. The method for playing panoramic video according to claim 3, characterized in that, The transcoding parameters include resolution and bitrate, and the step of adjusting the transcoding parameters based on the device information and the video feature information includes: If the device is a strong transcoding device, the resolution of the key region remains unchanged, while the bitrate of the key region is increased; and The bitrate of the non-critical regions remains unchanged or is reduced, and the resolution of the non-critical regions is reduced.

5. The method for playing panoramic video according to claim 3, characterized in that, The transcoding parameters include resolution and bitrate, and the step of adjusting the transcoding parameters based on the device information and the video feature information further includes: If the device is a weak transcoding device, reduce the resolution and bitrate of the critical region; and reduce the resolution and bitrate of the non-critical region.

6. A method for playing panoramic video according to any one of claims 4 to 5, characterized in that, The step of transcoding the panoramic video based at least on the adjusted transcoding parameters includes: Search for the corresponding transcoding file in the sandbox folder based on the filename of the panoramic video; Obtain the stitching parameters of the panoramic video, and write the stitching parameters and the adjusted transcoding parameters into the transcoding file to obtain a new transcoding file; and The panoramic video is transcoded using the new transcoding file.

7. The method for playing panoramic video according to claim 1, characterized in that, Also includes: The idle status of the device is determined based on the device information; as well as If the device is idle, transcoding of the panoramic video is triggered.

8. A panoramic video playback device, characterized in that, include: The device information acquisition module is configured to acquire device information, including the system version, in response to the panoramic video download operation. The feature information extraction module is configured to save the panoramic video to the sandbox folder if the system version is within a preset version range, and extract video feature information from the panoramic video. The transcoding parameter adjustment module is configured to adjust the transcoding parameters based on the device information and the video feature information; as well as The video transcoding module is configured to transcode the panoramic video based at least on the adjusted transcoding parameters in order to play the transcoded panoramic video.

9. An electronic device, characterized in that, include: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the method according to any one of claims 1 to 7.

Images