A method and device for playing panoramic video.

Abstract

This disclosure provides a method and apparatus for playing panoramic videos. The method includes, in response to a panoramic video download operation, acquiring device information, including the system version; if the system version is within a preset version range, saving the panoramic video to a sandbox folder and extracting video feature information from the panoramic video; adjusting transcoding parameters based on the device information and the video feature information; and transcoding the panoramic video based at least on the adjusted transcoding parameters to play the transcoded panoramic video. This ensures that high-resolution panoramic videos can be played smoothly on devices with lower system versions and improves the clarity of the played image.

Description

Technical Field

[0001] This disclosure generally relates to the field of video playback, and particularly to a method for playing panoramic video, a device for playing panoramic video, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the increasing demand for home security, surveillance cameras have become widely used in daily monitoring. Currently, common surveillance camera equipment on the market includes single-camera and dual-camera systems. Dual-camera systems are typically used to meet the needs of different monitoring perspectives. Specifically, dual-camera systems use two high-resolution lenses to capture images and / or videos from different angles, and then seamlessly stitch these images and / or videos together to create a panoramic image and / or panoramic video, thus meeting the needs of users requiring ultra-wide viewing angles. However, the panoramic video generated by dual-camera systems often cannot be played back properly, which has become a pressing problem in the current surveillance field. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for playing panoramic video is provided, comprising: in response to a panoramic video download operation, obtaining device information, the device information including a system version; if the system version is within a preset version range, saving the panoramic video to a sandbox folder and extracting video feature information from the panoramic video; adjusting transcoding parameters based on the device information and the video feature information; and transcoding the panoramic video based at least on the adjusted transcoding parameters to play the transcoded panoramic video.

[0004] In a second aspect of this disclosure, a panoramic video playback device is provided, comprising: a device information acquisition module configured to acquire device information, including a system version, in response to a panoramic video download operation; a feature information extraction module configured to save the panoramic video to a sandbox folder and extract video feature information from the panoramic video if the system version is within a preset version range; a transcoding parameter adjustment module configured to adjust transcoding parameters based on the device information and the video feature information; and a video transcoding module configured to transcode the panoramic video based at least on the adjusted transcoding parameters to play the transcoded panoramic video.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0010] Figure 2 A flowchart illustrating a method for playing panoramic video according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram illustrating the transcoding process of panoramic video according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A block diagram of a panoramic video playback device according to some embodiments of the present disclosure is shown; and

[0013] Figure 5 A block diagram of an apparatus capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0016] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0018] For ease of description, several concepts and terms are first defined here.

[0019] Video transcoding refers to converting a compressed and encoded video stream into another video stream to adapt to different network bandwidths, the processing capabilities of different electronic devices (terminals), and different user needs. Transcoding is essentially a process of first transcoding and then encoding.

[0020] React Native is an open-source framework based on React that allows developers to write native (iOS / Android) mobile applications using JavaScript. It achieves a near-native mobile application experience with front-end development efficiency.

[0021] Native Module refers to the core mechanism in the React Native framework that connects JavaScript with native platform (iOS / Android) code. It is a communication bridge between JavaScript and the native system, retaining the cross-platform advantages of JS while utilizing all the capabilities of the native system.

[0022] FFmpeg is an open-source, cross-platform audio and video processing framework that provides multimedia processing capabilities such as recording, format conversion, encoding and transcoding, and streaming media transmission.

[0023] As briefly mentioned earlier, panoramic videos captured by dual cameras typically have higher resolutions, such as 4608*1296, and of course, even higher resolutions are possible. Currently, traditional devices (such as surveillance cameras) often have older operating system versions, such as iOS 15.3 or lower. On these devices, panoramic videos may experience stuttering or even fail to play, and they cannot be shared with other devices or third-party software.

[0024] Therefore, embodiments of this disclosure propose a method for playing panoramic videos. According to various embodiments of this disclosure, in response to a panoramic video download operation, device information, including the system version, is obtained. Further, if the system version is within a preset version range, the panoramic video is saved to a sandbox folder, and video feature information is extracted from the panoramic video; transcoding parameters are adjusted based on the device information and the video feature information; and the panoramic video is transcoded at least based on the adjusted transcoding parameters to play the transcoded panoramic video. In this way, panoramic videos can be played normally and smoothly on electronic devices with lower system versions.

[0025] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0026] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In environment 100, generally speaking, electronic device 110 transcodes panoramic video 120 based on device information 130 to obtain transcoded panoramic video 140, and plays the transcoded panoramic video 140 on the device. Panoramic video 120 refers to panoramic video captured by a device with dual cameras. For example, the resolution of the panoramic video can reach 4608*1296, and it can also be other higher or lower resolution videos. The captured panoramic video 120 can be stored in a software platform or hardware platform such as the cloud or an SD memory card.

[0027] Electronic device 110 can be any device with computing capabilities. In some embodiments, at least a portion of electronic device 110 can be a server, edge node, computing device in a cloud environment, etc. For example, electronic device 110 can be a server controlled and coordinated by a cloud operating system in a cloud environment, providing data processing and other operations through virtual machines or similar means. In some embodiments, electronic device 110 can be a terminal device. Terminal devices can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, audio / video players, digital cameras / camcorders, positioning devices, television receivers, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or combinations thereof. In some embodiments, terminal devices can also support any type of user-facing interface (such as "wearable" circuitry). Electronic device 110 can be a device with playback functionality, or it can be integrated into a device with playback functionality and be able to interact with the device with playback functionality, such as sending or receiving data.

[0028] The following describes in further detail an example process of transcoding panoramic video based on device information according to some embodiments of the present disclosure to obtain a transcoded panoramic video, and then playing the transcoded panoramic video. Figure 2 A flowchart 200 illustrating a method for playing panoramic video according to some embodiments of the present disclosure is shown. This process 200 can be performed by… Figure 1 The electronic device 110 shown is performing this action.

[0029] like Figure 2 As shown, in some embodiments, in block 210, in response to a panoramic video download operation, device information is obtained, including the system version.

[0030] For example, in response to a user's request to download a panoramic video, electronic device 110 can download the panoramic video from the cloud or an SD memory card via an app (application) installed on the device, such as a camera app or other type of application. Alternatively, electronic device 110 can also respond to a user's request to record a screen, indicating that the screen is being recorded during the playback of a live stream, and that the recording ends within a predetermined time period to generate a panoramic video.

[0031] Furthermore, when the panoramic video is successfully downloaded or generated through screen recording, the electronic device 110 can obtain device information in any suitable manner, including the system version. For example, the system version installed on the device (e.g., a playback device) can include various versions of Android, iOS, HarmonyOS, etc. Taking various versions of iOS as an example, these can include iOS 15.3, versions older than iOS 15.3 such as iOS 16, iOS 18.3, and versions younger than iOS 15.3 such as iOS 14. It should be understood that the system version can be any version, and this disclosure does not impose specific limitations in this regard.

[0032] In box 220, if the system version is within the preset version range, the panoramic video is saved to the sandbox folder, and video feature information is extracted from the panoramic video.

[0033] In some embodiments, if the system version is within a preset version range, the electronic device 110 can save the panoramic video to a sandbox folder. For example, if the preset version is iOS 15.3, and the electronic device 110 determines from its device information that the current system version is below iOS 15.3, then it saves the downloaded and / or screen-recorded panoramic video to a sandbox folder, such as a sandbox folder named after the camera's device ID. This facilitates subsequent transcoding of the panoramic videos stored in the sandbox folder, the transcoding process of which will be described in detail below. In other words, the panoramic videos stored in the sandbox folder are all panoramic videos that are stuttering or not supported for playback on the electronic device 110. It should be understood that the preset version range can be below iOS 15.3, or it can be any other preset version range that causes stuttering or does not support playback of panoramic videos; this disclosure does not impose specific limitations in this regard.

[0034] Accordingly, if the electronic device 110 determines from the device information that the system version is Android, or a version higher than iOS 15.3, the downloaded and / or screen-recorded panoramic video can be stored in the electronic device 110's system album, such as a phone album named after the camera's device ID. The panoramic video stored in the system album does not require transcoding, making it easy for users to find the panoramic video in the system album and play it on the electronic device 110 or share it with other devices.

[0035] In some embodiments, the idle state of the device is determined based on device information; and if the device is in an idle state, transcoding processing for the panoramic video is triggered. For example, the electronic device 110 can also extract the current device state from the device information, and automatically trigger the transcoding process if it detects that the device is currently in an idle state.

[0036] The following section will introduce the overall transcoding process.

[0037] In some embodiments, a panoramic video stored in a sandbox folder is acquired, and key and non-key regions of each frame of the panoramic video are determined by performing semantic segmentation on the panoramic video; key feature information is extracted from the key regions, and non-key feature information is extracted from the non-key regions; and the key and non-key feature information are used as video feature information.

[0038] Electronic device 110 can acquire panoramic video stored in a sandbox folder and perform semantic segmentation on the panoramic video using a semantic segmentation algorithm. For example, it can perform semantic segmentation on all frames of the panoramic video to obtain semantic segmentation results. Electronic device 110 can determine the key and non-key regions of each frame of the panoramic video based on the semantic segmentation results. As an example, electronic device 110 is a surveillance camera device that captures panoramic video of a monitored scene. The panoramic video includes multiple frames of surveillance footage. The key region of each frame may include, for example, a specific area located in the center of the frame, while the non-key region may include the edge areas of the surveillance footage. Alternatively, the key region of each frame may include the area where one or more specific monitored objects (such as people, doors, windows, etc.) are located, while the non-key region may include areas where no monitored object is located (such as the sky, walls, etc.). It should be understood that the key and non-key regions can be adaptively adjusted, and this disclosure does not impose specific limitations in this regard.

[0039] Electronic device 110 can extract key feature information from key areas and non-key feature information from non-key areas, and use both key and non-key feature information as video feature information. For example, it can extract feature information of important objects under surveillance, such as people, doors and windows, from key areas as key feature information, and extract feature information of non-monitored objects, such as the sky and walls, from non-key areas as non-key feature information.

[0040] Existing technologies use a uniform transcoding strategy to transcode the entire video image, resulting in stuttering and blurring issues during playback of the transcoded panoramic video. This invention combines semantic analysis with transcoding processing of panoramic video, segmenting the image into critical and non-critical regions. Different transcoding strategies are assigned to these regions, and transcoding is performed separately. This improves the image clarity of the critical regions after transcoding, increases transcoding efficiency, and effectively avoids stuttering and blurring issues during panoramic video playback.

[0041] The following section describes the specific process of transcoding critical and non-critical regions using different transcoding strategies.

[0042] In box 230, transcoding parameters are adjusted based on device information and video feature information. Electronic device 110 dynamically adjusts transcoding parameters according to device information and video feature information, so that the transcoding parameters are adapted to the device's transcoding capabilities and the video features of the panoramic video, ensuring that the transcoded panoramic video can be played smoothly and clearly on the device.

[0043] Electronic device 110 acquires device information, including hardware information such as CPU model, GPU type, and memory size. Based on this device information, a transcoding scoring model is established. This model is then trained using device information from various types of devices to obtain a trained transcoding scoring model, which is used to score the transcoding capabilities of the devices.

[0044] In some embodiments, a trained transcoding scoring model is used to process device information and output a transcoding score for the device; if the transcoding score of the device is greater than a preset score, the device is identified as a strong transcoding device; or if the transcoding score of the device is less than a preset score, the device is identified as a weak transcoding device.

[0045] The device information indicates that the device's system version falls within a preset version range, preventing the device from playing high-resolution panoramic video correctly. Electronic device 110 can use a trained transcoding scoring model to process device information within the preset version range and predict the device's transcoding score. For example, if the device is a mobile phone of various models, such as an iPhone SE2, the predicted score is 60 points, while an iPhone 11's predicted transcoding score is 80 points.

[0046] For example, the preset score can be adaptively adjusted. For instance, the preset score could be 70 points. It should be understood that the preset score could also be any other score, such as 80, 90 points, etc., and this disclosure does not impose specific limitations in this regard. After scoring the transcoding capability of the device using the trained transcoding scoring model, the device is classified into two types: "weak transcoding device" and "strong transcoding device" using the preset score. For example, if the device's transcoding score is greater than or equal to the preset score of 70 points, the device is determined to be a strong transcoding device; if the device's transcoding score is less than the preset score of 70 points, the device is determined to be a weak transcoding device. The electronic device 110 can assign different transcoding strategies to devices with different transcoding capabilities, namely strong transcoding devices and weak transcoding devices.

[0047] In some embodiments, if the device is a strong transcoding device, the resolution of the critical region remains unchanged while the bitrate of the critical region is increased; and the bitrate of the non-critical region remains unchanged while the resolution of the non-critical region is decreased.

[0048] Electronic device 110 acquires transcoding parameters, including resolution and bitrate. Resolution refers to the number and density of pixels in an image or video, while bitrate refers to the number of bits transmitted per second. At the same resolution, a higher bitrate results in clearer image quality, but it must match the terminal's transcoding capabilities. After acquiring the transcoding parameters, it determines whether the device is a strong transcoding device. If the device is a strong transcoding device, electronic device 110 assigns a transcoding strategy to key areas in the panoramic video that maintains the resolution of the key areas while increasing the bitrate of the key areas, and assigns a transcoding strategy to non-key areas that maintains the bitrate while decreasing the resolution of the non-key areas.

[0049] As an example, if the device is a strong transcoding device (such as iPhone 11), the electronic device 110 assigns a transcoding strategy to key areas in the panoramic video, which keeps the resolution at 1920*1080 and reduces the bitrate from 8Mbps to 5Mbps. For non-critical areas in the panoramic video, the electronic device 110 assigns a transcoding strategy to reduce the resolution from 2688*1296 to 1280*720 and reduce the bitrate from 6Mbps to 2Mbps or keep it at 6Mbps.

[0050] In some embodiments, if the device is a weak transcoding device, the resolution and bitrate of critical regions are reduced; and the resolution and bitrate of non-critical regions are reduced.

[0051] After obtaining the transcoding parameters, it is determined whether the device is a weak transcoding device. If the device is a weak transcoding device, a transcoding strategy that reduces resolution and bitrate is applied to both critical and non-critical areas in the panoramic video. As an example, if the device is a weak transcoding device (such as an iPhone SE2), the electronic device 110 applies a transcoding strategy that reduces the resolution to 1280*720 and the bitrate to 3Mbps for critical areas in the panoramic video; the electronic device 110 applies a transcoding strategy that reduces the resolution to 960*540 and the bitrate to 1.5Mbps for non-critical areas in the panoramic video.

[0052] In existing technologies, transcoding parameters (such as bitrate and resolution) are usually fixed values, failing to consider hardware differences between devices (e.g., different devices have different transcoding capabilities). Videos transcoded using fixed parameters still experience stuttering during playback. This invention utilizes a trained transcoding scoring model to predict a device's transcoding capabilities, obtaining a transcoding score. By comparing this score with a preset score, it determines whether the device is a strong or weak transcoding device, dynamically assigning different transcoding strategies to devices with varying capabilities. Furthermore, it considers the transcoding requirements of critical and non-critical areas in panoramic videos, allocating different transcoding strategies to these areas. This invention dynamically adjusts transcoding parameters based on the device's transcoding capabilities and the transcoding requirements of critical and non-critical areas in the panoramic video, ensuring the adjusted parameters are adapted to the device. This improves the smoothness and clarity of panoramic video playback on older systems.

[0053] Furthermore, after determining the transcoding strategy, an adapted transcoding file is generated and stored in the sandbox folder. The electronic device 110 can build a transcoding file index based on the transcoding file, recording information such as the associated source file path, device adaptation type, generation time, and access frequency. When the same source file is detected being viewed on different devices (e.g., iPhone SE2 and iPhone 11), the corresponding transcoding file for that device is automatically generated and associated with the index, avoiding duplicate transcoding. Transcoding files not accessed within a certain period are automatically compressed (e.g., retaining the original resolution and bitrate of critical areas while reducing the resolution and bitrate of non-critical areas), thereby freeing up sandbox space. If the transcoding file is accessed again, it is quickly restored to the adapted resolution and bitrate.

[0054] In box 140, the panoramic video is transcoded at least based on the adjusted transcoding parameters in order to play the transcoded panoramic video on the device.

[0055] In some embodiments, the electronic device 110 can search for a corresponding transcoding file in the transcoding file index library in the sandbox folder according to the file name of the panoramic video. If a transcoding file corresponding to the file name of the panoramic video exists in the index library, the panoramic video is transcoded using the searched transcoding file to obtain the transcoded panoramic video.

[0056] To avoid image distortion after transcoding of panoramic video, in some embodiments, the electronic device 110 can search for the corresponding transcoding file in the sandbox folder based on the filename of the panoramic video; obtain the stitching parameters of the panoramic video, and write the stitching parameters and the adjusted transcoding parameters into the transcoding file to obtain a new transcoding file; and use the new transcoding file to transcode the panoramic video.

[0057] Electronic device 110 can acquire stitching parameters from downloaded or screen-recorded panoramic videos, such as the coordinates of the overlapping area of ​​the dual cameras and projection matrix parameters. It embeds the stitching parameters and adjusted transcoding parameters into a transcoding file to obtain a new transcoding file. This new transcoding file is then used to transcode the panoramic video, ensuring that the panoramic video can still be played in panoramic mode on devices with lower system versions, rather than in planar mode.

[0058] The panoramic video playback method provided by this invention dynamically adjusts the transcoding parameters based on the device's transcoding capabilities and the transcoding requirements of the panoramic video images. The panoramic video is then transcoded using the dynamically adjusted transcoding parameters to obtain the transcoded panoramic video. This ensures that the high-resolution panoramic video can be played smoothly on devices with lower system versions and improves the clarity of the played image.

[0059] Figure 3 A schematic diagram 300 of a panoramic video transcoding process according to some embodiments of the present disclosure is shown. The transcoding process 300 can be performed by… Figure 1 The electronic device 110 executes the commands. The electronic device 110 may include applications (App), systems, native modules, and multimedia processing frameworks (FFmpeg).

[0060] like Figure 3As shown, in response to the panoramic video download or screen recording operation performed by the user terminal 310, the app obtains device information from the system, including system version, device hardware information, and historical data of the user terminal 310. The system determines whether the device's system version is within a preset version range based on the device information. If it is, for example, if the system version is lower than iOS 15.3, the native module saves the downloaded or recorded panoramic video to the sandbox folder. If it is not within the preset version range, for example, if the system version is greater than or equal to iOS 15.3, or if the system version is Android, the app sends a request to write to the photo album. The system returns a permission status, and the native module saves the panoramic video to the system photo album.

[0061] The device's app can receive a request from user 310 to access the photo album. If the system version is iOS 15.3 or higher, or if the system version is Android, the native module reads the system photo album. If the system version is lower than iOS 15.3, the native module reads the sandbox folder and returns a list of videos from either the system photo album or the sandbox folder to user 310. When user 310 sends a request to select a video, if the system version is iOS 15.3 or higher, or if the system version is Android, user 310 executes the video selection operation, which typically includes standard video playback, sharing, and deletion functions. If the system version is lower than iOS 15.3, after the user selects a video (using device 310), the selected video can be named "videoname.mp4". The native module first checks if a file named "videoname_output.mp4" exists in the sandbox folder. If it exists, "videoname_output.mp4" is used as a playable file on the device, and it has the same regular video playback, sharing, and deletion functions. If a file named "videoname_output.mp4" does not exist, a message will appear indicating that the selected video cannot be played, and the user should upgrade to iOS 16 or later or transcode the panoramic video.

[0062] The native module checks if a transcoding file exists in the sandbox folder. If it does, it returns the path to the transcoding file in the App. The client 310 displays a transcoding prompt and a transcoding button. The system triggers the transcoding process after receiving a transcoding confirmation from the client 310, or automatically executes the transcoding process after returning the path to the transcoding file in the App. Specifically, it calls instructions from the multimedia processing framework (FFmpeg) to execute the transcoding process. Referring to the example above, the transcoding process involves semantic segmentation of the selected panoramic video, determining the key and non-key regions of each frame, extracting key feature information from the key regions, and extracting non-key feature information from the non-key regions. These key and non-key feature information are used as video feature information. A trained transcoding scoring model is then used to process the device information, outputting the device's transcoding score. If the device's transcoding score is greater than a preset score, the device is identified as a strong transcoding device; otherwise, if the device's transcoding score is less than the preset score, the device is identified as a weak transcoding device. If the device is a strong transcoding device, the resolution of critical areas remains unchanged, while the bitrate of critical areas is increased; the bitrate of non-critical areas remains unchanged or is reduced, while the resolution of non-critical areas is reduced. If the device is a weak transcoding device, the resolution and bitrate of critical areas of the panoramic video are reduced; the resolution and bitrate of non-critical areas of the panoramic video are also reduced. After the multimedia processing framework (FFmpeg) completes the transcoding process, it generates a file named "videoname_output.mp4" (the filename of the transcoded panoramic video) and returns it to the App. The client 310 refreshes the playback status and plays the transcoded panoramic video. The client 310 can perform functions such as sharing, deleting, and updating the transcoded panoramic video.

[0063] By transcoding panoramic videos, high-resolution panoramic videos can be played smoothly on devices with lower system versions (such as those with system versions lower than iOS 15.3), and the clarity of the playback image is improved, effectively avoiding problems such as playback stuttering, blurry images, and low clarity.

[0064] Figure 4 A schematic structural block diagram of an apparatus 400 for playing panoramic video according to certain embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in an electronic device 110. The various modules / components in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0065] As shown in the figure, device 400 includes a device information acquisition module 410, configured to acquire device information, including the system version, in response to a panoramic video download operation. Device 400 also includes a feature information extraction module 420, configured to save the panoramic video to a sandbox folder and extract video feature information from the panoramic video if the system version is within a preset version range. Device 400 further includes a transcoding parameter adjustment module 430, configured to adjust transcoding parameters based on the device information and video feature information. Device 400 also includes a video transcoding module 440, configured to transcode the panoramic video based at least on the adjusted transcoding parameters to play the transcoded panoramic video.

[0066] In some embodiments, the feature information extraction module 420 is further configured to acquire panoramic video stored in a sandbox folder, determine key and non-key regions of each frame of the panoramic video by performing semantic segmentation on the panoramic video, extract key feature information from the key regions, and extract non-key feature information from the non-key regions, and use the key and non-key feature information as video feature information.

[0067] In some embodiments, the transcoding parameter adjustment module 430 is further configured to process device information using a trained transcoding scoring model and output a transcoding score for the device; if the transcoding score of the device is greater than a preset score, the device is identified as a strong transcoding device; or if the transcoding score of the device is less than a preset score, the device is identified as a weak transcoding device.

[0068] In some embodiments, the transcoding parameter adjustment module 430 is further configured to, if the device is a strong transcoding device, keep the resolution of the critical region unchanged and increase the bitrate of the critical region; and keep the bitrate of the non-critical region unchanged or decrease the resolution of the non-critical region.

[0069] In some embodiments, the transcoding parameter adjustment module 430 is further configured to include the transcoding parameters as resolution and bitrate, and to reduce the resolution and bitrate of critical areas if the device is a weak transcoding device; and to reduce the resolution and bitrate of non-critical areas.

[0070] In some embodiments, the video transcoding module 440 is further configured to search for the corresponding transcoding file in the sandbox folder based on the filename of the panoramic video; obtain the stitching parameters of the panoramic video, and write the stitching parameters and the adjusted transcoding parameters into the transcoding file to obtain a new transcoding file; and use the new transcoding file to transcode the panoramic video.

[0071] In some embodiments, a transcoding trigger module is further included, configured to determine the idle state of the device based on device information; and if the device is in an idle state, to trigger transcoding processing for the panoramic video.

[0072] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to achieve Figure 1 Electronic devices 110.

[0073] like Figure 5 As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0074] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 500.

[0075] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0076] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0077] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0078] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0079] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0080] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0081] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0083] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method of playing a panoramic video, characterized by, The method comprises: in response to a panoramic video download operation, obtaining device information, the device information comprising a system version; if the system version is within a preset version range, saving the panoramic video into a sandbox folder, and extracting video feature information from the panoramic video, the step of extracting the video feature information from the panoramic video comprising: determining key areas and non-key areas of each frame image of the panoramic video from the panoramic video; adjusting transcoding parameters based on the device information and the video feature information, the transcoding parameters comprising resolution and code rate; the step of adjusting the transcoding parameters based on the device information and the video feature information comprising: if the device is a strong transcoding device, the resolution of the key areas remains unchanged, the code rate of the key areas is increased, the code rate of the non-key areas remains unchanged or is decreased, and the resolution of the non-key areas is decreased; or if the device is a weak transcoding device, the resolution and code rate of the key areas are decreased, and the resolution and code rate of the non-key areas are decreased; and transcoding the panoramic video based at least on the adjusted transcoding parameters to play the transcoded panoramic video.

2. The method of claim 1, wherein, The step of extracting the video feature information from the panoramic video further comprises: obtaining the panoramic video stored in the sandbox folder, determining the key areas and the non-key areas by performing semantic segmentation on the panoramic video; extracting key feature information from the key areas and non-key feature information from the non-key areas; and taking the key feature information and the non-key feature information as the video feature information.

3. The method of claim 2, wherein, The step of adjusting the transcoding parameters based on the device information and the video feature information comprises: processing the device information using a trained transcoding score model to output a transcoding score of the device; if the transcoding score of the device is greater than a preset score, determining the device as a strong transcoding device; or if the transcoding score of the device is less than the preset score, determining the device as a weak transcoding device.

4. The method of claim 1, wherein, The step of transcoding the panoramic video based at least on the adjusted transcoding parameters comprises: searching for a corresponding transcoding file from the sandbox folder according to the file name of the panoramic video; obtaining stitching parameters of the panoramic video, and writing the stitching parameters and the adjusted transcoding parameters into the transcoding file to obtain a new transcoding file; and transcoding the panoramic video using the new transcoding file.

5. The method of claim 1, wherein, The method further comprises: determining an idle state of the device based on the device information; and if the device is in the idle state, triggering transcoding processing for the panoramic video. The method comprises:

6. An apparatus for playing a panoramic video, the apparatus comprising: a device information obtaining module configured to, in response to a panoramic video download operation, obtain device information, the device information comprising a system version; ​ The characteristic information extraction module is configured to save the panoramic video into a sandbox folder and extract video characteristic information from the panoramic video if the system version is within a preset version range. The step of extracting video characteristic information from the panoramic video includes determining key areas and non-key areas of each frame image of the panoramic video from the panoramic video. The transcoding parameter adjustment module is configured to adjust transcoding parameters based on the device information and the video characteristic information. The transcoding parameters include resolution and code rate. The step of adjusting transcoding parameters based on the device information and the video characteristic information includes: if the device is a strong transcoding device, the resolution of the key areas remains unchanged, the code rate of the key areas is increased, the code rate of the non-key areas remains unchanged or is reduced, and the resolution of the non-key areas is reduced; or if the device is a weak transcoding device, the resolution and code rate of the key areas are reduced, and the resolution and code rate of the non-key areas are reduced. The video transcoding module is configured to transcode the panoramic video based on at least the adjusted transcoding parameters to play the transcoded panoramic video.

7. An electronic device, comprising: comprise: at least one processing unit; and at least one memory that is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, a computer program is stored thereon, which is executable by a processor to implement the method according to any one of claims 1 to 5.

Images