Cross-device playback method and electronic device

Abstract

The present application provides a cross-device playback method and an electronic device. The cross-device playback method comprises: a first device sends a query request to a second device, wherein the query request is used for requesting a cross-device playback service; the first device receives a query response from the second device, wherein the query response is used for indicating the second capability information of the second device, and the second capability information comprises at least one of the following items: the decoding capability information of the second device, the multi-link playback capability information of the second device, the digital rights management (DRM) capability information of the second device, and the high dynamic range (HDR) rendering technology capability information of the second device. The technical solution of the embodiments of the present application can support negotiation of a device capability during cross-device playback, thereby providing better playback experience for users.

Description

Cross-device playback method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on November 22, 2023, with application number 202311567165.2 and application name “Cross-device playback method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communications, and more specifically, to a cross-device playback method and electronic device. Background Art

[0003] With the development of internet-connected devices, users' demand for interconnection and sharing between different devices is also increasing. Cross-device playback technology enables media files on one device to be played on another. For example, users can play videos, audio, images, and other media files currently playing on their mobile phone on a smart TV, providing a large-screen viewing experience.

[0004] The Digital Living Network Alliance (DLNA) is a set of protocols widely used for interconnection between different electronic devices, supporting cross-device playback of media resources.

[0005] However, when using DLNA for cross-device playback, since DLNA only defines basic media resource file formats, it cannot meet the needs of different devices and manufacturers.

[0006] Summary of the Invention

[0007] The present application provides a cross-device playback method and electronic device, which can support negotiation of device capabilities during cross-device playback, thereby providing users with a better playback experience.

[0008] In a first aspect, a cross-device playback method is provided for a first device, comprising: sending a query request, the query request being used to request a cross-device playback service; receiving a query response from a second device, the query response being used to indicate first capability information of the second device, the first capability information comprising at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management (DRM) capability information of the second device, and high dynamic range rendering technology (HDR) capability information of the second device.

[0009] Decoding capability information can be represented by a string or an integer. The integer can be in binary, octal, hexadecimal, etc., and the number of bits can be set as required. Different integers can be mapped to different decoding types. Multi-link playback capability information can be represented by an integer, indicating whether the device supports multi-link playback, or can further represent other multi-link playback capability information. DRM capability information can include DRM type and security level, and can be represented by a string or a multi-bit integer. HDR capability can be represented by a string or an integer.

[0010] In this embodiment, since the query response sent by the second device includes the first capability information of the second device, the first device can obtain the capability of the second device, and the capability can be negotiated when the first device plays later, so that the user can obtain a better playback experience.

[0011] In combination with the first aspect, in some implementations of the first aspect, the method further includes: sending a resource message to the second device based on the first capability information, wherein the resource message is used to indicate the media resources of the cross-device playback service, and the media resources are associated with the first capability information.

[0012] Since the resource message sent by the first device is sent based on the first capability information of the second device, negotiation of media resources for cross-device playback is achieved. The second device can obtain the media resources associated with the first capability information of the second device, thereby enhancing the user experience.

[0013] In combination with the first aspect, in some implementations of the first aspect, the first capability information includes at least one of the following: screen resolution information of the second device, color gamut information of the second device, and sound effect information of the second device.

[0014] Resolution information, color gamut information, and sound effect information can all be expressed in string or integer form.

[0015] In combination with the first aspect, in certain implementations of the first aspect, the query response is used to indicate second capability information of the second device, and the second capability information is used to indicate at least one of the following features of the second device: whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

[0016] Each item of the second capability information may be directly represented by an integer 1 or 0 to indicate whether it has the corresponding capability, or may be represented by a character string.

[0017] In combination with the first aspect, in some implementations of the first aspect, the query response includes a second capability information field, where the second capability information field is used to indicate the second capability information.

[0018] In combination with the first aspect, in some implementations of the first aspect, the length of the second capability information field is 32 bits, and each item of the second capability information is indicated by one bit of the second capability information field.

[0019] The length of the second capability information field may also be 8 bits, 16 bits, 64 bits, etc., and an appropriate length may be set for the second capability information field according to the actual number of second capability information items required.

[0020] Specifically, whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution are each represented by one bit. When the second capability information also includes other feature items, it will also be represented by one bit.

[0021] Optionally, whether the screen of the second device supports the first resolution and whether the second device has the ability to play and render media resources at the first resolution can be combined into one bit to indicate, and only when both are met, the value of the bit is 1. Optionally, any two capability items in the second capability information can be combined into one bit to indicate, which is not limited in this application.

[0022] In combination with the first aspect, in some implementations of the first aspect, the first resolution is any one of 2K, 4K or 8K.

[0023] The first resolution may also be other values, which is not limited in this application.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the query response is used to indicate the device type of the second device, and the device type of the second device includes any one of the following: a mobile phone, a TV, a tablet computer, a computer, a screen projector, a projector, an electronic whiteboard, a display, a smart cockpit, a speaker, a network box, and a set-top box.

[0025] The device type can be represented by a string or an integer. Different integers represent different device types. Categorizing devices allows more information to be obtained through capability negotiation, which helps provide users with a better cross-device playback experience.

[0026] In combination with the first aspect, in certain implementations of the first aspect, the cross-device playback service is used for audio or video playback, the first capability information includes decoding capability information of the second device, and the media resources for cross-device playback are encoded using the type with the highest compression rate among the decoding types supported by the second device.

[0027] Alternatively, the resource with the lowest loss rate can be played based on the decoding capability of the second device, thereby improving the playback experience. Alternatively, the resource with the encoding method with the appropriate compression rate can be selected based on the current network conditions.

[0028] In combination with the first aspect, in certain implementations of the first aspect, the cross-device playback service is used for audio or video playback, the query response is used to indicate that the second device has multi-link playback capability, and at least one resource link among the multiple resource links indicates a media resource for cross-device playback.

[0029] When the device supports multi-link playback, multiple resource links are sent to the second device so that the second device can obtain multiple resources and play them continuously, reducing the overhead of video splicing and merging on the server side.

[0030] In combination with the first aspect, in some implementations of the first aspect, at least one resource link among the multiple resource connections indicates an advertising resource.

[0031] In combination with the first aspect, in some implementations of the first aspect, the protocol used by the first device and the second device for communication is a multicast domain name system (mDNS) protocol.

[0032] In combination with the first aspect, in certain implementations of the first aspect, the first capability information includes at least one of the refresh rate information of the second device, the device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device, and the device subtype of the second device is related to the manufacturer or brand of the second device.

[0033] In combination with the first aspect, in certain implementations of the first aspect, the query response is used to indicate the device identification of the second device, and the device identification includes any one of the following: the Internet Protocol IP address of the second device, the Media Access Control MAC address of the second device, the Universally Unique Identifier UUID of the second device, and the serial number of the second device.

[0034] In combination with the first aspect, in some implementations of the first aspect, the query response includes a first capability information field, where the first capability information field is used to indicate the first capability information, and the length of the first capability information field is less than or equal to 400 bytes.

[0035] The length of the optional first capability information field can be 100 bytes, 200 bytes, 300 bytes, 400 bytes, etc.

[0036] In combination with the first aspect, in some implementations of the first aspect, the query response is used to indicate the device name, service identifier, and port configuration of the second device.

[0037] The device name of the second device may be displayed in the cross-device playlist of the first device, and the name may be user-set or default, such as “TV in the living room”.

[0038] The service identifier indicates the cross-device playback service requested by the first device.

[0039] In a second aspect, a cross-device playback method is provided for a second device, including: receiving a query request, the query request being used to request a cross-device playback service; sending a query response, the query response being used to indicate first capability information of the second device, the first capability information including at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management capability information of the second device, and high dynamic lighting rendering technology capability information of the second device.

[0040] In combination with the second aspect, in some implementations of the second aspect, the method further includes: receiving a resource message, where the resource message is used to indicate a media resource of the cross-device playback service, and the media resource is associated with the first capability information.

[0041] In combination with the second aspect, in some implementations of the second aspect, the first capability information includes at least one of the following: screen resolution information of the second device, color gamut information of the second device, and sound effect information of the second device.

[0042] In combination with the second aspect, in certain implementations of the second aspect, the query response is used to indicate second capability information of the second device, and the second capability information is used to indicate at least one of the following features of the second device: whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

[0043] In combination with the second aspect, in some implementations of the second aspect, the query response includes a second capability information field, where the second capability information field is used to indicate the second capability information.

[0044] In combination with the second aspect, in some implementations of the second aspect, the length of the second capability information field is 32 bits, and each feature of the second capability information is indicated by one bit of the second capability information field.

[0045] In combination with the second aspect, in some implementations of the second aspect, the first resolution is any one of 2K, 4K or 8K.

[0046] In combination with the second aspect, in certain implementations of the second aspect, the query response is used to indicate the device type of the second device, and the device type of the second device includes any one of the following: a mobile phone, a TV, a tablet computer, a computer, a screen projector, a projector, an electronic whiteboard, a display, a smart cockpit, a speaker, a network box, and a set-top box.

[0047] In combination with the second aspect, in certain implementations of the second aspect, the cross-device playback service is used for audio or video playback, the first capability information includes decoding capability information of the second device, and the media resources for cross-device playback are encoded using the type with the highest compression rate among the decoding types supported by the second device.

[0048] In combination with the second aspect, in certain implementations of the second aspect, the cross-device playback service is used for audio or video playback, the query response is used to indicate that the second device has multi-link playback capability, and at least one resource link among the multiple resource links indicates a media resource for cross-device playback.

[0049] In combination with the second aspect, in some implementations of the second aspect, at least one resource link among the multiple resource connections indicates an advertising resource.

[0050] In combination with the second aspect, in some implementations of the second aspect, the protocol used by the second device for communication is the multicast domain name system (mDNS) protocol.

[0051] In combination with the second aspect, in certain implementations of the second aspect, the first capability information includes at least one of the refresh rate information of the second device, the device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device, and the device subtype of the second device is related to the manufacturer or brand of the second device.

[0052] In combination with the second aspect, in certain implementations of the second aspect, the query response is used to indicate the device identification of the second device, and the device identification includes any one of the following: the Internet Protocol IP address of the second device, the Media Access Control MAC address of the second device, the Universal Unique Identifier UUID of the second device, and the serial number of the second device.

[0053] In combination with the second aspect, in some implementations of the second aspect, the query response includes a first capability information field, where the first capability information field is used to indicate the first capability information, and the length of the first capability information field is less than 400 bytes.

[0054] In combination with the second aspect, in some implementations of the second aspect, the query response includes a device name, a service identifier, and a port configuration of the second device.

[0055] In a third aspect, an electronic device is provided for executing the method of the first aspect or any one of the implementations of the first aspect or the method of the second aspect or any one of the implementations of the second aspect.

[0056] In a fourth aspect, an electronic device is provided, comprising: one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs comprising instructions, which, when executed by the one or more processors, enable the electronic device to execute a method as in the first aspect or any one of the implementations of the first aspect, or a method as in the second aspect or any one of the implementations of the second aspect.

[0057] In a fifth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes a method as in the first aspect or any one of the implementations of the first aspect, or a method as in the second aspect or any one of the implementations of the second aspect.

[0058] In a sixth aspect, a computer program product is provided. When the computer program product runs on a computer, the computer is caused to execute the method described in the first aspect or any one of the implementations of the first aspect or the method described in the second aspect or any one of the implementations of the second aspect.

[0059] In the seventh aspect, a cross-device playback system is provided, comprising a first device and a second device, wherein the first device is connected to the second device for screen projection, wherein the first device is used to execute the method of the first aspect or any one of the implementations of the first aspect or the method of the second aspect or any one of the implementations of the second aspect, so as to perform cross-device playback on the second device. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] FIG1 is a schematic diagram of a system to which an embodiment of the present application is applicable.

[0061] FIG2 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0062] FIG3 is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application.

[0063] FIG4 is a schematic flowchart of a cross-device playback method provided in an embodiment of the present application.

[0064] FIG5 is a schematic flowchart of a cross-device playback method provided in an embodiment of the present application.

[0065] FIG6 is a schematic diagram of transmitting resources in different encoding formats provided by an embodiment of the present application.

[0066] FIG7 is a schematic diagram of transmission resources when a device according to an embodiment of the present application supports or does not support multi-link playback.

[0067] FIG8 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

[0068] FIG9 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solution in this application will be described below with reference to the accompanying drawings.

[0070] It should be noted that, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In the calculation formula, " / " means division, for example, C / D means the value of C divided by D or the ratio of C to D. In addition, in the description of the embodiments of the present application, "plurality" refers to two or more than two, and "at least one" and "one or more" refer to one, two or more. The singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context.

[0071] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0072] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0073] FIG1 is a system applicable to an embodiment of the present application. As shown in FIG1 , the system 100 includes multiple electronic devices 11 to 14 and at least one access point 15. Four electronic devices are schematically shown in the system 100: a mobile phone 11, a smart screen 12, a laptop computer 13, a tablet computer 14, and an access point 15. The multiple electronic devices can be connected to a local area network through the access point 15 and interconnected through the local area network for cross-device playback. Users can play across devices through the local area network, such as playing media content such as videos, audio, photos, or files across devices, thereby obtaining a better playback experience.

[0074] When playing content across devices, the source device projects the content it wants to display onto the destination device. The source device and the destination device can be connected via wired or wireless means to interact with the content played across devices. When the source device and the destination device establish a connection wirelessly, the wireless communication protocol adopted may be a wireless fidelity (Wi-Fi) protocol, a Bluetooth protocol, a ZigBee protocol, a near field communication (NFC) protocol, a voice over Internet protocol (VoIP) protocol, etc.; it may also be various cellular network protocols, such as the global system for mobile communications (GSM) protocol, code division multiple access (CDMA) protocol, wideband code division multiple access (WCDMA) protocol, WCDMA), time-division code division multiple access (TD-SCDMA) protocol, general packet radio service (GPRS) protocol, long term evolution (LTE) protocol, world-wide interoperability for microwave access (WiMAX) protocol, fifth generation (5G) mobile communication protocol, etc., which is not specifically limited in the embodiments of the present application.

[0075] In the system 100 of Fig. 1, any at least one electronic device can be a source device, and any at least one electronic device can be a destination device. The source device can include but is not limited to a smart phone, a cellular phone, a wireless headset, a wearable computing device, a smart watch, a tablet computer, a personal digital assistant (PDA) or any device that can communicate with the destination device via connection. The destination device can include but is not limited to an in-vehicle entertainment information device, a television, a computer, a laptop computer, a projector, a camera, a smart phone, a wearable computing device or any other device that can communicate with the source device and display the content received from the source device. The source device and the destination device can be a single device or a combination of devices, for example, the destination device can include a display for displaying a video and a speaker for playing audio.

[0076] For example, taking the electronic device as a mobile phone as an example, FIG2 shows a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0077] As shown in Figure 2, the electronic device 200 may include: a processor 110, a memory 120, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a camera 191, a display screen 192, etc.

[0078] The software system of the electronic device 200 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 200 is exemplified.

[0079] Figure 3 shows a software structure diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 3, the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, The system is divided into four layers: application layer, application framework layer, system runtime layer (including system libraries and Android runtime), and kernel layer. Below the kernel layer is the hardware layer.

[0080] The application layer can include a series of application (APP) packages. As shown in Figure 3, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0081] In the embodiment of the present application, the application that the user enables to play across devices can be set at the application layer.

[0082] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 3, the application framework layer may include a window manager, content provider, telephony manager, resource manager, notification manager, activity manager, and view system.

[0083] In the embodiment of the present application, the mDNS service or other modules or units for managing cross-device playback can be set at this layer.

[0084] The system runtime library layer (libraries) can be divided into two parts: system libraries and Android runtime.

[0085] The Android runtime is the Android operating environment, consisting of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core libraries consist of two parts: one for Java-based functions and the other for the Android core library.

[0086] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0087] The system library supports the application framework and can include multiple functional modules, such as: surface manager, media libraries, 2D graphics engine (such as SGL), 3D graphics processing library (such as OpenGL ES), image processing library, etc.

[0088] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0089] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0090] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing and layer processing.

[0091] A 2D graphics engine is a drawing engine for 2D drawings.

[0092] The kernel layer is the layer between hardware and software, providing essential operating system functions such as file management, memory management, process management, and network protocol stacks. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, and Bluetooth drivers.

[0093] To fully leverage the advantages of different electronic devices, device interconnection technology based on the DLNA protocol has become widely used. However, cross-device playback using the DLNA protocol requires both the source and destination devices to support the protocol. DLNA only supports basic file formats, resulting in a poor user experience.

[0094] The embodiment of the present application provides a cross-device playback method that can support negotiation of device capabilities during cross-device playback, thereby providing users with a better cross-device playback experience. As shown in Figure 4, the method includes:

[0095] S410: The first device sends a query request, where the query request is used to request a cross-device playback service.

[0096] Correspondingly, the second device receives the query request.

[0097] Cross-device playback services can include services for playing images / documents, audio / video, and more. Cross-device playback that requires using the destination device's display is also known as screen casting. Cross-device media resources can be stored locally on the first device or in the cloud or on a server. These media resources can include video, audio, images, documents, and other resources.

[0098] The query request may be triggered based on the user's operation. For example, the first device may be a mobile terminal device such as a mobile phone or tablet computer. The first device may have a smaller screen, lower resolution, or poorer sound effects, so that when the user uses the first device to play media resources, he or she expects to project the media resources played by the first device to other devices for playback, thereby obtaining a better playback experience. When the user needs to cast the screen, taking the first device as a mobile phone as an example, the user can click on the control for casting the screen through the drop-down interface of the mobile phone, so that the first device sends a query request; or the user can click on the control for casting the screen in the application, such as the casting control on the video playback interface of a video application, so that the first device responds to the user's operation and sends a query request.

[0099] The query request may be sent via broadcast or multicast, for example, so that other devices in the same local area network as the first device can receive the query request. Of course, the query request is not limited to being received by devices in the same local area network as the first device. In some embodiments, other devices in a non-local area network as the first device can also receive the query request through a network device or node.

[0100] The query request may indicate the required service. For example, for a cross-device playback service, the query request may include a service identifier "_cast-remote._tcp.local", where the service identifier indicates that the requested service type is a cross-device playback service.

[0101] S420: The second device sends a query response to the first device, where the query response is used to indicate the first capability information of the second device.

[0102] Accordingly, the first device receives a query response from the second device.

[0103] In one possible implementation, the first capability information may include at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management capability information of the second device, and high dynamic lighting rendering technology capability information of the second device.

[0104] Before cross-device playback can be achieved between two devices, service discovery is required. This application mainly introduces the technical solution of the embodiment of this application by taking the use of the multicast domain name system (mDNS) protocol based on different devices for service discovery (DNS-SD) as an example during cross-device playback. In the embodiment of this application, other communication protocols can also be used for communication between devices, and this application does not limit this. DNS-SD is a protocol for obtaining information such as the type of services or functions supported by the device, communication ports, etc. The above S410 and S420 are the process of service discovery. The first device as the source device can obtain information about the second device that supports the cross-device playback service through the above steps.

[0105] The other device can receive the query request sent by the first device and, based on whether it supports the service indicated by the query request, choose whether to respond to the query request. In addition to the second device, other devices may also support cross-device playback services and thus send query responses to the first device. The following describes the query response details using the second device as an example.

[0106] When devices communicate using the mDNS protocol, each device includes an mDNS service, which can be a module in the device's operating system that manages cross-device playback services. When devices communicate using other protocols, they can also include a module or unit that manages cross-device playback.

[0107] Before generating a query response, the second device pre-registers the services it supports with its mDNS service. For example, the second device can register when joining the local area network. The registration information may include first capability information, as well as the device name, device type, device identifier, service identifier, port information, and second capability information. For different supported services, the second device may record different service identifiers, port information, first capability information, and second capability information. The first capability information may also be referred to as extended information, and the second capability information may also be referred to as feature information.

[0108] In one implementation, the device identifier, device type, first capability information, and second capability information may be stored in a text record (TXT record), and the service identifier may be used as a field for retrieval after the second device receives a query request. When the service identifier indicated by the query request is retrieved, the second device will continue to query the port information and text record corresponding to the service identifier, thereby generating a query response using part or all of the information in the text record.

[0109] The device name can be user-defined or system-defined, making it easy for users to distinguish different devices by device name. For example, the device name can be "Living Room TV," "Zhang San's Mobile Phone," or "A HDTV," where A can be the brand or manufacturer of the TV. When the first device obtains a list of devices that can provide the required service through device discovery, the device names of these devices that can provide the required service can be displayed in the device list.

[0110] The port information is a port for communication between the first device and the second device when the second device provides a corresponding service, and may be a port randomly generated when the second device registers and publishes a corresponding service and waits for connection authentication.

[0111] The information in the text record can be saved in the form of key-value pairs in JSON format. The following describes the content of the text record.

[0112] The device identifier indicates the second device, the device type indicates the type of the device, the second capability information indicates basic parameters of the service, and the first capability information indicates detailed parameters of the service.

[0113] The value of the JSON key-value pair corresponding to the device identifier can be represented by a string. The device identifier can be a numerical sequence or string that can represent the device identity, such as the device model, serial number, universally unique identifier (UUID), Internet Protocol (IP) address, or Media Access Control (MAC) address. For example, when using a UUID as the device identifier, the device identifier can be represented as: "DeviceID":"UUID".

[0114] The value of the JSON key-value pair corresponding to the device type can be represented by an integer. The integer can be stored in binary, quaternary, octal, hexadecimal, or binomial. The length of the integer can be set as needed. For example, the length of the integer can be 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. Different integers can map different device types. If you need to classify devices more finely, you can make the length of the integer longer, allowing the integer to represent more types of devices. Table 1 schematically shows the correspondence between a device type and the value of a JSON key-value pair, taking an integer length of 16 bits as an example (hexadecimal):

[0115] Table 1

[0116] Illustratively, when the device is a smart TV, the device identifier can be represented as: "DeviceType": "0x0004". The above table only schematically illustrates the relationship between one device type and its corresponding value. In addition to the device types listed in Table 1, devices can also have other types, or devices can be classified in other ways, which is not limited in this application.

[0117] In one possible implementation, the second capability information can be used to indicate at least one of the following features of the second device: whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirroring display capability, whether the screen of the second device supports the first resolution, whether the second device has the ability to play and render media resources at the first resolution, and whether the second device can access the network normally.

[0118] Whether the screen of the second device supports the first resolution may refer to whether the display screen of the second device reaches the first resolution.

[0119] Whether the screen of the second device supports the first resolution and whether the second device has the ability to play and render media resources at the first resolution can be combined into one feature. Alternatively, any two or more features in the above second capability information can be combined into one feature, which is not limited in this application.

[0120] Mirrored display means that the source device and the destination device display exactly the same screen interface. For example, common multi-screen collaboration is an example of mirrored display, which can copy the smaller screen interface of a mobile phone, such as a mobile phone, to a computer or tablet.

[0121] The first resolution can be any one of 1K, 2K, 4K, and 8K. The first resolution can also be other lower or higher values, which is not limited in this application. For example, the first resolution can be 4K, then the second capability information can indicate whether the screen of the device supports 4K and whether the second device has the ability to play and render 4K media resources; for example, the first resolution can be 8K, and the second capability information can indicate whether the screen of the device supports 8K and whether the device has the ability to play and render 8K media resources. The second capability information can also indicate the device's support information for multiple resolutions at the same time. For example, the second capability information indicates whether the screen of the device supports 4K and whether it has the ability to play and render 4K media resources, and also indicates whether the screen of the device supports 8K and whether it has the ability to play and render 8K media resources.

[0122] The value of each item in the second capability information can be represented by a binary integer. Schematically, the binary number "0" can be used to indicate that the second device does not have the corresponding capability, and the binary integer "1" can be used to indicate that the second device has the corresponding capability. Thus, each item in the second capability information can be represented by only one bit. When any two items in the second capability information serve as a feature, only one bit can be used to represent the information of the two features, which can reduce the overhead of information storage. For example, one bit is used to indicate whether the device's screen supports the first resolution and whether it has the ability to play and render media resources at the first resolution.

[0123] In one embodiment, all items in the second capability information can be displayed together, that is, the entire second capability information is represented by a json key-value pair, so that the value of the json key-value pair can be an integer with a certain length, such as a 16-bit, 32-bit, 64-bit integer, etc., and the bits at different positions after the integer is converted to binary represent different second capability information. Certain bits can also be reserved for subsequent indication of other second capability information. Schematically, the first bit indicates whether the second device supports video playback, the second bit indicates whether the second device supports audio playback, and the third bit indicates whether the second device supports picture display. If the second capability information is expressed as "features": "111", it means that the second device supports video display, audio playback and picture display, and features can be the key of the json key-value pair of the second capability information pair.

[0124] It should be understood that the second capability information may also indicate other information, and the correspondence between different items of the second capability information and the position is only given for reference. For example, the first bit may be used to indicate whether the screen of the second device supports 4K. Each item in the second capability information may also be represented separately using a json key-value pair, for example, "Video": "1" indicates that the second device supports video playback, and "Video": "0" indicates that the second device does not support video playback. This application does not limit this.

[0125] In one possible implementation, the first capability information may include at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management capability information of the second device, high dynamic lighting rendering technology capability information of the second device, screen resolution information of the second device, color gamut information of the second device, sound effect information of the second device, screen refresh rate information of the second device, device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device.

[0126] Decoding capability information can indicate which video, audio, and image encoding methods the device supports for decoding, or which video, audio, and image formats it supports. The value of the JSON key-value pair representing decoding capability information can be an integer, and the length of the integer can be set as needed, for example, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, and so on.

[0127] There are many encoding methods for video resources, such as the Moving Pictures Expert Group (MPEG) format, including MPEG-1, MPEG-2, MPEG-4 Part 2, MPEG-4 Part 10, MPEG-4 AVC, etc., as well as H.261, H.263, H.264, H.265, H.266, WMV9, VC-1, Audio Video Interleaved (AVI) and other formats, which are not limited here.

[0128] For audio resources, the encoding method may include, for example, pulse code modulation (PCM) encoding, (windows media audio, WMA) encoding or linear predictive coding (LPC). In addition, it may also include other encoding methods, such as code excited linear prediction (CELP) or advanced audio coding (AAC), which are not limited here.

[0129] For image resources, encoding methods may include Joint Photographic Experts Group (JPEG), Portable Network Graphics (Portable Network Graphics), Graphics Interchange Format (GIF), Tagged Image File Format (Tag Image File Format), Bitmap (BMP), and Scalable Vector Graphics (SVG).

[0130] Schematically, taking 16-bit hexadecimal representation as an example, the decoding capability information can be expressed as "CodecCap": "0x0002", and the decoding method corresponding to 0x0002 can be MPEG-2.

[0131] The multi-link playback capability information indicates whether the device supports multi-link seamless playback. If the device supports multi-link playback, it can allow the device to seamlessly play the media resources corresponding to multiple resource links, such as seamless playback of advertising resources and video resources. If the device does not support multi-link playback, if the media resources corresponding to multiple resource links are to be played, the media resources corresponding to the multiple links need to be spliced ​​and compressed on the server side, which increases the pressure on the server. The multiple resource links can include not only advertising resources and video resources, but also multiple video resources, which is not limited to this application.

[0132] Illustratively, the value of the JSON key-value pair representing the multi-link playback capability information can be represented by an integer, and the length of the integer can be set as needed, for example, the length of the integer can be 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. Illustratively, the multi-link playback capability can be represented by a 1-bit integer as: "MultiLink": "0", indicating that the device does not support multi-link playback capability.

[0133] Optionally, the length of the value of the JSON key-value pair for multi-link playback can also be extended. For example, a 2-bit integer can be used to divide the multi-link playback capability of the device into: integer "0" corresponds to no support for multi-link playback capability, integer "1" corresponds to the device supporting 2-link playback capability, integer "2" corresponds to the device supporting 3-link playback capability, integer "4" corresponds to the device supporting more than 3 links playback capability, etc.

[0134] The above information can be represented not only by extending the length of the value of the JSON key-value pair, but also by other information related to multi-link playback capabilities, which is not limited in this application.

[0135] Digital rights management (DRM) capability information indicates the DRM type and DRM client security level supported by the device. DRM provides secure distribution, rights control, and operational management of digital content. This allows stakeholders to define different usage rights for each piece of digital content. Users must obtain authorization to consume digital content according to the corresponding rights and pay the corresponding fees.

[0136] DRM types can include Open Mobile Alliance (OMA), Microsoft Windows Media Digital Rights Management (WMDRM), Digital Entertainment Content Ecosystem (DECE), ChinaDRM, Google DRM digital rights management system Widevine, Microsoft's content Playready content access and protection technology, Apple Digital music DRM copyright protection system FairPlay and other mechanisms.

[0137] The DRM client security level may include: software security level (SW), hardware security level (HW), or enhanced hardware security level (EH).

[0138] Schematically, the value of the json key-value pair representing the DRM capability information can be in the form of a string. Specifically, the value of the key-value pair representing the DRM type can be in the form of an integer, and the length of the integer can be set as needed. For example, the length of the integer can be 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. The value of the key-value pair representing the DRM client security level can be in the form of a string. For example, the DRM capability information can be expressed as: "DRMCap": "DRMSchema": "0000", "securityLevel": "HW", and the DRM type corresponding to "0000" can be DECE, indicating that the DRM type of the device is DEC, and the client security level is the hardware security level. Of course, the value of the DRM type key value can also be in the form of a string and the value of the json key-value pair representing the DRM client security level can also be in the form of an integer, or both can use string form or integer form, and this application does not limit this.

[0139] The high dynamic range (HDR) capability indicates the rendering capability of the device. The value of the JSON key-value pair representing the capability information can be an integer. The length of the integer can be set as needed. For example, the length of the integer can be 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. For example, "0x0001" indicates that the device has HDR vivid capability, "0x0002" indicates that the device has HDR10 capability, "0x0004" indicates that the device has HDR10+ capability, and "0x0008" indicates that the device has Dolby Vision capability... Other HDR capabilities can also be represented by other 16-bit numbers. Schematically, the high dynamic range capability can be expressed as: "HDRCap": "0x0001", indicating that the device has HDR vivid capability.

[0140] Optionally, the HDR capability information may also be represented in a string format.

[0141] Screen resolution information can indicate the highest resolution supported by the device's display. Common screen resolutions (expressed as horizontal pixels × vertical pixels) include: 1280×720, 1600×900, 1920×1080, 2560×1440, 3840×2160, and so on. The value of a JSON key-value pair representing screen resolution information can be in string format. For example, screen resolution information can be represented as "ScreenResolution":"1920×1080". Alternatively, the value of a JSON key-value pair representing screen resolution information can be in integer format. The length of the integer can be set as needed, for example, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, and so on. Illustratively, the 16-bit integer "0x0001" indicates that the device's maximum screen resolution is 1280×720, "0x0002" indicates that the device's maximum screen resolution is 1600×900, and "0x0003" indicates that the device's maximum screen resolution is 1920×1080...then the device's screen resolution is "ScreenResolution": "0x0001", indicating that its maximum screen resolution is 1280×720.

[0142] Color gamut information can indicate the color gamut supported by the device's screen. Key color gamut capabilities can include the Universal Color Standard (sRGB), the National Television System Committee (NTSC), DCI-P3 proposed by the Digital Cinema Initiatives, and Adobe RGB. The JSON key-value pairs representing color gamut information can be integers, and the length of the integers can be customized, for example, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, or 128 bits. For example, "0x0001" indicates support for sRGB, "0x0002" indicates support for NTSC, "0x0004" indicates support for DCI-P3, and "0x0008" indicates support for Adobe RGB. For example, color gamut information can be represented as "Gamut": "0x0008," indicating that the device supports Adobe RGB color gamut display.

[0143] Optionally, the value of the JSON key-value pair representing the color gamut information can also be in string format.

[0144] The sound effect information can indicate the sound effects supported by the device. The main sound effects may include: Huawei's Audio Vivid sound effect, Dolby sound effect, Huawei's HISTEN sound effect, etc. The value of the json key-value pair representing the sound effect information can be in the form of an integer. The length of the integer can be set as needed. For example, the length of the integer can be 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. For example, "0x0001" indicates support for Audio Vivid, "0x0002" indicates support for Dolby, and "0x0003" indicates support for HISTEN. Schematically, the sound effect information can be expressed as: "AudioEffect": "0x0002", indicating that the device supports Dolby sound effect.

[0145] Optionally, the value of the JSON key-value pair representing the sound effect information can also be in string format.

[0146] The screen refresh rate information can indicate the maximum refresh rate of the screen, and the refresh rate of the screen can include: 60Hz, 90Hz, 120Hz, 144Hz, 165Hz, etc. The value of the json key-value pair representing the screen refresh rate information can be an integer, and the length of the integer can be set as needed. For example, the length of the integer can be 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. Exemplarily, 60Hz can be represented by "0x0001", 90Hz can be represented by "0x0002", 120Hz can be represented by "0x0003", 144Hz can be represented by "0x0004", and 165Hz can be represented by "0x0005". Schematically, the screen refresh rate can be expressed as: "ScreenFPS": "0x0001", indicating that the maximum refresh rate of the device is 60Hz.

[0147] But it is true that the value of the json key-value pair representing the screen refresh rate information can also be in string form.

[0148] Device subtypes can be further categorized than device types. For example, each device type can be further categorized by manufacturer or brand. The JSON key-value pair representing a device subtype can be a string. For example, a device subtype might be represented as "DeviceSubType":"brandA." Device subtypes can also be categorized more finely than device types. For example, computers can be categorized as desktops, all-in-ones, and laptops.

[0149] A macroblock is the smallest unit of media encoding and decoding. The maximum number of macroblocks in a decoder can indicate the decoder's maximum macroblock processing rate. The value of the JSON key-value pair representing the decoder's maximum macroblock number device subtype can be a string. For example, the maximum number of macroblocks in a decoder can be expressed as "MacroBlock":"width×height×FPS", where width refers to the number of macroblocks in the width of the decoder resolution, height refers to the number of macroblocks in the height of the decoder resolution, and FPS indicates the screen refresh rate.

[0150] It should be understood that the various registration details described above can be stored in different ways. For example, the first capability information and the second capability information may not be represented separately but stored as a single capability information item, or the individual items of the capability information may not belong to the corresponding capability information item. For example, some or all items of the first capability information may be stored as items of the second capability information item.

[0151] After receiving the query request from the first device, the second device can search the service list supported by the device to see whether the device supports the service corresponding to the service identifier in the query request. If so, it continues to obtain other information about the service and generates a query response based on the information.

[0152] In an embodiment of the present application, the query response may indicate the first capability information of the second device. In one possible implementation, the first capability information may include at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management (DRM) capability information of the second device, and high dynamic range rendering technology (HDR) capability information of the second device.

[0153] Schematically, the query request may include a first capability information field, which may indicate the above-mentioned first capability information. In this way, after receiving the query response from the second device, the first device may obtain the capability information of the second device, thereby performing cross-device operations according to the corresponding capability information in subsequent cross-device operations. For example, based on the decoding capability information of the second device, a resource with a higher compression rate among the decoding types supported by the second device may be selected, thereby reducing resource transmission overhead; for another example, if the second device supports multi-link playback, multiple resource links may be sent to the second device, thereby reducing the overhead of the server merging and splicing multiple media resources. The server only needs to send the corresponding media resources of each of the multiple resource links to the second device according to the multiple resource links; for another example, based on the DRM capability information of the second device, a link indicating the DRM resource may be sent to the second device to avoid problems in DRM resource playback due to device non-support; for another example, based on the HDR capability information of the second device, the corresponding resource may be sent to the second device, thereby enabling the second device to play video resources with better effects and improving the user experience.

[0154] Taking the cross-device video playback to a TV as an example, Table 2 schematically shows examples of capability information of different devices.

[0155] Table 2

[0156] Assuming the second device is a high-definition encrypted TV, the first device can obtain the corresponding capability information through the query response and send 4K resources or resource links in H.265 format to the second device, thereby achieving a higher compression ratio and higher video resolution. This results in higher video playback quality and faster transmission speeds, improving the user experience. If the second device is a regular TV, the first device can send 1080P resources in H.264 format to the second device to ensure normal video playback.

[0157] In some embodiments, the first capability information may further include at least one of the following: screen resolution information of the second device, color gamut information of the second device, and sound effect information of the second device.

[0158] In this way, the first device can send resources or resource links with higher resolutions supported by the current network conditions to the second device based on factors such as the second device's screen resolution and current network conditions. It can also send resources or resource links that match the second device's color gamut information based on the second device's color gamut information. Furthermore, it can send resources or resource links with higher sound effects supported by the second device based on the second device's sound effect information. This allows users to enjoy a better cross-device experience within the capabilities of the second device, improving their user experience.

[0159] In some embodiments, the first capability information may further include at least one of the following: refresh rate information of the second device, a device subtype of the second device, and a maximum number of macroblocks of a decoder of the second device.

[0160] In this way, the first device can send corresponding resources or resource links to the second device according to the corresponding capability information of the second device, thereby improving the cross-device playback experience.

[0161] In some embodiments, the length of the first capability information field may be less than or equal to a certain threshold, for example, the threshold may be 400 bytes, 800 bytes, 200 bytes, etc., thereby reducing the overhead of the query response.

[0162] In an embodiment of the present application, the query response may also indicate second capability information of the second device, and the second capability information may indicate at least one of the following features of the second device: whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

[0163] The first resolution may be 1K, 2K, 4K, 8K, etc. Of course, the second capability information may be used to indicate the device's support for multiple resolutions, and this application does not limit this.

[0164] For example, the query request may include a second capability information field, and the second capability information field may indicate the above-mentioned second capability information.

[0165] In some embodiments, each item of the above-mentioned second capability information can be represented by one bit, and the position of each bit in the second capability information field can have a certain correspondence with different items of the second capability information. For example, the first bit of the second capability information field indicates whether the second device supports video playback, and the second bit indicates whether the second device supports audio playback, etc.

[0166] Of course, two or more items of the second capability information can be represented as one item using only one bit. For example, whether the screen of the second device supports the first resolution and whether the second device has the ability to play and render media resources at the first resolution can be represented by one bit. Only when both are met, the value of the bit is 1.

[0167] The length of the second capability information field can be less than a certain threshold, for example, the threshold can be 128 bits, 64 bits, 32 bits, etc. In one example, the length of the second capability information field can be 32 bits, and the number of characteristic items in the second capability information that need to be indicated can be less than 32, so that a certain number of bits can be reserved for subsequent addition of indications of other items of the second capability information.

[0168] In the embodiment of the present application, the query response may further indicate the device type and / or device identification of the second device.

[0169] In some embodiments, the query response may further include the device name, service identifier, and port configuration of the second device.

[0170] The device name of the second device can be displayed as, for example, "TV in the living room", "TV A", etc. after the source device displays the cross-device playlist. A can be the brand of the TV. The device name of the second device can be a name set by the user or a system default name.

[0171] For explanations of the above-mentioned second capability information, first capability information, device type, device identifier, service identifier and port configuration, please refer to the previous introduction to the second device registration and release process, which will not be repeated here.

[0172] It should be understood that the different capability information of the second device introduced above is set in the first capability information field and the second capability information field. In some embodiments, some items in the first capability information field may also be set in the second capability information field, and some items in the second capability information field may also be set in the first capability information field. This application does not limit this.

[0173] In some embodiments, the query response may carry all first capability information and second capability information registered by the second device for the service requested in the query request, so that the second device can directly send the first capability information as the first capability information field and the second capability information as the second capability information field to the first device in the query response. In other embodiments, the second device may only use part of the registered first capability information and second capability information. For example, based on the type of resources played across devices by the first device, only capability information related to the type may be carried. For example, if the user needs to play pictures across devices, it is not necessary to transmit information such as decoding capabilities related to audio or video.

[0174] After receiving the query response sent by the second device, the first device may send, to the second device, media resources or media resource links that match the capabilities of the second device.

[0175] In one embodiment, when the first capability information includes the decoding capability information of the second device, the first device may send a resource message to the second device based on the decoding capability information of the second device. The resource message may indicate a resource for cross-device playback, encoded using the highest compression type supported by the second device. The resource message may be a resource link or an audio or video resource, which reduces resource transmission overhead due to the highest compression rate.

[0176] Alternatively, the first device may select a resource type based on network conditions. If the network conditions are good, the resource message may indicate an audio or video resource with the lowest compression rate, thereby improving the user experience. If the network conditions are poor, a resource with a higher compression rate may be selected to reduce resource transmission delays.

[0177] In another embodiment, when the query response indicates that the second device supports multi-link playback, if there is a demand for advertising playback, for example, the user is an ordinary user, the first device can send multiple resource links to the second device, and the multiple resource links include resource links indicating advertisements and resource links indicating audio or video, so that the second device can obtain advertising resources and video resources (or audio resources) through the multiple links and play them in sequence, reducing the overhead generated by server splicing.

[0178] The embodiment of the present application allows content providers to provide content with different parameters based on capability information. For example, the content provider is a video application manufacturer, and users can subscribe to different levels of services, and different levels of services allow users to play videos of different resolutions. When the user is an ordinary user, he does not have the privilege of screen casting service and can only cast screen to watch videos with a maximum resolution of 720P. When the user performs screen casting operation through the first device, although the second device can support up to 2K video playback, the first device will send a 720P video resource connection to the second device based on the service content subscribed by the user. On the other hand, if the user is a member user, he has the privilege of screen casting service and can cast screen to watch resources with a maximum resolution of 4K. Since the second device only supports 2K video playback, the first device will send a 2K video resource link to the second device to avoid directly sending a 4K resource link and causing playback failure.

[0179] The embodiment of the present application may also allow the user to select different destination devices for cross-device playback based on capability information. After multiple devices including the second device return a query response to the first device, a list of devices that can be played across devices may be displayed on the interface of the first device, and the list may include multiple different destination devices. For different destination devices, partial capability information corresponding to each destination device may be displayed on the interface. For example, if a user wants to cast a 2K video, if a certain destination device does not support the playback of 2K videos, when the user wants to select the destination device for cross-device playback, the first device may prompt the user that selecting the destination device for casting the screen will reduce the playback quality, so that the user can also select a more suitable destination device (such as another destination device that supports the playback of 2K videos) to perform cross-device playback operations based on the capability information of different destination devices.

[0180] Taking the example of casting a video to a TV via a mobile phone, FIG5 shows a schematic flow chart of a cross-device playback method provided by an embodiment of the present application. As shown in FIG5 , the method includes:

[0181] S510: The TV registers its capability information.

[0182] In one possible implementation, the capability information may include first capability information of the TV, which may include at least one of the following: decoding capability information of the TV, multi-link playback capability information of the TV, digital rights management capability information of the TV, high dynamic range lighting rendering technology capability information of the TV, screen resolution information of the TV, color gamut information of the TV, sound effect information of the TV, screen refresh rate information of the TV, device subtype of the TV, and maximum number of macroblocks of the TV decoder. Optionally, the capability information may also include second capability information, which may indicate at least one of the following features of the TV: whether the TV supports video playback, whether the TV supports audio playback, whether the TV supports picture display, whether the TV supports mirroring display capability, whether the TV screen supports the first resolution, whether the TV has the ability to play and render media resources at the first resolution, and whether the TV can access the network normally.

[0183] S520: The mobile phone sends a query request.

[0184] In one possible implementation, the query request may be sent in a broadcast or multicast format, and the query request may carry an identifier of the screen projection service.

[0185] S530: The TV sends a query response to the mobile phone.

[0186] In one possible implementation, after parsing the query request, the TV can confirm that it supports the screen casting service, obtain the registration information, and send some or all of the registration information corresponding to the screen casting service to the mobile phone via a query response. The query response can be sent in unicast form.

[0187] S540: The mobile phone sends a video resource link to the TV.

[0188] In one possible implementation, for example, if the mobile phone parses the query response and confirms that the TV supports H.266 video resources, it can send a link to the H.266 video resource to the TV, so that the TV ultimately plays the H.266 video resource. For another example, if the mobile phone parses the query response and confirms that the TV only supports H.264 video resources, it can send a link to the H.264 video resource to the TV, so that the TV ultimately plays the H.264 video resource.

[0189] S550: The TV sends a video resource link to the server.

[0190] S560: The server sends the video resource to the TV.

[0191] Figure 6 shows a schematic diagram of transmitting video resources in H.264 and H.266 formats. Figure 6 (a) shows a schematic diagram of transmitting H.264 video resources, and Figure 6 (b) shows a schematic diagram of transmitting H.266 video resources. Taking the compression rate of H.264 video resources as 100% as an example, the compression rate of H.265 video resources is about 50%, and the compression rate of H.266 video resources is about 25%. When using the DLNA protocol for cross-device playback, H.264 format video is generally transmitted as shown in Figure 6 (a). However, the method of the embodiment of the present application can be used as shown in Figure 6 (b) to transmit H.266 video resources when the device supports H.266 format video. Since the query response returned by the destination device carries its capability information, it is used to indicate that the destination device can support the playback of videos with higher compression rates. As a result, the size of the resources transmitted by the server to the destination device can be reduced, especially when the network quality is poor, giving users a better playback experience.

[0192] Optionally, if the query response sent by the TV indicates whether the TV supports multi-link playback capability, the message transmission between the mobile phone, the TV and the server may be as shown in FIG. 7 .

[0193] As shown in Figure 7(a), when the destination device does not support multi-link transmission, the server can splice the advertising resources and video resources according to Figure 7(a) and return the video resources with the spliced ​​advertisements to the TV. In order to avoid playback errors when transmitting according to the existing DLNA protocol, the video resources are also processed in the manner shown in Figure 7(a), which puts a lot of pressure on the server. It can be understood that there are many types of advertising resources and video resources. The server needs to splice the specific advertising resources and specific video resources corresponding to each request, so the server is under great pressure.

[0194] When the device supports multi-link playback, the mobile phone can send advertising links and video links to the TV as shown in Figure 7 (b), and the server also returns advertising resources and video resources, so the server does not need to splice the video, reducing the processing pressure on the server.

[0195] An embodiment of the present application also provides a cross-device playback method, which is applied to a first device, including: receiving a first message from a second device, the first message including first capability information of the second device, the first capability information including at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management (DRM) capability information of the second device, and high dynamic range rendering technology (HDR) capability information of the second device; based on the first message, sending a resource message to the second device, the resource message being used to indicate media resources for cross-device playback, and the media resources being associated with the first capability information.

[0196] Accordingly, the second device sends the first message.

[0197] The first message may be sent at a certain period, or may be sent when the second device is powered on or accesses the network.

[0198] In this method, when the first device and the second device perform device discovery, the second device may send a first message, which may be a broadcast message or a multicast message. When the first device needs corresponding services, it may interconnect with the second device according to the first message, thereby enabling cross-device playback with the second device. The difference between this method and the aforementioned method is that the first device can directly obtain the first message sent by the second device through broadcast or multicast without first sending a query request. Compared with the embodiment shown in FIG5 , under this method, the mobile phone does not need to execute S520, and the TV directly informs nearby devices of the first capability information of the TV through broadcast or multicast, so that when the mobile phone is near the TV, the mobile phone can directly obtain the first capability information of the TV through the broadcast or multicast message sent by the TV.

[0199] Similar to the previous embodiment, the first capability information may further include at least one of the following: screen resolution information of the second device, color gamut information of the second device, and sound effect information of the second device.

[0200] Optionally, the first message is also used to indicate second capability information of the second device, and the second capability information is used to indicate at least one of the following characteristics of the second device: whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

[0201] Optionally, the first message further includes a second capability information field, where the second capability information field is used to indicate the second capability information.

[0202] Optionally, the length of the second capability information field is 32 bits, and each feature of the second capability information is indicated by one bit of the second capability information field.

[0203] Optionally, the first resolution is any one of 2K, 4K or 8K.

[0204] Optionally, the first message is used to indicate the device type of the second device, and the device type of the second device includes any one of the following: a mobile phone, a TV, a tablet computer, a computer, a screen projector, a projector, an electronic whiteboard, a display, a smart cockpit, a speaker, a network box, and a set-top box.

[0205] Optionally, the cross-device playback service is used for audio or video playback, the first capability information includes decoding capability information of the second device, and the media resources played across the device are encoded using the decoding type with the highest compression rate among the decoding types supported by the second device.

[0206] Optionally, the cross-device playback service is used for audio or video playback, the first message is used to indicate that the second device has multi-link playback capability, the resource message includes multiple resource links, and at least one resource link among the multiple resource links indicates a media resource for cross-device playback.

[0207] Optionally, at least one resource link among the multiple resource connections indicates an advertising resource.

[0208] Optionally, the protocol used for communication between the first device and the second device is a multicast domain name system (mDNS) protocol.

[0209] Optionally, the first capability information includes at least one of the refresh rate information of the second device, the device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device, where the device subtype of the second device is related to the manufacturer or brand of the second device.

[0210] Optionally, the first message is used to indicate the device identification of the second device, and the device identification includes any one of the following: the Internet Protocol IP address of the second device, the Media Access Control MAC address of the second device, the Universally Unique Identifier UUID of the second device, and the serial number of the second device.

[0211] Optionally, the first message includes a first capability information field, where the first capability information field is used to indicate the first capability information, and the length of the first capability information field is less than 400 bytes.

[0212] Optionally, the first message includes the device name, service identifier, and port configuration of the second device.

[0213] For details of the first message, please refer to the details of the query response in the previous text. For other details, please refer to the description of the corresponding embodiment, which will not be repeated here.

[0214] An embodiment of the present application further provides an electronic device 800 , which includes a sending unit 810 and a receiving unit 820 .

[0215] In one embodiment, the electronic device 800 is used to execute the action previously performed by the first device, and the sending unit 810 is used to send a query request, which is used to request a cross-device playback service; the receiving unit 820 is used to receive a query response from the second device, which is used to indicate the first capability information of the second device, and the first capability information includes at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management DRM capability information of the second device, and high dynamic range rendering technology HDR capability information of the second device.

[0216] Optionally, the sending unit 810 is further used to: send a resource message to the second device according to the first capability information, where the resource message is used to indicate a media resource of the cross-device playback service, and the media resource is associated with the first capability information.

[0217] Optionally, the first capability information includes at least one of the following: screen resolution information of the second device, color gamut information of the second device, and sound effect information of the second device.

[0218] Optionally, the query response is used to indicate second capability information of the second device, and the second capability information is used to indicate at least one of the following characteristics of the second device: whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

[0219] Optionally, the query response includes a second capability information field, where the second capability information field is used to indicate the second capability information.

[0220] Optionally, the length of the second capability information field is 32 bits, and each item of the second capability information is indicated by one bit of the second capability information field.

[0221] Optionally, the first resolution is any one of 2K, 4K or 8K.

[0222] Optionally, the query response is used to indicate the device type of the second device, and the device type of the second device includes any one of the following: a mobile phone, a TV, a tablet computer, a computer, a screen caster, a projector, an electronic whiteboard, a display, a smart cockpit, a speaker, a network box, and a set-top box.

[0223] Optionally, the cross-device playback service is used for audio or video playback, the first capability information includes decoding capability information of the second device, and the media resources played across the device are encoded using the decoding type with the highest compression rate among the decoding types supported by the second device.

[0224] Optionally, the cross-device playback service is used for audio or video playback, the query response is used to indicate that the second device has multi-link playback capability, the resource message includes multiple resource links, and at least one resource link among the multiple resource links indicates a media resource for cross-device playback.

[0225] Optionally, at least one resource link among the multiple resource links indicates an advertising resource.

[0226] Optionally, the protocol used for communication between the first device and the second device is a multicast domain name system (mDNS) protocol.

[0227] Optionally, the first capability information includes at least one of the refresh rate information of the second device, the device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device, where the device subtype of the second device is related to the manufacturer or brand of the second device.

[0228] Optionally, the query response is used to indicate the device identification of the second device, and the device identification includes any one of the following: the Internet Protocol IP address of the second device, the Media Access Control MAC address of the second device, the Universally Unique Identifier UUID of the second device, and the serial number of the second device.

[0229] Optionally, the query response includes a first capability information field, where the first capability information field is used to indicate the first capability information, and the length of the first capability information field is less than 400 bytes.

[0230] Optionally, the query response includes the device name, service identifier, and port configuration of the second device.

[0231] In another embodiment, the electronic device 800 is equivalent to the second device described above and executes the action performed by the second device. A receiving unit 820 is configured to receive a query request for a cross-device playback service; a sending unit 810 is configured to send a query response indicating first capability information of the second device, the first capability information including at least one of the following: decoding capability information of the second device, multi-link playback capability information of the second device, digital rights management capability information of the second device, and high dynamic range rendering capability information of the second device.

[0232] Optionally, the receiving unit 820 is further used to: receive a resource message, where the resource message is used to indicate a media resource of a cross-device playback service, and the media resource is associated with the first capability information.

[0233] Optionally, the first capability information includes at least one of the following: screen resolution information of the second device, color gamut information of the second device, and sound effect information of the second device.

[0234] Optionally, the query response is used to indicate second capability information of the second device, and the second capability information is used to indicate at least one of the following characteristics of the second device: whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports image display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

[0235] Optionally, the query response includes a second capability information field, where the second capability information field is used to indicate the second capability information.

[0236] Optionally, the length of the second capability information field is 32 bits, and each feature of the second capability information is indicated by one bit of the second capability information field.

[0237] Optionally, the first resolution is any one of 2K, 4K or 8K.

[0238] Optionally, the query response is used to indicate the device type of the second device, and the device type of the second device includes any one of the following: a mobile phone, a TV, a tablet computer, a computer, a screen caster, a projector, an electronic whiteboard, a display, a smart cockpit, a speaker, a network box, and a set-top box.

[0239] Optionally, the cross-device playback service is used for audio or video playback, the first capability information includes decoding capability information of the second device, and the media resources played across the device are encoded using the decoding type with the highest compression rate among the decoding types supported by the second device.

[0240] Optionally, the cross-device playback service is used for audio or video playback, the query response is used to indicate that the second device has multi-link playback capability, the resource message includes multiple resource links, and at least one resource link among the multiple resource links indicates a video resource for cross-device playback.

[0241] Optionally, at least one resource link among the multiple resource links indicates an advertising resource.

[0242] Optionally, the protocol used by the second device for communication is the multicast domain name system (mDNS) protocol.

[0243] Optionally, the first capability information includes at least one of the refresh rate information of the second device, the device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device, where the device subtype of the second device is related to the manufacturer or brand of the second device.

[0244] Optionally, the query response is used to indicate the device identification of the second device, and the device identification includes any one of the following: the Internet Protocol IP address of the second device, the Media Access Control MAC address of the second device, the Universally Unique Identifier UUID of the second device, and the serial number of the second device.

[0245] Optionally, the query response includes a first capability information field, where the first capability information field is used to indicate the first capability information, and the length of the first capability information field is less than 400 bytes.

[0246] Optionally, the query response includes the device name, service identifier, and port configuration of the second device.

[0247] Figure 9 shows a schematic block diagram of an electronic device provided by an embodiment of the present application. The electronic device 900 shown in Figure 9 includes a memory 910, a processor 920, a communication interface 930, and a bus 940. The memory 910, processor 920, and communication interface 930 are connected to each other via the bus 940.

[0248] The memory 910 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 910 may store programs. When the program stored in the memory 910 is executed by the processor 920, the processor 920 is configured to execute the various steps of the cross-device playback method according to the embodiment of the present application.

[0249] The processor 920 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to perform the cross-device playback method of the embodiment of the present application.

[0250] The processor 920 may also be an integrated circuit chip with signal processing capabilities. During implementation, the various steps of the cross-device playback method of the present application may be completed by hardware integrated logic circuits or software instructions in the processor 920. The processor 920 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit, a field programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory 910. The processor 920 reads the information in the memory 910 and, in conjunction with its hardware, executes the cross-device playback method of the embodiments of the present application.

[0251] The communication interface 930 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the electronic device 900 and other devices or a communication network.

[0252] The bus 940 may include a path for transmitting information between various components of the electronic device 900 (eg, the memory 910 , the processor 920 , and the communication interface 940 ).

[0253] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0254] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0255] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0256] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0257] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0258] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0259] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A cross-device playback method, characterized in that: For a first device, comprising: Sending a query request, where the query request is used to request a cross-device playback service; A query response is received from a second device, where the query response is used to indicate first capability information of the second device, where the first capability information includes at least one of the following: The decoding capability information of the second device, the multi-link playback capability information of the second device, the digital rights management (DRM) capability information of the second device, and the high dynamic range rendering technology (HDR) capability information of the second device.

2. The method according to claim 1, characterized in that The method further comprises: A resource message is sent to the second device according to the first capability information, where the resource message is used to indicate a media resource of the cross-device playback service, where the media resource is associated with the first capability information.

3. The method according to claim 1 or 2, characterized in that: The first capability information also includes at least one of the following: The screen resolution information of the second device, the color gamut information of the second device, and the sound effect information of the second device.

4. The method according to any one of claims 1 to 3, characterized in that The query response is used to indicate second capability information of the second device, where the second capability information is used to indicate at least one of the following characteristics of the second device: Whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports picture display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

5. The method according to claim 4, characterized in that The query response includes a second capability information field, where the second capability information field is used to indicate the second capability information.

6. The method according to claim 5, characterized in that The length of the second capability information field is 32 bits, and each feature of the second capability information is indicated by one bit of the second capability information field.

7. The method according to any one of claims 4 to 6, characterized in that The first resolution is any one of 2K, 4K or 8K.

8. The method according to any one of claims 1 to 7, characterized in that The query response is used to indicate the device type of the second device, and the device type of the second device includes any one of the following: a mobile phone, a TV, a tablet computer, a computer, a screen projector, a projector, an electronic whiteboard, a display, a smart cockpit, a speaker, a network box, and a set-top box.

9. The method according to claim 2, characterized in that: The cross-device playback service is used for audio or video playback, the first capability information includes decoding capability information of the second device, and the media resources for cross-device playback are encoded using the type with the highest compression rate among the decoding types supported by the second device.

10. The method according to claim 2, characterized in that The cross-device playback service is used for audio or video playback, the query response is used to indicate that the second device has multi-link playback capability, the resource message includes multiple resource links, and at least one resource link among the multiple resource links indicates the media resource for cross-device playback.

11. The method according to claim 10, characterized in that At least one resource link among the plurality of resource connections indicates an advertising resource.

12. The method according to any one of claims 1 to 11, characterized in that The protocol used by the first device and the second device for communication is the multicast domain name system mDNS protocol.

13. The method according to any one of claims 1 to 12, characterized in that The first capability information includes at least one of the refresh rate information of the second device, the device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device. The device subtype of the second device is related to the manufacturer or brand of the second device.

14. A cross-device playback method, characterized in that: For a second device, comprising: receiving a query request, wherein the query request is used to request a cross-device playback service; Send a query response, where the query response is used to indicate first capability information of the second device, where the first capability information includes at least one of the following: The decoding capability information of the second device, the multi-link playback capability information of the second device, the digital rights management capability information of the second device, and the high dynamic range lighting rendering technology capability information of the second device.

15. The method according to claim 14, characterized in that The method further comprises: A resource message is received, where the resource message is used to indicate a media resource of the cross-device playback service, where the media resource is associated with the first capability information.

16. The method according to claim 14 or 15, characterized in that The first capability information includes at least one of the following: The screen resolution information of the second device, the color gamut information of the second device, and the sound effect information of the second device.

17. The method according to any one of claims 14 to 16, characterized in that The query response is used to indicate second capability information of the second device, where the second capability information is used to indicate at least one of the following characteristics of the second device: Whether the second device supports video playback, whether the second device supports audio playback, whether the second device supports picture display, whether the second device supports mirror display, whether the screen of the second device supports the first resolution, and whether the second device has the ability to play and render media resources at the first resolution.

18. The method according to claim 17, characterized in that The query response includes a second capability information field, where the second capability information field is used to indicate the second capability information.

19. The method according to claim 18, characterized in that The length of the second capability information field is 32 bits, and each feature of the second capability information is indicated by one bit of the second capability information field.

20. The method according to any one of claims 17 to 19, characterized in that The first resolution is any one of 2K, 4K or 8K.

21. The method according to any one of claims 14 to 20, characterized in that The query response is used to indicate the device type of the second device, and the device type of the second device includes any one of the following: a mobile phone, a TV, a tablet computer, a computer, a screen projector, a projector, an electronic whiteboard, a display, a smart cockpit, a speaker, a network box, and a set-top box.

22. The method according to claim 15, characterized in that The cross-device playback service is used for audio or video playback, the first capability information includes decoding capability information of the second device, and the media resources for cross-device playback are encoded using the type with the highest compression rate among the decoding types supported by the second device.

23. The method according to claim 15, characterized in that The cross-device playback service is used for audio or video playback, the query response is used to indicate that the second device has multi-link playback capability, the resource message includes multiple resource links, and at least one resource link among the multiple resource links indicates the media resource for cross-device playback.

24. The method according to claim 23, characterized in that At least one resource link among the plurality of resource connections indicates an advertising resource.

25. The method according to any one of claims 14 to 24, characterized in that The protocol used by the second device for communication is the multicast domain name system mDNS protocol.

26. The method according to any one of claims 14 to 25, characterized in that The first capability information includes at least one of the refresh rate information of the second device, the device subtype of the second device, and the maximum number of macroblocks of the decoder of the second device. The device subtype of the second device is related to the manufacturer or brand of the second device.

27. An electronic device, characterized in that: include: one or more processors; one or more memories; The one or more memories store one or more computer programs, which include instructions. When the instructions are executed by the one or more processors, the electronic device performs the method as described in any one of claims 1 to 13 or 14 to 26.

28. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 13 or 14 to 26.

29. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 1 to 13 or 14 to 26.

30. A cross-device playback system, characterized in that: It includes a first device and a second device, wherein the first device is connected to the second device for screen casting, wherein the first device is used to execute the method described in any one of claims 1 to 13 or 14 to 26 to perform cross-device playback on the second device.

Images